Subjects: Computer Science >> Computer Software submitted time 2024-04-23
Abstract： We propose a method to guide Large Language Models (LLMs) in generating structured content adhering to specific conventions without fine-tuning. By utilizing coroutine-based content generation constraints through a pre-agreed context-free grammar (CFG), LLMs are directed during decoding to produce formal language compliant outputs. This enhances stability and consistency in generating target data structures, types, or instructions, reducing application development complexities. Experimentally, error rates of GPT-2 and Gemma exceed 95% for DSLs longer than 36 and 282 tokens, respectively. We introduce YieldLang, a coroutine-based DSL generation framework, and evaluate it with LLMs on various tasks including JSON and Mermaid flowchart generation. Compared to benchmarks, our approach improves accuracy by 1.09 to 11.6 times, with LLMs requiring only about 16.5% of the samples to generate JSON effectively. This enhances usability of LLM-generated content for computer programs.
Peer Review Status:
Awaiting Review
Subjects: Computer Science >> Information Security Subjects: Computer Science >> Computer Application Technology submitted time 2024-04-23
Abstract： Image steganography has become a focal point of interest for researchers due to its capacity for the covert transmission of sensitive data. Traditional diffusion models often struggle with image steganography tasks involving paired data, as their core principle of gradually removing noise is not directly suited for maintaining the correspondence between carrier and secret information. To address this challenge, this paper conducts an in-depth analysis of the principles behind diffusion models and proposes a novel framework for an image steganography diffusion model. The study begins by mathematically representing the steganography tasks of paired images, introducing two optimization objectives: minimizing the secrecy leakage function and embedding distortion function. Subsequently, it identifies three key issues that need to be addressed in paired image steganography tasks and, through specific constraint mechanisms and optimization strategies, enables the diffusion model to effectively handle paired data. This enhances the quality of the generated stego-images and resolves issues such as image clarity. Finally, on public datasets like CelebA, the proposed model is compared with existing generation model-based image steganography techniques, analyzing its implementation effects and performance parameters. Experimental results indicate that, compared to current technologies, the model framework proposed in this study not only improves image quality but also achieves significant enhancements in multiple performance metrics, including the imperceptibility and anti-detection capabilities of the images. Specifically, the PSNR of its stego-images reaches 93.14dB, and the extracted images’ PSNR reaches 91.23dB, an approximate improvement of 30% over existing technologies; the attack success rate is reduced to 2.4x10-38. These experimental outcomes validate the efficacy and superiority of the method in image steganography tasks.
Peer Review Status:Awaiting Review
Subjects: Computer Science >> Computer Software Subjects: Linguistics and Applied Linguistics >> Linguistics and Applied Linguistics submitted time 2024-04-21
Abstract： This slide presentation describes the research on Guiding Large Language Models to Generate Computer-Parsable Content in terms of Background, Motivation, Method, Effect, Prospect and Acknowledgements. For the full paper, please refer to: https://arxiv.org/abs/2404.05499
Peer Review Status:Awaiting Review
Subjects: Psychology >> Applied Psychology Subjects: Computer Science >> Computer Application Technology submitted time 2024-04-10
Abstract： Integrative complexity is a concept used in psychology to measure the structure of an individual’s thinking in two aspects: differentiation and integration. The measurement of integrative complexity relies primarily on manual analysis of textual content, which can be written materials, speeches, interview transcript large language models, or any other form of oral or written expression. To solve the problems of high cost of manual assessment methods, low accuracy of automated assessment methods, and the lack of Chinese text assessment scheme, this study designed an automated assessment scheme for integrative complexity on Chinese and English texts. We utilized text data enhancement technique of the large language model and the model migration technique for the assessment of integrative complexity, and explored the automated assessment methods for the two sub-structures of integrative complexity, namely, the fine integration complexity and the dialectical integration complexity. In this paper, two studies are designed and implemented. Firstly, a prediction model for the integration complexity of English text is implemented based on the text data enhancement technology of large language model; secondly, a prediction model for the integration complexity of Chinese text is implemented based on the model transfer technology. The results showed that: 1) We used GPT-3.5-Tubo for English text data enhancement, a pre-trained multilingual Roberta model for word vector extraction, and a text convolutional neural network model as a downstream model. The Spearman correlation coefficient between this model’s prediction of integration complexity and the manual scoring results was 0.62, with a dialectical integration complexity correlation coefficient of 0.51 and a fine integration complexity Spearman correlation coefficient of 0.60. It is superior to machine learning methods and neural network models without data enhancement. 2) In Study 2, a model with the same structure as the neural network in Study 1 was established, and the final model parameters in Study 1 were also transferred to the model in this study to train the integration complexity prediction model based on Chinese text. In the case of zero samples, the Spearman correlation coefficients of the transfer learning model for integrative complexity are 0.31, the Spearman correlation coefficient of dialectical integration complexity is 0.31, and the correlation coefficient of fine integration complexity is 0.33, all of which are better than the model in the case of random parameters (integrative complexity: 0.17, dialectical integrative complexity: 0.10, fine integrative complexity: 0.10). In the case of small samples, the Spearman correlation coefficient of the transfer learning model was 0.73, with a dialectical integration complexity correlation coefficient of 0.51 and a fine integration complexity correlation coefficient of 0.73.
Peer Review Status:Awaiting Review
Subjects: Computer Science >> Computer Application Technology submitted time 2024-04-10
Exploration of the Integration and Application of Large Model and Standard Literature Knowledge Base
Abstract： In the context of artificial intelligence and big data technology, the use of large models and the construction of standard literature knowledge bases are of great value for scientific research innovation, knowledge mining, and information retrieval. The standard literature knowledge base provides solid support for the standardization and standardization of various industries. This study first explores the current status of standard literature, then builds a framework for integrating large models and standard literature knowledge bases based on retrieval enhancement, and proposes exploration of enhancement optimization in each stage. Finally, it looks forward to future research directions and application prospects.
Peer Review Status:Awaiting Review
Subjects: Psychology >> Applied Psychology Subjects: Computer Science >> Computer Application Technology submitted time 2024-04-09
Abstract： In recent years, the Linguistic Inquiry and Word Count (LIWC) tool has garnered increasing attention, offering the promise of objective, automated, and transparent psychological text analysis. This resurgence has reignited enthusiasm among psychologists for language analysis research. The recent revision of the LIWC-22 dictionary has introduced numerous variables aimed at assessing various socio-psychological structures, thus expanding the application potential of the LIWC tool. To further promote the cultural adaptation of the LIWC tool, we have revised and validated the Simplified Chinese Linguistic Inquiry and Word Count Dictionary 2024 (SCLIWC2024) to better align with the features of LIWC-22. In Study One, building upon the SCLIWC dictionary, we revised SCLIWC2024 by comparing it with the LIWC-22 and CLIWC2015 dictionaries. In Study Two, we conducted two experiments to validate the efficacy of SCLIWC2024 in detecting different psychological semantics in online texts, addressing crucial questions regarding how to more effectively utilize SCLIWC2024 for detecting the psychological semantics of short texts on social networking platforms.
Peer Review Status:Awaiting Review
Subjects: Computer Science >> Computer Application Technology submitted time 2024-04-07
Abstract： Physical function monitoring (PFM) plays a crucial role in healthcare especially for the elderly. Traditional assessment methods such as the Short Physical Performance Battery (SPPB) have failed to capture the full dynamic characteristics of physical function. Wearable sensors such as smart wristbands offer a promising solution to this issue. However, challenges exist, such as the computational complexity of machine learning methods and inadequate information capture. This paper proposes a multi-modal PFM framework based on an improved TimeMAE, which compresses time-series data into a low-dimensional latent space and integrates a self-enhanced attention module. This framework achieves effective monitoring of physical health, providing a solution for real-time and personalized assessment. The method is validated using the NHATS dataset, and the results demonstrate an accuracy of 70.6% and an AUC of 82.20%, surpassing other state-of-the-art time-series classification models.
Peer Review Status:Awaiting Review
Subjects: Computer Science >> Computer Software Subjects: Linguistics and Applied Linguistics >> Linguistics and Applied Linguistics submitted time 2024-04-07
Abstract： Large language models (LLMs) have demonstrated remarkable capabilities in learning patterns from massive text corpora, including word relationships, sentence structures, and even complex semantic and pragmatic information. However, it remains challenging to induce pre-trained language models to generate structured content that strictly follows specific conventions.We propose a scheme for guiding LLMs to generate highly usable content for computers without the need for fine-tuning and additional neural network inference, by introducing coroutine-based content generation constraints through a pre-agreed context-free grammar (CFG), which guides the autoregressive model Transformer to sample the correct tokens during its decoding phase to form a program-compliant form in the decoding phase of the autoregressive model Transformer to form a formal language that conforms to the program conventions. This will effectively improve the stability and consistency of LLMs in generating target data structures, types or instructions, and reduce the difficulty of application development and integration.We first verified that the error rate of models such as GPT-2 and Gemma reaches 95% when the length of the generated DSLs are greater than 36 and 282, respectively, through the experiment of matching bracket pairs , which illustrates the performance problem of some current LLMs in the generation of specific DSLs. We also present YieldLang, a coroutine-based DSL generation framework, and conduct experiments using LLMs on multiple task datasets, including tasks such as JSON, Mermaid flowchart, and function call expression generation. These experiments show that the approach in this paper improves its accuracy by a factor of 1.09 to 11.6 compared to the benchmarks, and in the best case is able to reduce the number of samples used by the LLMs to generate JSON to about 16.5% of the benchmarks, which will effectively improve the usability of the content generated by the LLMs for computer programs.
Peer Review Status:Awaiting Review
Subjects: Computer Science >> Computer Application Technology submitted time 2024-04-05
Abstract： The rapid development of 3D acquisition technology has made it possible to obtain point clouds of real-world terrains. However, due to limitations in sensor acquisition technology or specific requirements, point clouds often contain defects such as holes with missing data. Inpainting algorithms are widely used to patch these holes. However, existing traditional inpainting algorithms rely on precise hole boundaries, which limits their ability to handle cases where the boundaries are not well-defined. On the other hand, learning-based completion methods often prioritize reconstructing the entire point cloud instead of solely focusing on hole filling. Based on the fact that real-world terrain exhibits both global smoothness and rich local detail, we propose a novel representation for terrain point clouds. This representation can help to repair the holes without clear boundaries. Specifically, it decomposes terrains into low-frequency and high-frequency components, which are represented by B-spline surfaces and relative height maps respectively. In this way, the terrain point cloud inpainting problem is transformed into a B-spline surface fitting and 2D image inpainting problem. By solving the two problems, the highly complex and irregular holes on the terrain point clouds can be well-filled, which not only satisfies the global terrain undulation but also exhibits rich geometric details. The experimental results also demonstrate the effectiveness of our method.
Peer Review Status:Awaiting Review
Subjects: Computer Science >> Computer Application Technology submitted time 2024-04-04
Abstract： Text analysis and processing have become increasingly important topics, and there are many examples of Chinese word segmentation in jieba. However, there is little research on Japanese language word segmentation. This article aims to introduce the MeCab library’s function of segmenting Japanese words in Python, and provide relevant case codes to implement Japanese word segmentation as needed.
Peer Review Status:Awaiting Review
Subjects: Information Science and Systems Science >> Basic Disciplines of Information Science and Systems Science Subjects: Computer Science >> Other Disciplines of Computer Science Subjects: Engineering and technical science >> Engineering Mathematics submitted time 2024-03-30
Abstract： What is intelligence? is one of the core key questions of artificial intelligence, but there is no universally accepted definition. Based on the relationship between intelligence and life, this paper proposes that intelligence is the basic ability and characteristic attribute of living organisms, and it is the ability to achieve the maximum amount of information with the minimum energy as much as possible, and adapt to the environment and maintain existence through information processing. On this basis, this paper puts forward a new view that intelligence is the ability to convert material energy and information, further puts forward new concepts such as the measurement calculation method of intelligence, average intelligence, and comprehensive intelligence, and finally discusses the quantitative conversion relationship between matter, energy and information, points out the upper bound of intelligence and the lower bound of energy conversion into information, and further gives a dimensionless calculation formula for intelligence measurement in order to facilitate practical application. A feasible calculation method is given for the quantitative analysis of the intelligence of the intelligent system..
Peer Review Status:Awaiting Review
Subjects: Psychology >> Applied Psychology Subjects: Computer Science >> Computer Application Technology submitted time 2024-03-23
Abstract： In recent years, researchers have recognized the impact of Zhong-yong Thinking Style on mental health. However, it is not clear how Zhong-yong thinking style affects mental health through internal psychological mechanisms. Previous studies found that individuals with a better ability to coordinate agency (a motivation representing self-interest) and communion (a motivation representing altruism) tend to have a higher level of moral centrality. Moral centrality reflects the balance of internal motivation system, which can reduce the conflict between agency and communion, helping individuals reach a state that the opposing motivations support and energies each other. Moral centrality may play a potential mediating role in the impact of Zhong-yong thinking style on mental health. Although there are relatively mature methods for measuring individual moral centrality, it involves the complex task of coding values in personal strivings, making the measurement of moral centrality particularly complicated and labor-intensive. However, with the development of large language models(LLM) like ChatGPT, they have demonstrated excellent contextual comprehension skills and offered new possibilities for text analysis and coding work. Accordingly, this study intends to apply large language models to the coding work of psychological research, reduce the time and labor cost required in the process of measuring individual moral centrality, and explore how Zhong-yong thinking style affects individual mental health through moral centrality. Study 1 involves training GPT-3.5 Turbo to recognize values contained in personal strivings (achievement / power / universalism / benevolence) using differentiated prompts and evaluating its accuracy, precision, and recall rates, in order to obtain a model that meets the requirements for application. Study 2 applies above GPT-3.5 Turbo models in the process of measuring moral centrality, exploring how moral centrality mediates the impact of Zhong-yong thinking style on depression and anxiety. The findings are as follows: (1) The GPT-3.5 Turbo demonstrated an accuracy rate of not less than 0.80 in recognizing values of power, achievement, universlaism, and benevolence, showing the potential application of ChatGPT in psychological research; (2) Moral centrality played a mediating role in the impact of Zhong-yong thinking style on depression/anxiety. Specifically, individuals with a higher level of Zhong-yong thinking style could better integrate agency and communion, enhancing their moral centrality, and thereby reducing levels of depression/anxiety. In summary, this study utilized large language models to break through the technical limitations of traditional psychological research, exploring the mechanisms through which Zhong-yong thinking style affects mental health and verifying the mediating role of moral centrality. On the one hand, it proves the application potential of large language models in the field of psychological research. On the other hand, it deepens our understanding of the mechanisms through which Zhong-yong thinking style influence mental health, enriching the theoretical foundation of this field. It suggests that policymakers could use the advantages of Zhongyong thinking culture, advocating for values that emphasize individual development while also focusing on collective well-being, helping people improve moral centrality, thereby mitigating the negative impact of economic inequality on mental health.
Peer Review Status:Awaiting Review
Subjects: Psychology >> Applied Psychology Subjects: Computer Science >> Computer Application Technology submitted time 2024-03-23
Abstract： In recent years, researchers have increasingly recognized the impact of unequal income distribution on individual mental health. However, it is not clear how it affects mental health through internal psychological mechanisms. As the macro environment in which individuals live, economy shape people’s different values and make individuals have different levels of motivation orientation. Previous studies have indicated that individuals with a better ability to coordinate agency and communion tend to have a relatively high level of moral centrality. Moral centrality reflects the balance of internal motivation system, which can reduce the conflict between agency and communion, helping individuals reach a state that the opposing motivations support and energies each other. Thus, individuals are not only able to efficiently realize their personal values but also more easily allow for the attainment of eudaimonic well-being, thereby reducing the risk of mental health problems. Therefore, moral centrality may play a potential mediating role in the impact of income distribution inequality on mental health. Overall, with income distribution inequality as independent variables, this study aims to explore the mechanisms through which it affects mental health, by examining how income distribution influences individual moral centrality and, in turn, affect mental health. Our research not only enriches the theoretical foundation of the mental health field, but also provides a theoretical basis for interventions, and helps to formulate targeted strategies to improve the psychological well-being of the public. With the help of social media big data and natural language processing technology, we use posts made by regional microblogs to extract word frequency features representing the group’s moral centrality and group’s mental health level through the psychosemantic lexicon, and use panel data analysis to examine how the inequality in income distribution affects the negative emotions and suicide risk of the regional group through moral centrality. The results confirm that moral centrality plays a mediating role in the effect of regional income distribution inequality on group negative emotions/suicide risk, and that regions with higher income distribution inequality tend to be accompanied by lower levels of group moral centrality, which in turn leads to an increase in negative emotions/suicide risk among groups in the region.
Peer Review Status:Awaiting Review
Subjects: Computer Science >> Integration Theory of Computer Science submitted time 2024-03-16
Abstract： A novel federated learning training framework for heterogeneous environments is presented, taking into account the diverse network speeds of clients in realistic settings. This framework integrates asynchronous learning algorithms and pruning techniques, effectively addressing the inefficiencies of traditional federated learning algorithms in scenarios involving heterogeneous devices, as well as tackling the staleness issue and inadequate training of certain clients in asynchronous algorithms. Through the incremental restoration of model size during training, the framework expedites model training while preserving model accuracy. Furthermore, enhancements to the federated learning aggregation process are introduced, incorporating a buffering mechanism to enable asynchronous federated learning to operate akin to synchronous learning. Additionally, optimizations in the process of the server transmitting the global model to clients reduce communication overhead. Our experiments across various datasets demonstrate that: (i) significant reductions in training time and improvements in convergence accuracy are achieved compared to conventional asynchronous FL and HeteroFL; (ii) the advantages of our approach are more pronounced in scenarios with heterogeneous clients and non-IID client data.
Peer Review Status:Awaiting Review
Subjects: Astronomy >> Astronomical Instruments and Techniques Subjects: Physics >> Nuclear Physics Subjects: Computer Science >> Computer Application Technology submitted time 2024-03-10
Abstract： The HADAR experiment, which will be constructed in Tibet, China, combines the wide-angle advantages of traditional EAS array detectors with the high sensitivity advantages of focused Cherenkov detectors. Its physics objective is to observe transient sources such as gamma-ray bursts and counterparts of gravitational waves. The aim of this study is to utilize the latest AI technology to enhance the sensitivity of the HADAR experiment. We have built training datasets and models with distinctive creativity by incorporating relevant physical theories for various applications. They are able to determine the kind, energy, and direction of incident particles after careful design. We have obtained a background identification accuracy of 98.6 %, a relative energy reconstruction error of 10.0 %, and an angular resolution of 0.22-degrees in a test dataset at 10 TeV. These findings demonstrate the enormous potential for enhancing the precision and dependability of detector data analysis in astrophysical research. Thanks to deep learning techniques, the HADAR experiment’s observational sensitivity to the Crab Nebula has surpassed that of MAGIC and H.E.S.S. at energies below 0.5 TeV and remains competitive with conventional narrow-field Cherenkov telescopes at higher energies. Additionally, our experiment offers a fresh approach to dealing with strongly connected scattered data.
Subjects: Psychology >> Applied Psychology Subjects: Computer Science >> Computer Application Technology submitted time 2024-02-29
Abstract： Objective With the development of network big data and machine learning, more and more studies starting to combine text analysis and machine learning algorithms to predict individual satisfaction. In the studies focused on building life satisfaction prediction models, it is often difficult to obtain large amounts of valid and labeled data. This study aims at solving this problem using data augmentation and optimizing the prediction model of life satisfaction. Method Using 357 life status descriptions annotated by self-rating life satisfaction scale scores as original text data. After preprocessing using DLUT-Emotionontology, EAD and back-translation method was applied and the prediction model was built using traditional machine learning algorithms. Results Results showed that (1) the prediction accuracy was largely enhanced after using the adapted version of DLUT-Emotionontology; (2) only linear regression model was enhanced after data augmentation; (3) rigid regression model showed the greatest prediction accuracy when trained by original data (r = 0.4131). Conclusion The improvement of feature extraction accuracy can optimize the current life satisfaction prediction model, but the text data augmentation methods, such as back translation and EDA may not be applicable for the life satisfaction prediction model based on word frequency.
Peer Review Status:Awaiting Review
Subjects: Computer Science >> Other Disciplines of Computer Science submitted time 2024-02-20
Abstract： This paper analyzes the main problems of the current LLM and proposes specific solutions, pointing out the fact that: the expression and computation of the conceptualized structural model combined with probability is the key, and provides a brief explanation of the related technology-Deep Semantic Model (DSM), and finally enumerates the direction of the subsequent key work.
Peer Review Status:Awaiting Review
Subjects: Computer Science >> Natural Language Understanding and Machine Translation submitted time 2024-02-20
Abstract： OpenAI's Generative Pre-trained Transformer 4 (GPT-4) is a powerful large language model with a certain degree of intelligence in understanding and generating coherent text. We are exploring whether GPT-4 is capable of acting as a die, i.e. generating random numbers. We show that GPT-4 does not appear to generate independent and identically distributed random numbers. Examples imply that GPT-4 tries to compensate for the uniformity of random numbers by sacrificing independence when acting as a die.
Peer Review Status:Awaiting Review
Subjects: Computer Science >> Information Security submitted time 2024-01-22
Abstract： With the rapid development of AIGC (Artificial Intelligence Generated Content) technology, hyper-realistic forged facial videos have become capable of deceiving human visual perception. As a result, a significant number of facial anti-forgery detection algorithms have been proposed for the identification of these fake facial videos. However, effectively evaluating the efficacy and applicability of these forgery detection algorithms remains a substantial challenge. To effectively promote the quantitative assessment of facial anti-forgery detection performance and the iterative development of anti-forgery technologies, this paper introduces a large-scale Chinese data benchmark for facial video anti-forgery identification and releases the world's first CHN-DF Chinese dataset (https://github.com/HengruiLou/CHN-DF), filling the gap in facial video anti-forgery datasets in terms of large-scale Chinese data. The paper details the process of constructing the CHN-DF dataset and the Chinese data evaluation benchmark and validates the complexity and realism of the CHN-DF dataset through experiments. It is hoped that this evaluation benchmark will assist researchers in building more practical and effective facial video anti-forgery detection models, thereby advancing the technology in the field of anti-forgery detection. Additionally, this paper addresses the challenges posed by Chinese face video anti- forgery detection benchmark datasets and anti-forgery detection technology. It also proposes potential future research directions, offering valuable insights to advance the development of face video anti-forgery detection technology.
Peer Review Status:Awaiting Review
Subjects: Linguistics and Applied Linguistics >> Linguistics and Applied Linguistics Subjects: Computer Science >> Natural Language Understanding and Machine Translation submitted time 2024-01-11
Abstract： The research and engineering paradigm of natural language processing has been shifted with the rapid development of large languages models represented by the GPT series. It makes a significant impact on the related fields such as healthcare, education, judiciary and finance. At the same time, it also brings new possibilities for linguistics, the study of language itself. In this paper, we employ GPT4, Baichuan2 as well as ChatGLM3 and investigate their abilities of analyzing complex linguistic phenomena, taking ambiguity as an example. The experimental results show that GPT4 can effectively perceive and understand complex linguistic phenomena by integrating ambiguity resolution and syntactic analysis. For Baichuan2, if it is guided properly via prompt engineering, its analytical ability can be improved without parameter optimization. In addition, the relationship between linguistic phenomena and large language models can be visually demonstrated by monitoring the internal features and neuron activities of the models when processing ambiguous sentences in different context. In general, our experiments indicate that large language models are beneficial to better understanding the analyzing complex linguistic phenomena, hence providing new alternatives for linguistic research.
Hits
Hits
Downloads
Comment
YES