Critique ability, defined as the capacity to identify and rectify flaws in text generation, is crucial for the applications of Large Language Models (LLMs). As a meta-cognitive capability, enhancing the critique ability of LLMs poses significant challenges. Recent studies have proposed improving this ability through fine-tuning on critique datasets. However, the static data distribution of existing datasets often leads to a mismatch between the training data and the diverse optimization needs of target models, thereby hindering their effectiveness. To address this issue, we introduce a novel Dynamic Iterative Data Distribution Optimization Method (DIDD) that dynamically adjusts training data distributions to align with the specific optimization requirements of target models. Specifically, DIDD detects the vulnerable data distribution of target optimization models by conducting the meta-critique on synthesized test set. The detected vulnerable data distribution are then leveraged to construct the training dataset that aligns with target model more closely, improving the effectiveness of the training dataset. Extensive experimental results across four benchmarks demonstrate that our proposed DIDD effectively alleviates the mismatch between the training dataset and target optimization models.

Depression is a common and serious mental disorder, characterized by persistent low mood, loss of interest, cognitive dysfunction, and physiological changes. Patients may experience symptoms such as sleep disturbances, changes in appetite, fatigue, and low self-esteem, with severe cases potentially leading to suicidal behavior. There are differences in emotional processing and attention allocation between patients with depression and healthy controls, eye movement characteristics such as fixation patterns, saccade amplitude, and attentional bias have been used as physiological signals for depression detection. Many researchers have developed depression recognition models based on ocular imaging. However, convolutional neural networks, which utilize local receptive fields, can only capture local features in ocular imaging. This paper proposes Multi-Scale Temporal-Frequency Attention Network (MTFNet), which innovatively integrates Multi-Scale time-frequency domain attention into the Video Swin Transformer. Through Multi-Scale Temporal-Frequency Attention Module (MTFAM), MTFNet learns the most important regions in eye movement images, enabling it to capture features more effectively from sequential data and gain a deeper understanding of the structure within eye movement images. Experimental results show that the proposed method achieves a high accuracy of 76.8% on a self-collected eye movement image dataset, outperforming most models. This work provides a novel approach to research on depression recognition based on eye movement images.

Depression remains a leading cause of suicide among college students, highlighting the need for effective and scalable screening methods. Internet usage behavior has shown strong potential for identifying depressive tendencies, but privacy concerns limit its practical use. In this study, we propose a privacy-conscious cross-scale adaptive transformer designed for irregular time series data derived from weakly private online behavior, such as application categories and usage patterns, while excluding content-sensitive or personally identifiable information. Our model incorporates an adaptive sampling strategy to unify temporal resolutions and uses a cross-scale attention mechanism to capture depression-related behavioral patterns. We compared several classic models for irregular time series data, and the proposed method outperformed them, offering a promising, non-intrusive approach for depression detection based on privacy-conscious online activity patterns.

Objective depression assessments based on physiological data offer new opportunities for clinical diagnostic support, but their usage is constrained by the limitations of data collection devices. Ubiquitous digital devices, now deeply integrated into everyday life, have the potential to effectively capture and represent digital phenotypes of depression. However, existing research in this field faces fundamental problems, such as device thresholds and insufficient utilization of distributed computational power, which have hindered significant research and application efforts in this area. In this article, we proposed the Computation-Oriented Hierarchical Depression Detection Internet of Things (IoT) Framework, which allows IoT devices to collaborate in a layered and distributed manner for psychological data collection and depression detection. In addition, we developed a depression detection model, i.e., spike memory transformer (SMT), which significantly reduces inference energy consumption, facilitating the deployment of depression detection capabilities across various IoT terminal devices. Experimental results demonstrate that our model achieved up to 70% accuracy on the D-Vlog dataset while reducing inference power consumption by an average of 38% compared to classical deep learning methods, thus validating the feasibility of proposed method.

Large AI models, connected to terminal devices via high-speed mobile communication networks, enable task collaboration and resource sharing, forming an intelligent framework for the Symbiotic Internet of Things (SIoT) paradigm in industry applications. Despite large language models hold significant potential for psychological intervention, their emotional interaction capabilities remain limited. This paper introduces a SIoT framework for psychological intervention and proposes an emotion-enhanced human-machine interaction architecture incorporating behavioral information captured by IoT devices. The system leverages ubiquitous sensing devices such as cameras and microphones, along with technologies like speech recognition and generation, as well as hyper-realistic digital humans, to create a natural interaction interface. Additionally, we introduce the end-of-utterance detection method and the behavior pattern control algorithm to facilitate smoother and more goal-oriented conversations. The proposed methods and prototype system have been validated through subjective and objective experiments, with results demonstrating their feasibility and suggesting that this approach could become one of the primary forms of humanmachine interaction for psychological intervention in the future.

The heterogeneity has been recognized as a large obstacle to the treatment of autism spectrum disorder (ASD). Recent studies have identified several subgroups of ASD that exhibited heterogeneous alterations in brain. However, most of them primarily depicted the pairwise similarity between individuals, relying solely on a single imaging modality. This leads to an underestimation of the complexity in inter-individual relationships and the rich information provided by multimodal images. To capture the high-order relationships among individuals, we utilized multi-task method to construct multilayer hypergraph based on brain structure and function. We then developed a novel co-optimized community detection algorithm, which jointly optimizes the modular structure across hypergraph layer, with the aim of categorizing subtypes of ASD by fusing multimodal information. By applying the proposed method on the Autism Brain Imaging Data Exchange repository data (n = 287/303, ASD/typical development [TD]), we identified two ASD subtypes with distinct alteration patterns in both brain structure and function. Distinct clinical manifestations in social and communication were observed between the two subtypes. Furthermore, subtyping significantly enhanced the diagnostic accuracy of ASD by over 10%. In addition, our method exhibited superior clustering performance that outperformed traditional community detection algorithms on graphs. Taken together, our study demonstrated the effectiveness of subtyping ASD through a multimodal multilayer hypergraph, highlighting its potential in elucidating the heterogeneity of autism and improving clinical diagnosis.

Depression is a major mental health issue in contemporary society, with an estimated 350 million people affected globally. The number of individuals diagnosed with depression continues to rise each year. Currently, clinical practice relies entirely on self-reporting and clinical assessment, which carries the risk of subjective biases. In this article, we propose a multimodal method based on facial expression and pupil to detect depression more objectively and precisely. Our method first extracts the features of facial expressions and pupil diameter using residual networks and 1-D convolutional neural networks. Second, a cross-modal fusion model based on self-attention networks (CMF-SNs) is proposed, which utilizes cross-modal attention networks within modalities and parallel self-attention networks between different modalities to extract CMF features of facial expressions and pupil diameter, effectively complementing information between different modalities. Finally, the obtained features are fully connected to identify depression. Multiple controlled experiments show that compared to single modality, the multimodal fusion method based on self-attention networks has a higher ability to recognize depression, with the highest accuracy of 75.0%. In addition, we conducted comparative experiments under three different stimulation paradigms, and the results showed that the classification accuracy under negative and neutral stimuli was higher than that under positive stimuli, indicating a bias of depressed patients toward negative images. The experimental results demonstrate the superiority of our multimodal fusion method.

Recent research on the assessment of college teachers’ promotions has predominantly focused on qualitative approaches, offering suggestions to enhance fairness, objectivity and efficiency within promotion systems. Addressing this gap, this study proposes a method to predict promotion outcomes based on teachers’ characteristics, including teaching and research abilities. This article integrates qualitative and quantitative analyses to evaluate promotion criteria and find factors affecting teachers’ promotion. The promotion results are correlated with the application characteristics, confirming their statistical significance through hypothesis testing. Additionally, the quantitative analysis reduces subjective bias based on visual data and statistical methods. Machine learning methods are applied to teacher promotion prediction. Results categorizing promotion outcomes are refined using Logistic regression, K-nearest neighbor, and BP neural network models, with the BP neural network showing superior performance. Furthermore, an optimized BP neural network using principal component analysis demonstrates improved metrics. This study contributes a precise method to predict college teachers’ promotion outcomes, offering insights to refine faculty evaluation systems.

Facial expressions have been widely used for depression recognition because it is intuitive and convenient to access. Pupil diameter contains rich emotional information that is already reflected in facial video streams. However, the spatiotemporal correlation between pupillary changes and facial behavior changes induced by emotional stimuli has not been explored in existing studies. This paper presents a novel multimodal fusion algorithm - Trial Selection Tensor Canonical Correlation Analysis (TSTCCA) to optimize the feature space and build a more robust depression recognition model, which innovatively combines the spatiotemporal relevance and complementarity between facial expression and pupil diameter features. TSTCCA explores the interaction between trials and obtains an effective fusion representation of two modalities from a trial subset related to depression. The experimental results show that TSTCCA achieves the highest accuracy of 78.81% with the subset of 25 trials.

In contemporary society, the issue of psychological health has become increasingly prominent, characterized by the diversification, complexity, and universality of mental disorders. Cognitive Behavioral Therapy (CBT), currently the most influential and clinically effective psychological treatment method with no side effects, has limited coverage and poor quality in most countries. In recent years, researches on the recognition and intervention of emotional disorders using large language models (LLMs) have been validated, providing new possibilities for psychological assistance therapy. However, are large language models truly possible to conduct cognitive behavioral therapy? Many concerns have been raised by mental health experts regarding the use of LLMs for therapy. Seeking to answer this question, we collected real CBT corpus from online video websites, designed and conducted a targeted automatic evaluation framework involving three aspects, namely the evaluation of emotion tendency of generated text, structured dialogue pattern and proactive inquiry ability. Considering limited CBT-related texts in a general chat LLM’s training corpus, we evaluated the CBT ability of the LLM after integrating a CBT knowledge base to explore the influence of introducing additional knowledge. Four LLM variants with exceptional performance are evaluated, and the experimental result shows the great potential of LLMs in psychological counseling realm, especially after combining with other technological means.

Unmanned aerial vehicles (UAVs) often require real-time and accurate control in military, rescue, and transportation applications. There is an urgent demand from operators for usability and interoperability. Human-machine collaboration enhancement and tiny machine learning aimed at edge computing hold the potential to greatly improve the operational efficiency of UAVs. In this paper, we propose a novel UAV control framework based on eye-tracking technology, i.e. Eye-Tracking-based UAV (ETUAV), which combines with lightweight object detection to assist the control of the UAV for attitude adjustment through the gaze information of the UAV operator. The eye-tracking-based UAV control method we propose involves real-time tracking of the operator’s gaze using our self-developed head-mounted eye tracker, combined with object detection to achieve automatic targeting. We also propose an incremental proportion integration differentiation (PID) control algorithm for adjusting the UAV’s attitude, which provides automatic and real-time UAV control. We develop a system prototype based on the DJI UAV and conduct performance benchmarks and comparisons. The experimental results indicate that the operational latency of our method is significantly less than manual operation and the built-in automatic tracking function in the DJI application. The “Aim Where You Look” UAV operation framework proposed in this article significantly streamlines the tasks for operators in time-critical scenarios.

Smartphones are widely used as portable data collectors for wearable and healthcare sensors that can passively collect data streams related to the environment, health status, and behaviors. Recent research shows that the collected data can be used to monitor not only the physical states but also the mental health of individuals. However, extracting the features of digital phenotypes that characterize major depressive disorder (MDD) is technically challenging and may raise significant privacy concerns. Addressing such challenges has become the focus of many researchers. This article provides a comprehensive analysis of several key issues related to ubiquitous sensing to aid in detecting MDD. Specifically, this article analyzes existing methodologies and feature extraction algorithms used to detect possible MDD through digital phenotyping from smartphone data. In particular, five types of features are summarized and explained, namely, location, movement, rhythm, sleep, and social and device usage. Finally, related limitations and challenges are discussed to provide paths for further research and engineering.

Autism spectrum disorder (ASD) is accompanied by impaired social cognition and behavior. The expense of supporting patients with ASD turns into a significant problem for society. Parsing neurobiological subtypes is a crucial way for delineating the heterogeneity in autistic brains, with significant implications for improving ASD diagnosis and promoting the development of personalized intervention models. Nevertheless, a comprehensive understanding of the heterogeneity in cortical morphology of ASD is still lacking, and the question of whether neuroanatomical subtypes remain stable during cortical development remains unclear. Here, we used T1-weighted images of 515 male patients with ASD, including 216 autistic children (6–11 years), 187 adolescents (12–17 years), and 112 young adults (18–29 years), along with 595 age and gender-matched typically developing (TD) individuals. Cortical thickness (CT), surface area (SA), and volumes of cortical (CV) and subcortical (SV) regions were extracted. A single network layer was established by calculating the covariance of each feature across brain regions between participants, thereby constructing a multilayer intersubject covariance network. Applying a community detection algorithm to multilayer networks derived from different feature combinations, we observed that the network comprising CT and CV layers exhibited the most prominent modular organization, resulting in three subtypes of ASD for each of the three age groups. Subtypes within the corresponding age group significantly differed in terms of brain morphology and clinical scales. Furthermore, the subtypes of children with ASD underwent reorganization with development, transitioning from childhood to adolescence and adulthood, rather than consistently persist. Additionally, subtype categorization largely improved the diagnostic accuracy of ASD compared to diagnosing the entire ASD cohort. These findings demonstrated distinct neuroanatomical manifestations of ASD subtypes across various developmental periods, highlighting the significance of age-related subtyping in facilitating the etiology and diagnosis of ASD.

As one of the most prevalent mental disorders, depression is associated with a high rate of self-harm and suicide, particularly among college students. It is urgently needed to discover prospective cases of depression disorder among college students, enabling timely intervention to reduce its impact on their academic performance and daily lives. This study investigates a method for identifying groups that may have early depressive tendencies through their Internet usage on campus networks. This article proposes a heterogeneous graph attention network (H-GAT) model that incorporates an attention mechanism based on ablation experiments in heterogeneous graphs to analyze the patterns and correlations within the surfing behavior data of students. This model makes full use of the interaction relationships between heterogeneous nodes in the graph to capture the affective tendencies reflected in the cyber-activity patterns. The proposed H-GAT model exhibits excellent performance, with nearly 80% accuracy and recall. Our work offers a potential approach to detect depression on college campuses using nonintrusive methods, which could ultimately contribute to early warnings for both individuals experiencing depression and higher education institutions.

This edition of the publication includes a robust collection of 104 regular papers and features a Special Issue on Knowledge- Infused Learning for Computational Social Systems. This special issue delves into the sophisticated integration of advanced technologies and knowledge-based methodologies within the analysis of computational social systems. Spanning 12 articles, the issue addresses a wide spectrum of topics, from big data management to refining machine learning models with domain-specific insights. It encompasses areas such as energy management in sensor networks, acoustic analysis of heartbeats, detection of fraudulent activities in online ratings, and the management of rumors on social networks, exemplifying the significant role that knowledge-infused learning plays in enhancing technological applications and fostering innovation in social computational systems.

Eye-tracking technology is extensively utilized in affective computing research, enabling the investigation of emotional responses through the analysis of eye movements. Integration of eye-tracking with other modalities, allows for the collection of multimodal data, leading to a more comprehensive understanding of emotions and their relationship with physiological responses. This paper presents a novel head-mounted eye-tracking system for multimodal data acquisition with a completely redesigned structure and improved performance. We propose a novel method for pupil-fitting with high efficiency and robustness based on deep learning and RANSAC, which gets better performance of pupil segmentation when it is partially occluded, and build a 3D model to obtain gaze points. Existing eye trackers for multi-modal synchronous data collection either have limited device support or suffer from significant synchronization delays. Our proposed hard real-time synchronization mechanism implements microsecond level latency with low cost, which facilitates multimodal analysis for affective computing research. The uniquely designed exterior effectively reduces facial occlusion, making it more comfortable for the wearer while facilitating the capture of facial expressions.

Depression is one of the most common mental illnesses, but few of the currently proposed in-depth models based on social media data take into account both temporal and spatial information in the data for the detection of depression. In this paper, we present an efficient, low-covariance multimodal integrated spatio-temporal converter framework called DepMSTAT, which aims to detect depression using acoustic and visual features in social media data. The framework consists of four modules: a data preprocessing module, a token generation module, a Spatial-Temporal Attentional Transformer (STAT) module, and a depression classifier module. To efficiently capture spatial and temporal correlations in multimodal social media depression data, a plug-and-play STAT module is proposed. The module is capable of extracting unimodal spatio-temporal features and fusing unimodal information, playing a key role in the analysis of acoustic and visual features in social media data. Through extensive experiments on a depression database (D-Vlog), the method in this paper shows high accuracy (71.53%) in depression detection, achieving a performance that exceeds most models. This work provides a scaffold for studies based on multimodal data that assists in the detection of depression.

Mental health has long been studied, and many AI-based approaches have been proposed for diagnosis and adjunctive therapy. The emergence of Pre-trained Large Language Models (LLMs) has had a profound impact on various fields, but the potential of using ChatGPT for cognitive behavior therapy is largely unexplored. Therefore, there is an urgent need to build and design a virtual interactive framework for assisted diagnosis/treatment. In this paper, we present a virtual interaction framework based on LLMs that allows participants to engage in a dialogue with a virtual character, analyse mental health issues through augmented LLMs, and make suggestions during the dialogue to alleviate the psychological problems they are currently facing. Based on this framework, we develop a use case for the application of ChatGPT in the field of emotional disorders. Specifically, we use data from question-and-answer dialogues in real-life scenarios to populate the current exploration of ChatGPT’s potential for depression and anxiety detection. The case study shows the great potential of ChatGPT in the analysis of depression and anxiety tests. The feasibility of a virtual interaction framework based on LLMs has been preliminarily demonstrated.

Human-Computer Interaction (HCI) and Machine Learning (ML) technologies have potential for the behavioral screening of autistic children but how to design a tool and analyse behavior reliably is challenging. Based on psychophysiological computation, this paper proposes an interactive behavior perception analytical model for autism screening. We presented the multi-scenario reactive behavior paradigms that designed based on the atypical characteristics of autistic children. We recorded the eye movement data and facial data of 91 participants, and performed multi-modal feature extraction, used machine learning to train classification model. We conducted comparative experiments, and the experimental results verified the advantages of multi-scenario paradigms and multi-modal feature groups, which indicates that our analysis methods and screening models are effective and reliable and have real research significance.

Pupil dilation and eye movements are closely related to human emotional and cognitive processes. Visual stimulus, especially video clips, is widely used in computer-assisted experimental paradigms as emotional inducers. However, the level of attraction as a critical factor to such visual stimulus still needs comprehensive investigation. This article conducts a novel study of attraction assessment with pupil diameter and eye movements. We collected high temporal resolution ocular variation data from 50 subjects while they viewed a variety of emotional video stimuli. Besides, this article proposes two orthogonal-moment-based feature extraction methods for emotion classification, i.e., Legendre moment and Krawtchouk moment. The results of experiments show that our proposed feature sets achieve better classification performance compared with conventional time- or frequency-domain feature sets. The accuracy of predicting attraction level reached 87.4% and 91.0% when new features were used alone and combined with conventional features, respectively. Compared with the conventional features with an accuracy of 86.6%, our proposed features can improve the accuracy by 4.4%.This study conducts a ground investigation of quantitative attraction assessment for video clips, which might provide reference for paradigm design of affective computing research.

Generalized anxiety disorder (GAD), as one of the most common chronic anxiety disorders, faces difficulties in clinical diagnosis. With the rapid development and wide application of smartphones in recent years, smartphones have a vivid application prospect in the field of mental disease monitoring and diagnosis. Based on WeChat applet platform on smartphones, an APP that integrates scale testing and inertial sensor data collection is developed to study the detection of subjects with GAD in task state. A behavior patterns extraction method is proposed using sliding windows to split behavior data, and processing data segments for clustering. Distribution information are extracted from the subjects' behavior patterns and are combined with the descriptive statistical features of the sample to identify GAD. The results show that this method has an accuracy of 66.44% for female subjects and 71.43% for male subjects in GAD recognition.