Spatiotemporal analysis of facial behavior is a crucial method for evaluating the mental state of depression patients. However, in practice, depressed patients often display facial behaviors similar to healthy individuals due to masking tendencies. Additionally, facial expressions among depressed patients are also different, increasing the difficulty of assessment. To address this, we propose a video-based automatic depression assessment model Dep-MAP for complex facial behaviors of depression patients. Dep-MAP adopts a dual-branch architecture to extract visual features of facial behavior and capture corresponding emotional semantic features. Specifically, the extracted deep semantic features are clustered, resulting in semantically distinct prototype sets, where each severity group learns a set of discriminative facial behavior prototype representations, to suppress inter-class semantic confusion. Subsequently, we propose a semantic prototype-supervised contrastive learning method, which aligns latent semantics between shallow and deep features, realizing emotional semantic guidance and self-knowledge distillation for the visual feature branch, effectively suppressing intra-class difference. Then, we integrate key depression cues across multiple spatiotemporal scales via a multi-scale weighted fusion strategy, achieving automatic depression assessment. Experimental results demonstrate that Dep-MAP effectively identifies potential key frames in temporal sequences, and aggregates key frame representations with semantic consistency, achieving significantly superior state-of-the-art results on the AVEC2013 and AVEC2014 public datasets.

Spatial transcriptomics (ST) technologies provide powerful tools for analyzing spatial distribution patterns of gene expression in tissue samples. However, they are limited by sparse gene detection and incomplete expression coverage. Several computational approaches based on reference scRNA-seq have been proposed to impute ST data and have achieved impressive results. However, these methods fail to fully explore latent temporal dependencies among cells and cannot accurately capture hidden gene-level regulatory mechanisms. To overcome those limitations, we propose SpaLSTF, a novel method for enhancing ST gene expression using a conditional diffusion model guided by scRNA-seq data. SpaLSTF captures gene expression relationships through a dual Markov process: one progressively perturbs scRNA-seq data with noise, while the other denoises it to reconstruct the original distribution. To effectively model contextual dependencies among cell states, we adopt a bidirectional long short-term memory (BiLSTM) network. Furthermore, we design a cross-covariance attention mechanism within a Transformer (XCA-Transformer) to efficiently compute attention coefficients between gene expression and accurately predict the noise added at each step. In addition, we introduce a variational lower bound (VLB) objective and introduce Kullback-Leibler (KL) divergence as a regularization term, along with mean squared error loss, to ensure that the generated noise follows the target distribution. We compared the performance of SpaLSTF with seven state-of-the-art methods on twelve cross-platform datasets covering a variety of tissues and organs using nine evaluation metrics. Experimental results demonstrated that SpaLSTF outperforms competing methods in gene expression imputation, cell population identification, and spatial structure preservation.

[This corrects the article DOI: 10.1371/journal.pcbi.1013667.].

Acoustic features are crucial behavioral indicators for depression detection. However, prior speech-based depression detection methods often overlook the variability of emotional patterns across samples, leading to interference from speaker identity and hindering the effective extraction of emotional changes. To address this limitation, we developed the Emotional Word Reading Experiment (EWRE) and introduced a method combining self-supervised and supervised learning for depression detection from speech called MFE-Former. First, we generate fine-grained emotional representations for response segments by computing cosine similarity between intra-sample and inter-sample contexts. Concurrently, orthogonality constraints decouple identity information from emotional features, while a Transformer decoder reconstructs spectral structures to improve sensitivity to depression-related emotional patterns. Next, we propose a multi-scale emotion change perception module and a Bernoulli distribution-based joint decision module integrate multi-level information for depression detection. By enhancing the distribution differences among positive, neutral, and negative emotional features, we find that patients with depression are more inclined to express negative emotions, whereas healthy individuals express more positive emotions. The experimental results on EWRE and AVEC 2014 show that MFE-Former outperforms state-of-the-art temporal methods under conditions of variability in emotional patterns across samples.

Facial expression recognition (FER) is essential for emotional assessment in the treatment and monitoring of mental health conditions. However, the scarcity of facial expression data from patients with mental illnesses, coupled with the subtlety and complexity of their expressions, such as limited emotional intensity and minimal facial movement presents significant challenges. To address this, we introduced the Voluntary Facial Expression Mimicry (VFEM) experiment, collecting data on seven types of expressions from patients with depression and anxiety. Based on VFEM dataset, we developed a novel Multi-Scale Adaptive Fusion Network (MSAF-Net). First, we designed a Multi-Dimensional Feature Refinement Module to enhance expression feature representation. Next, we proposed a Multi-Scale Adaptive Fusion Module to improve feature fusion and consistency. Finally, by incorporating Center Cross-Entropy Loss, we optimized feature distribution and classification. Extensive experiments on the VFEM dataset, compared with state-of-the-art models, show that our approach achieves competitive results in mental health facial expression recognition.

The rapid development of spatial transcriptomics (ST) has made it possible to effectively integrate gene expression and spatial information of cells and accurately identify spatial domains. A large number of deep learning (DL)-based methods have been proposed to perform spatial domain identification and achieved impressive results. However, these methods have some limitations. First, these methods rely on a fixed similarity metric and cannot fully utilize neighborhood information. Second, they cannot efficiently and adaptively integrate key information when fusing and reconstructing gene expression using purely additive methods. Finally, these methods ignore key nonlinear features and introduce noise during clustering. To address these limitations, we propose a novel DL model SpaMWGDA based on multi-view weighted fused graph convolutional network (GCN) and data augmentation. By modeling spatial information using different similarity metrics, the model is able to successfully capture comprehensive neighborhood information of the spot features. By combining data augmentation and contrastive learning, SpaMWGDA is able to learn key gene expressions. SpaMWGDA uses a multi-view GCN encoder to model the similarities between spatial information and gene features, and uses a view-level attention mechanism for weighted fusion to adaptively learn the dependencies between them and learn the key features of each view. Experimental results not only demonstrate that SpaMWGDA outperforms competing methods in spatial domain identification and trajectory inference but also show the ability of SpaMWGDA to analyse tissue structure and function. The source code for SpaMWGDA is available at https://github.com/nathanyl/SpaMWGDA .

Clinical research has demonstrated that exploring behavioral signal differences between depressed patients and non-depressed people using audiovisual technology is an effective approach for achieving depression recognition. Hence, in this paper we propose an emotion word reading experiment (EWRE), and extract features from facial expressions and audios for depression recognition. Building upon this, we propose a depression recognition model (DEP-Former), which deeply integrates multimodal features. DEP-Former first designs a modality adapter to achieve emotion space mapping and the sharing of multimodal features, addressing cross-modal inconsistencies. Simultaneously, it proposes a mechanism of attention index sharing, exceeding the limitations of cognitive subjectivity by calculating confidence in key emotional information across modalities. Finally, we propose a multimodal cross-attention module and a Bernoulli distribution feature fusion prediction module to achieve deep integration of multilevel information, thereby enabling depression recognition. Compared with existing advanced multimodal models, DEP-Former demonstrates superior performance in EWRE, achieving an accuracy of 0.9500 and an F1 score of 0.9499, significantly enhancing depression recognition over the single-modality methods. Furthermore, its robust generalization ability is validated on the AVEC 2014 dataset. Through the attention query of the interpretability analysis module, we discover that depressed patients exhibit heightened sensitivity to negative emotional words, such as dismissal and tragedy. In contrast, healthy individuals tend to be more attuned to positive emotional words, including passion, purity, and justice. Additionally, depressed patients exhibit a degree of psychological state diversity, showing sensitivity to some positive emotional words as well. Our codes and data are available athttps://github.com/QLUTEmoTechCrew/DEP-Former.

Facial Expression Recognition (FER) is a critical method for evaluating the emotional states of patients with mental disorders, playing a significant role in treatment monitoring. However, due to privacy constraints, facial expression data from patients with mental disorders is severely limited. Additionally, the more complex inter-class and intra-class similarities compared to healthy individuals make accurate recognition of facial expressions challenging. Therefore, we propose a Voluntary Facial Expression Mimicry (VFEM) experiment, which collected facial expression data from schizophrenia, depression, and anxiety. This experiment establishes the first dataset designed for facial expression recognition tasks exclusively composed of patients with mental disorders. Simultaneously, based on VFEM, we propose a Vision Transformer FER model tailored for Complex mental disorder patients (CmdVIT). CmdVIT integrates crucial facial expression features through both explicit and implicit mechanisms, including explicit visual center positional encoding and implicit sparse attention center loss function. These two key components enhance positional information and minimize the facial feature space distance between conventional attention and critical attention, effectively suppressing inter-class and intra-class similarities. In various FER tasks for different mental disorders in VFEM, CmdVIT achieves more competitive performance compared to contemporary benchmark models. Our works are available at https://github.com/yjy-97/CmdVIT.

Facial expressions are important nonverbal behaviors that humans use to express their feelings. Clinical research have shown that depressed patients have poor facial expressiveness and mimicry. As a result, we propose a VFEM experiment with seven expressions to explore variations in facial expression features between depressed patients and normal people, including anger, disgust, fear, happiness, neutrality, sadness, and surprise. It has been discovered through VFEM experiments that depressed patients frequently exhibit negative facial expressions. Meanwhile, we propose a depression facial expression recognition (Dep-FER) model in this research. Dep-FER involves three innovative and crucial components: Mask Multi-head Self-Attention (MMSA), facial action unit similarity loss function (AUs Loss), and case-control loss function (CC Loss). MMSA can filter out disturbing samples and force to learn the relationship between different samples. AUs Loss utilizes the similarity between each expression AU and the model output to improve the generalization ability of the model. CC Loss addresses the intrinsic link between the depressed and normal patient categories. Dep-FER achieves excellent performance in VFEM and outperforms existing comparative models.

Modern technology innovations feature a successive and even recurrent procedure. Intervals between old and new generations of technology are shrinking, and the Internet and Web services have facilitated the fast adoption of an innovation even before the convergence of its predecessor. While the adoption and diffusion of innovations have been studied for decades, most theories and analyses focus on single and one-time innovations. Meanwhile, limited work has investigated successive innovations while lacking user-level analysis, possibly due to the unavailability of fine-grained adoption behavior data. In this study, we present the first large-scale analysis of the adoption of recurrent innovations in the context of mobile app updates, investigating how millions of users consume various versions of thousands of apps on their mobile devices. Our analysis reveals novel patterns of crowd and individual adoption behaviors, which suggest the need for new categories of adopters to be added on top of the Rogers model of innovation diffusion. We show that standard machine learning models are able to pick up various sources of signals to predict whether users in these different categories will adopt a new version of an app and how soon they will adopt it.

With the rise in societal pressures, depression and anxiety have increasingly become prominent mental health conditions impacting people’s lives. To enhance the efficacy of automatic detection for these disorders, we have developed an experimental framework called the Voluntary Facial Expression Mimicry(VFEM). This framework led to the creation of the VFEM Dataset, which supports related research endeavors. Subsequently, we introduce the LI-FPN designed specifically for the automatic identification of depression and anxiety disorders. The LI-FPN comprises two core components: the Learning and Imitation Module(LIM) and the Spatio-temporal Feature Pyramid Network(STFPN). Within the LIM, we leverage sequence features to facilitate comprehensive feature extraction through learning and imitation steps. The STFPN is designed to focus on outliers in multi-scale features for further screening. Compared with traditional attention methods, LI-FPN is more suitable for processing sequence data features and small sample datasets. Upon training using the VFEM Dataset, the LI-FPN achieves impressive accuracies: 0.850 for depression detection, 0.835 for anxiety detection, and 0.786 for co-occurrence detection of depression and anxiety. Meanwhile, LI-FPN also achieves SOAT results on AVEC2014 dataset. The source code for LI-FPN is accessible at https://github.com/muzixingyun/LI-FPN

The emergence of Transformer and its derivative models brings new opportunities to tasks of NLP (Natural Language Processing). Transformer is not only a separate model, but also the core of different text task systems. Therefore, Transformer has become an important component of many powerful models. However, Transformer is not without defects. Researchers are still puzzled by the huge amount of computation generated in the process of self-attention. Especially in long text data sets. We propose a new ProbSparse self-attention Transformer model based for text classification. In the following, we will call it SpaL Transformer. We query important attention factors through KL divergence and add Long Short Term Memory(LSTM) to positional encoding, and only focus on the main query. Then, we select the most important relevant attention based on the confidence score to focus the overall attention. At the same time, we propose LSTM positive encoding to obtain relative position information to optimize the model. In long text dataset IMDB, our model improves the accuracy of Transformer. And F1 score improved by 0.064.

Many clinical studies have shown that facial expression recognition and cognitive function are impaired in depressed patients. Different from spontaneous facial expression mimicry (SFEM), 164 subjects (82 in a case group and 82 in a control group) participated in our voluntary facial expression mimicry (VFEM) experiment using expressions of neutrality, anger, disgust, fear, happiness, sadness and surprise. Our research is as follows. First, we collected a large amount of subject data for VFEM. Second, we extracted the geometric features of subject facial expression images for VFEM and used Spearman correlation analysis, a random forest, and logistic regression-based recursive feature elimination (LR-RFE) to perform feature selection. The features selected revealed the difference between the case group and the control group. Third, we combined geometric features with the original images and improved the advanced deep learning facial expression recognition (FER) algorithms in different systems. We propose the E-ViT and E-ResNet based on VFEM. The accuracies and F1 scores were higher than those of the baseline models, respectively. Our research proved that it is effective to use feature selection to screen geometric features and combine them with a deep learning model for depression facial expression recognition.

Depression is one of the most prevalent mental illnesses on a global scale. In view of the inefficiency of current depression screening methods. This paper proposes a depression detection model based on a deep model hybrid architecture to assist doctors in diagnosing depression. 157 Chinese subjects were investigated in this study. It is worth noting that we propose a word reading experiment to make the subject’s emotional change rapidly. We extract the Low-level audio features to find out the different emotional change in the process of reading different parts of speech words. We use convolutional neural network to extract deep spectrum features and Multi-mlp to detect depression. The experimental results show that the accuracy rate of speech depression recognition reaches 82.70%, which can effectively assist doctors in diagnosing depression.

Depression is a serious psychiatric disorder that is prevalent worldwide and is usually characterized by persistently depressed mood, impaired mobility, and delayed thinking and cognitive functions. This experiment uses the Kinect V2 device to record simple kinematic skeletal data of body joints of depressed patients and non-depressed patients, directly extracting the presented spatial features and low-level features from the recorded raw Kinect-3D coordinates. Aiming at the symptoms of delayed thinking and cognitive function and impaired mobility in patients with depression, the features of reaction time are obtained from preprocessing, and this prior knowledge is added to the deep learning model to assist the recognition and classification of the model, thereby improving the classification accuracy. The objective of this project is to develop a deep learning model for detecting depression using preprocessed data.

Depression is a common mental illness. Unlike normal mood fluctuations which affect individuals only temporarily, depressed episodes can profoundly disrupt a person’s daily life and even lead to suicide. Eye movement data are commonly employed in depression identification because they are simple to collect and can show psychological processes. Given the above, we proposed EnSA, a novel model based on eye movement data. We established forward and reverse target stimuli to identify the subject’s saccade reaction and capture the subject’s eye movement data. The gathered eye movement data were entered into EnSA first, and the self-attention weights of each characteristic were calculated. To produce more expressive features, the self-attention features were convolved and then summed feature outputs. According to the results, our model performed well, with an accuracy of 93.5% and 95.5% in the prosaccade and antisaccade experiments, respectively.

With the rapid development of autonomous driving, tracking on-road pedestrians raises more attention in the public. Currently, most researches focus on single camera based tracking or tracking across multiple static cameras. Tracking across multiple moving cameras has not been well studied yet. In this paper, we propose a workflow for tracking pedestrians across multiple moving cameras, leveraging the state-of-the-art single camera based tracking method of FairMOT. We consider different factors such as appearance features, motion information, and camera spatial distribution to improve the tracking performance. The experimental results carried on a multi-target multi-moving camera tracking dataset show the feasibility of the proposed scheme in solving the tracking issue in a complex environmental setting.

In this paper we share several experiments trying to automatically translate informal mathematics into formal mathematics. In our context informal mathematics refers to human-written mathematical sentences in the LaTeX format; and formal mathematics refers to statements in the Mizar language. We conducted our experiments against three established neural network-based machine translation models that are known to deliver competitive results on translating between natural languages. To train these models we also prepared four informal-to-formal datasets. We compare and analyze our results according to whether the model is supervised or unsupervised. In order to augment the data available for auto-formalization and improve the results, we develop a custom type-elaboration mechanism and integrate it in the supervised translation.

Eye movement trajectories of depressed patients and normal persons are different. The eye-tracking data obtained by the eye tracker can adequately summarize the characteristics of the eye movement trajectory. Based on the characteristics of eye movement trajectory, this paper proposes a new depression detection model by using an artificial neural network, which can better assist doctors in the diagnosis of depression. First, we extract the feature of eye movement trajectory, which obtains from time-series data recording the trajectory of the eye. Then, we convert the data from three-dimensional to two-dimensional, and perform feature extraction and transformation. Finally, we propose a new depression detection model by using artificial neural networks. The experimental results show that the best result of the model evaluation is 83.17%, which can effectively assist doctors in the diagnosis of depression.

Web applications are interactive programs that are deployed on the world wide Web. Their execution is usually controlled very heavily by user choices and user data. This makes them vulnerable to abnormal behavior from invalid inputs as well as security attacks. Thus, Web applications invest heavily in validating user inputs according to defined constraints on the values. This work focuses on validation done on the client, which uses two types of technologies; restrictions in HTML form fields and scripts that check values. Unfortunately users have the ability to subvert or skip client-side validation. Bypass testing has been developed to test the behavior of Web applications when client-side validation is skipped. This paper presents results from an industry case study of bypass testing applied to a project from Avaya Research Labs, NPP. The paper presents a process for designing, implementing, automating and developing bypass tests. The theory of bypass testing had to be adapted to the unique characteristics of NPP software, which represented a significant engineering challenge. The 184 tests that were generated resulted in 63 unique failures, providing significant experience and numerous lessons learned. The case study also revealed several difficult problems that need to be addressed in future research.