Learning a generalized EEG representation under limited data and subject variability is a long-standing challenge. Most studies utilized data augmentation to extend the distribution of training data, which may hinder the diversity of augmented samples to cover more subject variability. In this paper, we propose a decomposed statistics-diverse feature augmentation (DSFA) framework for generalized EEG learning. The wavelet-based decomposed representation module decomposes signals into approximation and detail features, thus deriving semantics from original signals into approximation features to avoid over-transformation. The statistics-diverse feature augmentation module augments features to extend beyond the original feature space by manipulating the statistics of features. Extensive experiments on three datasets demonstrate that our approach achieves state-of-the-art performance in different classification tasks. Our repository is public at https://github.com/1940653868/DSFA.

Data-Knowledge dual-driven fashion can enhance model performance by complementing data-driven basis with expert knowledge. However, cutting-edge works in speech emotion recognition (SER) primarily evolve in data-driven training, failing to incorporate prior knowledge to form a closed loop and posing an obstacle to capture task-specific details when used independently. In this paper, we propose a Dual-Drive Broad Attentive Network (DiBAN) to achieve comprehensive emotional learning for SER. Specifically, the Dual-Drive Emotional Modeling module incorporates handcrafted extractor, pre-trained model and tailored base models, to conduct integral emotional modeling. Subsequently, the Multi-Model Attention-Aware Learning module is designed to refine the data-knowledge emotional disparities based on the attention-enhanced entropy loss. Finally, the Broad Adaptive Decision Fusion module performs adaptive fusion of emotional decisions from different drives. Extensive experiments on seven SER corpora demonstrate that DiBAN achieves significant improvements over the base models and outperforms comparative methods, fully showcasing its superiority.

Limited EEG data and subject variability pose significant challenges to the generalization of EEG-based emotion recognition. Most existing approaches augment EEG data using deterministic methods, often neglecting to ensure both diversity and fidelity in the generated samples. This oversight leads to insufficient domain diversity and emotional semantic information for a generalized model independent of individuals. This paper proposes an Information-Aware Graph Augmentation (InfoGA) framework for generalized EEG emotion recognition. The graph uncertainty augmentation module augments both the connectivity and features of EEG graphs by modeling statistical uncertainty, enabling the model to simulate domain shifts and improve generalizability against subject variability. Additionally, two information-aware constraints are introduced to ensure diversity and fidelity in the augmented EEG graphs. The graph diversity constraint enriches the emotional knowledge of the augmented graphs, while the graph fidelity constraint preserves their emotional semantic fidelity by integrating consistency learning with supervised learning. Extensive experiments on three public EEG emotion datasets, i.e., SEED, SEED-IV, and SEED-V, demonstrate that InfoGA achieves superior generalizability compared to baseline methods.

The existence of emotion-irrelevant representations and individual variability impedes the extraction of robust emotional representations, limiting the adaptability of EEG emotion recognition. Massive studies focus on the mining of emotion-aware information, overlooking emotion-agnostic information, which is insufficient for the extraction of emotion-relevant features against redundancy and variation. In this paper, Graph Orthogonal Purification Network (Grop) is proposed to enhance individual adaptability through improvements in the orthogonality and transferability between emotion-relevant and emotion-irrelevant features. Specifically, the proposed Grop utilized a graph representation extraction module to capture both emotion-relevant and emotion-irrelevant features by the dual graph. The representation orthogonal purification module is developed to eliminate redundant information through feature projection and feature purification. Moreover, the dual emotional space alignment module is imposed to align distribution discrepancies in different emotion feature spaces. To assess the effectiveness of the proposed Grop, various experiments are conducted on two public EEG emotion datasets, i.e., SEED and SEED-IV. The results achieve state-of-the-art performance, demonstrating the capability of the Grop to capture robust emotion features and alleviate the intra- and inter-subject discrepancies.

Trustworthy Graph Neural Networks (GNNs) for EEG emotion recognition should identify emotions accurately and elucidate corresponding rationales. Current GNNs have achieved notable performance by dynamically modeling emotional connections between EEG channels. However, these GNNs lack interpretability due to the absence of explicit rationale behind their predictions. This paper conducts a comprehensive identification of important EEG channels to enhance the interpretability of EEG emotion recognition from the perspective of mutual information. Specifically, an Adjacency-Explainable Graph Neural Network (AEG) for ante-hoc interpretability is proposed to capture genuine EEG emotional connections, which gives a theoretical guarantee to remove spurious connections. Moreover, a Channel-wise Adaptive Class Activation Mapping Explainer (CACA) for post-hoc interpretability is developed to locate the EEG channels that contribute most to predictions. Experimental results on three datasets, i.e., SEED, SEED-IV, and DREAMER, prove that imbuing training processes with enhanced interpretability ensures significant performance improvements in emotion recognition. Quantitative comparisons of post-hoc interpretability also demonstrate the superiority of CACA. Furthermore, this paper illustrates two potential applications of the proposed methodologies, showing their broader utility and significance.

The underlying time-variant and subject-specific brain dynamics lead to statistical uncertainty in electroencephalogram (EEG) representations and connectivities under diverse individual biases. Current works primarily augment statisticallike EEG data based on deterministic modes without comprehensively considering uncertain statistical discrepancies in representations and connectivities. This results in insufficient domain diversity to cover more domain variations for a generalized model independent of individuals. This article proposes an uncertainty-guided graph augmentation network (Ugan) to generalize EEG emotion recognition across subjects by comprehensively mimicking and constraining the uncertain statistical shifts across individuals. Specifically, an uncertainty-guided graph augmentation module is employed to augment both connectivities and features of EEG graph by manipulating domain statistical characteristics. With the original and augmented EEG graph covering diverse domain variations, the model can mimic the uncertain domain shifts to achieve better generalizability against potential subject variability. To extract discriminative characteristics and preserve emotional semantics after augmentation, a graph coteaching learning module is designed to facilitate coteaching knowledge learning between the original and augmented views. Moreover, a coteaching regularization module is developed to constrain semantic domain invariance and consistency, thereby rendering the model invariant to uncertain statistical shifts. Extensive experiments on three public EEG emotion datasets, i.e., Shanghai Jiao Tong University emotion EEG dataset (SEED), SEED-IV, and SEED-V, validate the superior generalizability of Ugan compared to the state-of-the-art methods.

The underlying time-variant and subject-specific brain dynamics lead to inconsistent distributions in electroencephalogram (EEG) topology and representations within and between individuals. However, current works primarily align the distributions of EEG representations, overlooking the topology variability in capturing the dependencies between channels, which may limit the performance of EEG emotion recognition. To tackle this issue, this article proposes an adaptive attention-modulated graph network (AdamGraph) to enhance the subject adaptability of EEG emotion recognition against connection variability and representation variability. Specifically, an attention-modulated graph connection module is proposed to explicitly capture the individual important relationships among channels adaptively. Through modulating the attention matrix of individual functional connections using spatial connections based on prior knowledge, the attention-modulated weights can be learned to construct individual connections adaptively, thereby mitigating individual differences. Besides, a deep node-graph representation learning module is designed to extract long-range interaction characteristics among channels and alleviate the over-smoothing problem of representations. Furthermore, a graph domain co-regularized learning module is imposed to tackle the individual distribution discrepancies in connection and representations across different domains. Extensive experiments on three public EEG emotion datasets, i.e., SEED, DREAMER, and MPED, validate the superior performance of AdamGraph compared with state-of-the-art methods.

The asymmetry of emotional responses is observed in electroencephalogram (EEG) of different frequency bands across various spatial brain regions in neuroscience research. Many prior works have primarily emphasized the dependencies among channels in the spatial domain, neglecting the dynamic interaction of EEG in both spatial and frequency domains, which may limit the performance of EEG emotion recognition. To address these issues, we propose the dual-domain attention based adaptive graph convolutional network (DDA-AGCN) for EEG emotion recognition. Specifically, we propose the lightweight dual-domain attention mechanism (DDA) based on random vector similarity measurement and the squeezeexcitation technique to capture important characteristics in the channel and frequency domain respectively. Furthermore, the adaptive graph convolutional network (AGCN) is utilized to adaptively filter and refine low signal-to-noise ratio EEG data, while also learning the dynamic connectivity patterns among important EEG channels and extracting higher-level abstract features for emotion recognition tasks. To validate the effectiveness of the proposed method, experimental comparisons were conducted on SEED, SEED-IV, and MPED. The experimental results show that our method achieves highly competitive classification performance compared to existing methods. Moreover, under fair comparison, the DDA demonstrates better performance and computational efficiency than self-attention.

Cross-subject EEG emotion recognition suffers a major setback due to high inter-subject variability in emotional responses. Many prior studies have endeavored to alleviate the inter-subject discrepancies of EEG feature distributions, ignoring the variable EEG connectivity and prediction deviation caused by individual differences, which may cause poor generalization to the unseen subject. This paper proposes a graph domain disentanglement network (GDDN) to generalize EEG emotion recognition across subjects in terms of EEG connectivity, representation, and prediction. More specifically, a graph domain disentanglement module is proposed to extract common-specific characteristics on both EEG graph connectivity and graph representation, enabling a more comprehensive network transferability to the unseen individual. Meanwhile, to strengthen stable emotion prediction capability, a domain-adaptive classifier aggregation module is developed to facilitate adaptive emotional prediction for the unseen individual conditioned on the domain weights of the input individuals. Finally, an auxiliary supervision module is imposed to alleviate the domain discrepancy and reduce information loss during the disentanglement learning. Extensive experiments on three public EEG emotion datasets, i.e., SEED, SEED-IV, and MPED, validate the superior generalizability of GDDN compared with the state-of-the-art methods.

EEG emotion recognition has been hampered by the clear individual differences in the electroencephalogram (EEG). Nowadays, domain adaptation is a good way to deal with this issue because it aligns the distribution of data across subjects. However, the performance for EEG emotion recognition is limited by the existing research, which mainly focuses on the global alignment between the source domain and the target domain and ignores much fine-grained information. In this study, we propose a method called Graph-based Unsupervised Subdomain Adaptation (Gusa), which simultaneously aligns the distribution between the source and target domains in a fine-grained way from both the channel and emotion subdomains. Gusa employs three modules, such as the Node-wise Domain Constraints Module to align each EEG channel and obtain a domain-variant representation, the Class-level Distribution Constraints Module, and the Emotion-wise Domain Constraints Module, to collect more fine-grained information, create more discriminative representations for each emotion, and lessen the impact of noisy emotion labels. The studies on the SEED, SEED-IV, and MPED datasets demonstrate that Gusa significantly improves the ability of EEG to recognize emotions and can extract more granular and discriminative representations for EEG.

Graph neural networks (GNNs) have been widely applied in various decision-crucial fields, where accurate predictions with high interpretability are desired. Thus, numerous post-hoc explainers for GNNs have been proposed. However, some prioritize human-intelligible explanations through graph rules, such as the connection rule, which undermines the explanation’s faithfulness to the model. This paper proposes an innovative method, FACExplainer, that re-examines the role of spatial information within GNNs for generating model-faithful explanations. FACExplainer employs activation maps from the last graph convolution to narrow down a compact search space. Our approach further identifies the subgraph that maximizes mutual information as the explanation, eliminating the need for domain-specific knowledge about the downstream task. Empirical analysis of FACExplainer on seven benchmark datasets with three classical GNNs reveals significantly improved explanation quality while consuming less time when compared to leading explainers. The source code of FACExplainer is freely available at https://github.com/HuaYangttt/Facexplainer/.

When a multi-modal affective analysis model generalizes from a bimodal task to a trimodal or multi-modal task, it is usually transformed into a hierarchical fusion model based on every two pairwise modalities, similar to a binary tree structure. This easily leads to large growth in model parameters and computation as the number of modalities increases, which limits the model's generalization. Moreover, many multi-modal fusion methods ignore that different modalities contribute differently to affective analysis. To tackle these challenges, this article proposes a general multi-modal fusion model that supports trimodal or multi-modal affective analysis tasks, called Multi-modal Interactive Attention Network (MIA-Net). Instead of treating different modalities equally, MIA-Net takes the modality that contributes the most to emotion as the main modality and the others as auxiliary modalities. MIA-Net introduces multi-modal interactive attention modules to adaptively select the important information of each auxiliary modality one by one to improve the main-modal representation. Moreover, MIA-Net enables quick generalization to trimodal or multi-modal tasks through stacking multiple MIA modules, which maintains efficient training and only requires linear computation and stable parameter counts. Experimental results of the transfer, generalization, and efficiency experiments on the widely-used datasets demonstrate the effectiveness and generalization of the proposed method.

Emotion recognition based on text-audio modalities is the core technology for transforming a graphical user interface into a voice user interface, and it plays a vital role in natural human-computer interaction systems. Currently, mainstream multimodal learning research has designed various fusion strategies to learn intermodality interactions but hardly considers that not all modalities play equal roles in emotion recognition. Therefore, the main challenge in multimodal emotion recognition is how to implement effective fusion algorithms based on the auxiliary structure. To address this problem, this article proposes an adaptive interactive attention network (AIA-Net). In AIA-Net, text is treated as a primary modality, and audio is an auxiliary modality. AIA-Net adapts to textual and acoustic features with different dimensions and learns their dynamic interactive relations in a more flexible way. The interactive relations are encoded as interactive attention weights to focus on the acoustic features that are effective for textual emotional representations. AIA-Net performs well in adaptively assisting the textual emotional representation with the acoustic emotional information. Moreover, multiple collaborative learning (co-learning) layers of AIA-Net achieve multiple multimodal interactions and the deep bottom-up evolution of emotional representations. Experimental results on three benchmark datasets demonstrate the great effectiveness of the proposed method over the state-of-the-art methods.

An innovative quantification method for 18O labeled LC-MS data is proposed based on I-Ching divination evolutionary algorithm(IDEA). Considering label efficiency for calculating the least squares regression function, traditional methods based on genetic algorithm(GA) or other optimized algorithms will bring high level of computation complexity. The proposed method applies very flexible I-Ching operators(ICOs)— intrication operator, turnover operator, and mutual operator. The objective is the function of determining coefficients, which include the 18O/16O ratio r, the label efficiency f, and the abundance a of 16O. Comparing with GA, the proposed algorithm can significantly improve the accuracy and precision of peptide ratio measurements and better performs in the evolution procedure over mathematically calculating the function. Simultaneously we run the experiment with mix peptide raw data of predefined ratio. The result shows that our proposal algorithm is superior to the conventional GA in exploring optimum solution for better quantification accuracy.