Emotional neuromodulation refers to the direct manipulation of the nervous system using techniques such as electrical or magnetic stimulation to manage and adjust an individual's emotional experiences. Transcranial electrical stimulation (tES) targeting the right ventrolateral prefrontal cortex (rVLPFC) has been widely used to modulate emotions. However, the impact of emotions on brain network changes and modulation during tES remains unclear. In this study, we developed a subject-adaptive dynamic graph convolution network with fused features (FusSADGCNN) to decode the impact of tES on neuromodulation for emotion recognition and emotion elicitation. Specifically, we developed a fused feature, CPE, which integrates the average sub-frequency phase-locking value representing global functional connectivity with differential entropy characterizing local activation to explore network differences across emotional states, while incorporating an improved dynamic graph convolution to adaptively integrate multi-receptive neighborhood information for precise decoding of individual tES effects. On the SEED dataset and our laboratory data, the FusSADGCNN model outperforms the state-of-the-art methods. Furthermore, we utilized these tools to assess the emotional modulation states induced by tES. Results indicated that in the experiment involving music-elicited emotional modulation, the tools effectively identified improvements in negative emotions under true stimulation, with predictive accuracy significantly related to the average connectivity strength of the brain network. In the active facial emotion recognition modulation experiment, jointed stimulation of rVLPFC and temporo-parietal junction achieved better modulation effects. These findings highlight that the FusSADGCNN effectively evaluate the neuromodulation states during tES-induced emotional regulation, providing a reliable foundation for integrating emotion recognition and neuromodulation.

The complexity of Alzheimer's disease (AD) is influenced by population heterogeneity, prompting the study of subtypes through imaging phenotypes. However, the role of neural morphological heterogeneity and the correlated gene expression (CGE) in AD subtypes remains unclear. This study links cortical thickness deviations of AD subtypes with CGE connectivity patterns. Transcriptional activity was measured based on the Allen Human Brain Atlas, and a density-based clustering algorithm identified AD subtypes. Subtype 1 mainly exhibits cortical thinning, while subtype 2 shows cortical thickening. Using whole-brain gene expression data, we found that deviations in transcriptionally connected neighboring regions predicted regional deviations in both subtypes. Gene enrichment analysis revealed that epicenter regions were associated with biological processes like synaptic dysfunction and phosphorylation regulation. Our study establishes associations between CGE connectivity and neural morphological alterations, identifies distinct epicenter regions in AD subtypes, and provides novel insights into how molecular-level gene expression shapes subtype-specific pathology.

Sleep staging is essential for assessing sleep quality and diagnosing disorders. Current deep learning models exhibit limitations in spatial feature modeling, cross-frequency coupling awareness, contextual modeling, and stage transition dynamics. We propose a novel deep neural architecture integrating multi-band and multimodal modeling. It includes Frequency-Time-Spatial Convolution (FTSC) with wavelet decompositions for multi-scale frequency features,$1 \times 1$convolutions for crossfrequency coupling, and spatiotemporal convolutions for local patterns. Enhanced by Multimodal Bottleneck Transformer (MBT) and Epoch Transformer (ET) for cross-modal semantics and a Conditional Random Field (CRF) for physiologically plausible transitions, our approach demonstrates state-of-the-art performance and robustness in experiments on multiple public datasets.

Significant progress has been made in the study of Alzheimer’s disease (AD) and Mild Cognitive Impairment(MCI) progression using multimodal approaches. Recent studies have identified peripheral blood gene expression data as valuable biomarkers for distinguishing AD and MCI progression subtypes. However, these studies either rely entirely on prior data for gene selection or are purely data-driven. These strategies are not conducive to discovering potentially important genes or may lead to results unrelated to brain neural function pathways. This study adopts a data-driven approach based on prior knowledge, using genes mapped onto morphologically different brain regions as features for selection. These features are then input into our Generative Adversarial Network framework to obtain attention masks for cortical morphological indicators, which are weighted and used in training the structural MRI feature extraction main network. Our method ensures that the selected genes are correlated with brain regions and group differences, and through post-hoc interpretability analysis, we identify potential biomarkers in both genes and brain regions across two modalities.

Sleep staging plays a crucial role in assessing sleep quality and diagnosing sleep-related disorders. Although previous studies have attempted to recognize sleep stages using automatic classification methods, achieving relatively high classification accuracy, most current automatic staging algorithms mainly rely on unimodal data and are limited to time-domain or frequency-domain features. To overcome these limitations, this study proposes a deep neural network model based on a multimodal fusion state-space sequence mechanism (AdaptiveSleepNet), designed to utilize multimodal signals for automatic sleep stage classification. AdaptiveSleepNet integrates a masking module, Adaptive Feature Recalibration (AFR) module, and State Space Model(SSM). The masking module is capable of masking missing modality signals. Then, features are extracted from multiple frequency bands of the signal through the Adaptive Feature Recalibration module, and channel feature weights are obtained using the residual Squeeze-and-Excitation (SE) attention mechanism. Next, the state-space sequence coupling module is used to learn cross-modal sequence relationships between signals. Finally, the fully connected layer outputs the sleep stage classification results. Evaluation of the Sleep-EDF-20 and Sleep-EDF-78 datasets shows that AdaptiveSleepNet achieved a classification accuracy of 84.74% and 82.34% through five-fold cross-subject classification with the EMG signal masked. Experimental results demonstrate that the AdaptiveSleepNet model proposed in this study effectively optimizes the performance of sleep staging and outperforms current state-of-the-art methods in staging performance.

Sleep staging is a critical component of sleep assessment and holds significant value for disease diagnosis. Although existing research has made notable progress in terms of performance, there are still shortcomings in the interpretability of algorithmic decision-making and outcomes. This "black-box" issue is particularly concerning in high-risk applications such as medical decision-making. Therefore, this paper proposes an interpretable hybrid deep learning model that incorporates expert knowledge for sleep stage classification. First, we employ a Morlet wavelet-based kernel layer to separate features of different frequencies in the sleep data. This layer is designed based on the visual analysis principles used by human experts in polysomnography(PSG) recordings. Next, we utilize Multi-Scale Group Convolution(MSGC) to extract and reduce the dimensionality of the time series corresponding to different frequency bands. Subsequently, a Transformer encoder is applied to model the contextual information within the time series. Finally, a multimodal bilinear pooling method is adopted to fuse features from different modalities, enabling effective interaction between modalities. Compared with previous studies, the proposed end-to-end model not only demonstrates excellent performance but also provides visualizations aligned with expert knowledge, thereby enhancing interpretability.

Infrared and visible image fusion (IVIF) is increasingly applied in critical fields such as video surveillance and autonomous driving systems. Significant progress has been made in deep learning-based fusion methods. However, these models frequently encounter out-of-distribution (OOD) scenes in real-world applications, which severely impact their performance and reliability. Therefore, addressing the challenge of OOD data is crucial for the safe deployment of these models in open-world environments. Unlike existing research, our focus is on the challenges posed by OOD data in real-world applications and on enhancing the robustness and generalization of models. In this paper, we propose an infrared-visible fusion framework based on Multi-View Augmentation. For external data augmentation, Top-k Selective Vision Alignment is employed to mitigate distribution shifts between datasets by performing RGB-wise transformations on visible images. This strategy effectively introduces augmented samples, enhancing the adaptability of the model to complex real-world scenarios. Additionally, for internal data augmentation, self-supervised learning is established using Weak-Aggressive Augmentation. This enables the model to learn more robust and general feature representations during the fusion process, thereby improving robustness and generalization. Extensive experiments demonstrate that the proposed method exhibits superior performance and robustness across various conditions and environments. Our approach significantly enhances the reliability and stability of IVIF tasks in practical applications.

Accurate decoding in electroencephalography (EEG) technology, particularly for rapid visual stimuli, remains challenging due to the low signal-to-noise ratio (SNR). Additionally, existing neural networks struggle with issues related to generalization and interpretability. This article proposes a cross-modal aligned network, E2IVAE, which leverages shared information from multiple modalities for self-supervised alignment of EEG to images for extracting visual perceptual information and features a novel EEG encoder, ISTANet, based on algorithm unrolling. This network framework significantly enhances the accuracy and stability of EEG decoding for object recognition in novel classes while reducing the extensive neural data typically required for training neural decoders. The proposed ISTANet employs algorithm unrolling to transform the multilayer sparse coding algorithm into an end-to-end format, extracting features from noisy EEG signals while incorporating the interpretability of traditional machine learning. The experimental results demonstrate that our method achieves SOTA top-1 accuracy of 62.39% and top-5 accuracy of 88.98% on a comprehensive rapid serial visual presentation (RSVP) dataset for public comparison in a 200-class zero-shot neural decoding task. Additionally, ISTANet enables visualization and analysis of multiscale atom features and overall reconstruction features, exploring biological plausibility across temporal, spatial, and spectral dimensions. On another more challenging RSVP large-scale dataset, the proposed framework also achieves significantly above chance-level performance, proving its robustness and generalization. This research provides critical insights into neural decoding and brain-computer interfaces (BCIs) within the fields of cognitive science and artificial intelligence.

Efficiently recognizing kilometer markers in railway systems is crucial for ensuring the safety and reliability of train operations, particularly within the Industrial Internet of Things (IIoT) framework where edge devices are often resource-constrained. This paper highlights the significance of a real-time AI-driven approach to railway kilometer marker recognition. We introduce an innovative method that employs spatio-temporal slimming and multi-view fusion techniques, elevating both precision and computational efficiency for real-time analytics in IIoT. Our approach begins with the implementation of an Adaptive Region of Interest (AROI) and a spatio-temporal calibration mechanism for effective marker detection in real-time. Furthermore, a multi-view fusion method addresses challenges such as occlusion, blurriness , and low-light conditions, common in real-world industrial environments, which includes a variation-aware memory bank for constructing informative views and a fusion network. Experimental results demonstrate the effectiveness of our method in a real-world rail transportation setting, significantly enhancing the accuracy and efficiency of kilometer marker recognition, thereby contributing to the safety and operational efficiency of rail systems.

Granger causality-based effective brain connectivity provides a powerful tool to probe the neural mechanism for information processing and the potential features for brain computer interfaces. However, in real applications, traditional Granger causality is prone to the influence of outliers, such as inevitable ocular artifacts, resulting in unreasonable brain linkages and the failure to decipher inherent cognition states. In this work, motivated by constructing the sparse causality brain networks under the strong physiological outlier noise conditions, we proposed a dual Laplacian Granger causality analysis (DLap-GCA) by imposing Laplacian distributions on both model parameters and residuals. In essence, the first Laplacian assumption on residuals will resist the influence of outliers in electroencephalogram (EEG) on causality inference, and the second Laplacian assumption on model parameters will sparsely characterize the intrinsic interactions among multiple brain regions. Through simulation study, we quantitatively verified its effectiveness in suppressing the influence of complex outliers, the stable capacity for model estimation, and sparse network inference. The application to motor-imagery (MI) EEG further reveals that our method can effectively capture the inherent hemispheric lateralization of MI tasks with sparse patterns even under strong noise conditions. The MI classification based on the network features derived from the proposed approach shows higher accuracy than other existing traditional approaches, which is attributed to the discriminative network structures being captured in a timely manner by DLap-GCA even under the single-trial online condition. Basically, these results consistently show its robustness to the influence of complex outliers and the capability of characterizing representative brain networks for cognition information processing, which has the potential to offer reliable network structures for both cognitive studies and future brain-computer interface (BCI) realization.

Liang Xu, Hai Hu, Xuanwei Zhang, Lu Li, Chenjie Cao, Yudong Li, Yechen Xu, Kai Sun, Dian Yu, Cong Yu, Yin Tian, Qianqian Dong, Weitang Liu, Bo Shi, Yiming Cui, Junyi Li, Jun Zeng, Rongzhao Wang, Weijian Xie, Yanting Li, Yina Patterson, Zuoyu Tian, Yiwen Zhang, He Zhou, Shaoweihua Liu, Zhe Zhao, Qipeng Zhao, Cong Yue, Xinrui Zhang, Zhengliang Yang, Kyle Richardson, Zhenzhong Lan. Proceedings of the 28th International Conference on Computational Linguistics. 2020.

In this paper, we present FigExplorer, a novel general system that supports the retrieval and exploration of research article figures. Specifically, FigExplorer can support 1) figure retrieval using keyword queries, 2) exploration of related figures of a given figure, 3) exploration of a figure topic using the citation network, and 4) search result re-ranking using an example figure. The different functions were implemented using either classical IR models or neural network-based figure embeddings. Finally, the system was designed to facilitate the collection of user data for training and test purposes and it is flexible enough such that it can be extended to include new functions and algorithms. As an open-source system, FigExplorer can help advance the research, evaluation, and development of applications in this area.

Motor imagery (MI) requires subjects to visualize the requested motor behaviors, which involves a large-scale network that spans multiple brain areas. The corresponding cortical activity reflected on the scalp is characterized by event-related desynchronization (ERD) and then by event-related synchronization (ERS). However, the network mechanisms that account for the dynamic information processing of MI during the ERD and ERS periods remain unknown. Here, we combined ERD/ERS analysis with the dynamic networks in different MI stages (i.e. motor preparation, ERD and ERS) to probe the dynamic processing of MI information. Our results show that specific dynamic network structures correspond to the ERD/ERS evolution patterns. Specifically, ERD mainly shows the contralateral networks, while ERS has the symmetric networks. Moreover, different dynamic network patterns are also revealed between the two types of MIs, in which the left-hand MIs exhibit a relatively less sustained contralateral network, which may be the network mechanism that accounts for the bilateral ERD/ERS observed for the left-hand MIs. Similar to the network topologies, the three MI stages also appear to be characterized by different network properties. The above findings all demonstrate that different MI stages that involve specific brain networks for dynamically processing the MI information.

This paper formulates the bandwidth allocation problem in train-to-ground wireless communication networks in operational process of trains. It is shown that the Nash Bargaining game provides an Asymmetric Nash Bargaining Solution which is fair to bandwidth allocation problems in different services. We proposed a bandwidth allocation model for train-to-ground communication system. It can effectively reflect allocation strategies for services with different bargaining power. We define a dynamic adaptive function of bargaining power in order to match utility functions under diverse bandwidths. Easily to implement as it is, an algorithm for bandwidth allocation is derived and then simulated. The simulation show that the proposed scheme has high rate of resource utilization, and is suitable for train-to-ground communication systems.

Magnetic sensors can be applied in vehicle recognition. Most of the existing vehicle recognition algorithms use one sensor node to measure a vehicle‖s signature. However, vehicle speed variation and environmental disturbances usually cause errors during such a process. In this paper we propose a method using multiple sensor nodes to accomplish vehicle recognition. Based on the matching result of one vehicle‖s signature obtained by different nodes, this method determines vehicle status and corrects signature segmentation. The co-relationship between signatures is also obtained, and the time offset is corrected by such a co-relationship. The corrected signatures are fused via maximum likelihood estimation, so as to obtain more accurate vehicle signatures. Examples show that the proposed algorithm can provide input parameters with higher accuracy. It improves the average accuracy of vehicle recognition from 94.0% to 96.1%, and especially the bus recognition accuracy from 77.6% to 92.8%.