Visual neural decoding not only aids in elucidating the neural mechanisms underlying the processing of visual information but also facilitates the advancement of brain-computer interface technologies. However, most current decoding studies focus on developing separate decoding models for individual subjects and specific tasks, an approach that escalates training costs and consumes a substantial amount of computational resources. This paper introduces a Prompt-Guided Generative Visual Language Decoding Model (PG-GVLDM), which uses prompt text that includes information about subjects and tasks to decode both primary categories and detailed textual descriptions from the visual response activities of multiple individuals. In addition to visual response activities, this study also incorporates a multi-head cross-attention module and feeds the model with whole-brain response activities to capture global semantic information in the brain. Experiments on the Natural Scenes Dataset (NSD) demonstrate that PG-GVLDM attains an average category decoding accuracy of 66.6% across four subjects, reflecting strong cross-subject generalization, and achieves text decoding scores of 0.342 (METEOR), 0.450 (Sentence-Transformer), 0.283 (ROUGE-1), and 0.262 (ROUGE-L), establishing state-of-the-art performance in text decoding. Furthermore, incorporating whole-brain response activities significantly enhances decoding performance by enabling the integration of distributed neural signals into coherent global semantic representations, underscoring its methodological importance for unified neural decoding. This research not only represents a breakthrough in visual neural decoding methodologies but also provides theoretical and technical support for the development of generalized brain-computer interfaces.

Exploring how local perturbations of cortical activity propagate across the brain network not only helps us understanding causal mechanisms of brain networks, but also offers a network insight into neurobiological mechanisms for transcranial magnetic stimulation (TMS) treatment response. The concurrent combination of TMS and electroencephalography (EEG) enables researchers to track the TMS-evoked activity, defined here as scalp-recorded electrical signals reflecting the brain's response to TMS, with millisecond-level temporal resolution. Based on this technique, we proposed a quantitative framework which combined sparse non-negative matrix factorization and stage-dependent effective connectivity methods to infer the causal pathways in TMS-evoked brain responses. We found that single-pulse TMS firstly induces local activity in the directly stimulated regions (left primary motor cortex, M1), and then propagates to the contralateral hemisphere and other brain regions. Finally, it propagates back from the contralateral region (right M1) to the stimulation region (left M1). This study provides preliminary evidence demonstrating how local perturbations propagate through brain networks to influence various cortical regions, and offers insights into the neural mechanism of TMS-evoked brain responses from a network perspective.

Emotion recognition based on electroencephalogram (EEG) is fundamentally associated with human-like intelligence system. However, due to the noise-sensitive characteristics of EEGs and the individual variability of emotions, it is very challenging to extract inherent emotion dependent patterns from emotional EEG signals. In this work, we propose a L1-norm space defined discriminative brain network manifold learning model (L1-SGL), in which the EEG noise outliers can be effectively separated and the pseudolabeled samples caused by subjective feelings can be automatically corrected. Off-line experimental results consistently indicate that the L1-SGL can effectively suppress the influence of noise and achieve an incomparable superiority performance over other existing methods in EEG emotion recognition. Besides, benefiting from the time efficiency of the L1-SGL, an online emotion monitoring and regulation system is further implemented in this work. On-line emotion decoding experimental results (86.30%) of 25 participants prove that the L1-SGL can effectively satisfy the real-time requirements of on-line emotional monitoring applications, and the significant negative emotion regulation experimental results ( $p \lt 0.001$ ) further confirm the feasibility and effectiveness of L1-SGL model in real-time emotion regulation and interactive applications. Overall, the L1-SGL provides a promising solution for the real-time online affective brain-computer interfaces (aBCIs) and the intelligent clinical closed-loop treatments.

Benefiting from the high-temporal resolution of electroencephalogram (EEG), EEG-based emotion recognition has become one of the hotspots of affective computing. For EEG-based emotion recognition systems, it is crucial to utilize state-of-the-art learning strategies to automatically learn emotion-related brain cognitive patterns from emotional EEG signals, and the learned stable cognitive patterns effectively ensure the robustness of the emotion recognition system. In this work, to realize the efficient decoding of emotional EEG, we propose a graph learning system [Graph Convolutional Network framework with Brain network initial inspiration and Fused attention mechanism (BF-GCN)] inspired by the brain cognitive mechanism to automatically learn graph patterns from emotional EEG and improve the performance of EEG emotion recognition. In the proposed BF-GCN, three graph branches, i.e., cognition-inspired functional graph branch, data-driven graph branch, and fused common graph branch, are first elaborately designed to automatically learn emotional cognitive graph patterns from emotional EEG signals. And then, the attention mechanism is adopted to further capture the brain activation graph patterns that are related to emotion cognition to achieve an efficient representation of emotional EEG signals. Essentially, the proposed BF-CGN model is a cognition-inspired graph learning neural network model, which utilizes the spectral graph filtering theory in the automatic learning and extracting of emotional EEG graph patterns. To evaluate the performance of the BF-GCN graph learning system, we conducted subject-dependent and subject-independent experiments on two public datasets, i.e., SEED and SEED-IV. The proposed BF-GCN graph learning system has achieved 97.44% (SEED) and 89.55% (SEED-IV) in subject-dependent experiments, and the results in subject-independent experiments have achieved 92.72% (SEED) and 82.03% (SEED-IV), respectively. The state-of-the-art performance indicates that the proposed BF-GCN graph learning system has a robust performance in EEG-based emotion recognition, which provides a promising direction for affective computing.

Constructing computational decoding models to account for the cortical representation of semantic information plays a crucial role in understanding visual perception. The human visual system processes interactive relationships among different objects when perceiving the semantic contents of natural visions. However, the existing semantic decoding models commonly regard categories as completely separate and independent visually and semantically and rarely consider the relationships from prior information. In this work, a novel semantic graph learning model was proposed to decode multiple semantic categories of perceived natural images from brain activity. The proposed model was validated on the functional magnetic resonance imaging data collected from five normal subjects while viewing 2750 natural images comprising 52 semantic categories. The results showed that the Graph Neural Network-based decoding model achieved higher accuracies than other deep neural network models. Moreover, the co-occurrence probability among semantic categories showed a significant correlation with the decoding accuracy. Additionally, the results suggested that semantic content organized in a hierarchical way with higher visual areas was more closely related to the internal visual experience. Together, this study provides a superior computational framework for multi-semantic decoding that supports the visual integration mechanism of semantic processing.

The directed brain functional network construction gives us the new insights into the relationships between brain regions from the causality point of view. The Granger causality analysis is one of the powerful methods to model the directed network. The complex brain network is also hierarchically constructed, which is particularly suited to facilitate segregated functions and the global integration of the segregated functions. Therefore, it is of great interest to explore new approach to model the hierarchical architecture of the directed network. In the present study, we proposed a new approach, namely, stepwise multivariate Granger causality (SMGC), considering both the directed and hierarchical features of brain functional network to explore the stepwise causal relationship in the network. The simulation study demonstrated that the diverse and complex hierarchical organization could be embedded in the apparently simple directed network. The proposed SMGC method could capture the multiple hierarchy of the directed network. When applying to the real functional magnetic resonance imaging (fMRI) datasets, the core triple resting-state networks in human brain showed within-network directed connections in the first-level directed network and rich and diverse between-network pathways in the second-level hierarchical network. The default mode network (DMN) had a prominent role in the resting-state acting as both the causal source and the important relay station. Further exploratory research on the adaption of directed hierarchical network in athletes suggested the enhanced bidirectional communication between the DMN and the central executive network (CEN) and the enhanced directed connections from the salience network (SN) to the CEN in the athlete group. The SMGC approach is capable of capturing the hierarchical architecture of the brain directed functional network, which refreshes the new stepwise causal relationship in the directed network. This might shed light on the potential application for exploring the altered hierarchical organization of brain directed network in neuropsychiatric disorders.

Multichannel electroencephalogram (EEG) is an array signal that represents brain neural networks and can be applied to characterize information propagation patterns for different emotional states. To reveal these inherent spatial graph features and increase the stability of emotion recognition, we propose an effective emotion recognition model that performs multicategory emotion recognition with multiple emotion-related spatial network topology patterns (MESNPs) by learning discriminative graph topologies in EEG brain networks. To evaluate the performance of our proposed MESNP model, we conducted single-subject and multisubject four-class classification experiments on two public datasets, MAHNOB-HCI and DEAP. Compared with existing feature extraction methods, the MESNP model significantly enhances the multiclass emotional classification performance in the single-subject and multisubject conditions. To evaluate the online version of the proposed MESNP model, we designed an online emotion monitoring system. We recruited 14 participants to conduct the online emotion decoding experiments. The average online experimental accuracy of the 14 participants was 84.56%, indicating that our model can be applied in affective brain-computer interface (aBCI) systems. The offline and online experimental results demonstrate that the proposed MESNP model effectively captures discriminative graph topology patterns and significantly improves emotion classification performance. Moreover, the proposed MESNP model provides a new scheme for extracting features from strongly coupled array signals.

Intrinsic neural activity ubiquitously persists in all physiological states. However, how intrinsic brain activity (iBA) changes over a short time remains unknown. To uncover the brain dynamics' theoretic underpinning, electrophysiological relevance, and neuromodulation, we identified iBA dynamics on simulated data, electroencephalogram-functional magnetic resonance imaging (EEG-fMRI) data, and repetitive transcranial magnetic stimulation (rTMS) fMRI data using sliding-window analysis. The temporal variability (dynamics) of iBA were quantified using the variance of the amplitude of low-frequency fluctuations (ALFF) over time. We first used simulated fMRI data to examine the effects of various parameters including window length, and step size on dynamic ALFF. Second, using EEG-fMRI data, we found that the heteromodal association cortex had the most variable dynamics while the limbic regions had the least, consistent with previous findings. In addition, the temporal variability of dynamic ALFF depended on EEG power fluctuations. Moreover, using rTMS fMRI data, we found that the temporal variability of dynamic ALFF could be modulated by rTMS. Taken together, these results provide evidence about the theory, relevance, and adjustability of iBA dynamics.

Benign epilepsy with centrotemporal spikes (BECTS) is a common childhood epilepsy syndrome associated with abnormalities in neurocognitive domains, particularly during interictal epileptiform discharges (IEDs). Here, we investigated the effects of IEDs on brain's intrinsic connectivity networks in 43 BECTS patients and 28 matched healthy controls (HCs). Patients were further divided into IED and non-IED subgroups based on simultaneous EEG-fMRI recordings. Functional connectivity within and between five networks, corresponding to seizure origination and cognitive processes, were analyzed to measure IED effects. We found that patients exhibited increased connectivity within the auditory network (AN) and the somato-motor network (SMN), and decreased connectivity within the basal ganglia network and the dorsal attention network, suggesting that both transient and chronic seizure activity may disturb normal network organization. The IED group showed decreased functional connectivity within the default mode network (DMN) compared with the non-IED group and HCs, implying that the DMN was selectively impaired during epileptiform discharges associated with altered self-referential cognitive functions. Moreover, the IED group exhibited increased positive correlations between the AN and the SMN, which suggests a possible excessive influence of centrotemporal spiking on information processing in the auditory system. The association between epileptic activity and network dysfunctions highlights their importance in investigating the pathological mechanism underlying BECTS.

Schizophrenia is a severe psychiatry disorder characterized by the positive and negative syndrome. Previous studies often treat the two syndromes independently. In our study, we proposed a multi-task regression and stability selection combined method to explore whether the two syndromes shared same brain functional deficits. 35 drug-naïve adolescent participants with first-episode schizophrenia were recruited from The Second Affiliated Hospital of XinXiang Medical University. A multi-task regression method combining stability selection was proposed and utilized on the resting-state fMRI data of subjects to explore the shared atypical brain regions which contributed to the positive and negative syndromes. Finally we found that default mode network contributed in both positive and negative syndromes of schizophrenia, indicating a shared neural basis of these two syndromes. Overall, we revealed the applicability of multi-task regression method to explore the shared neural deficits of multi syndromes of psychiatry disorder.

Although it is accepted that linear Granger causality can reveal effective connectivity in functional magnetic resonance imaging (fMRI), the issue of detecting nonlinear connectivity has hitherto not been considered. In this paper, we address kernel Granger causality (KGC) to describe effective connectivity in simulation studies and real fMRI data of a motor imagery task. Based on the theory of reproducing kernel Hilbert spaces, KGC performs linear Granger causality in the feature space of suitable kernel functions, assuming an arbitrary degree of nonlinearity. Our results demonstrate that KGC captures effective couplings not revealed by the linear case. In addition, effective connectivity networks between the supplementary motor area (SMA) as the seed and other brain areas are obtained from KGC.

The self-organizing mapping (SOM) and hierarchical clustering (HC) methods are integrated to detect brain functional activation; functional magnetic resonance imaging (fMRI) data are first processed by SOM to obtain a primary merged neural nodes image, and then by HC to obtain further brain activation patterns. The conventional Euclidean distance metric was replaced by the correlation distance metric in SOM to improve clustering and merging of neural nodes. To improve the use of spatial and temporal information in fMRI data, a new spatial distance (node coordinates in the 2-D lattice) and temporal correlation (correlation degree of each time course in the exemplar matrix) are introduced in HC to merge the primary SOM results. Two simulation studies and two in vivo fMRI data that both contained block-design and event-related experiments revealed that brain functional activation can be effectively detected and that different response patterns can be distinguished using these methods. Our results demonstrate that the improved SOM and HC methods are clearly superior to the statistical parametric mapping (SPM), independent component analysis (ICA), and conventional SOM methods in the block-design, especially in the event-related experiment, as revealed by their performance measured by receiver operating characteristic (ROC) analysis. Our results also suggest that the proposed new integrated approach could be useful in detecting block-design and event-related fMRI data.

This paper reports a new delay subspace decomposition (DSD) algorithm. Instead of using the canonical zero-delay correlation matrix, the new DSD algorithm introduces a delay into the correlation matrix of the subspace decomposition to suppress noises in the data. The algorithm is applied to functional magnetic resonance imaging (fMRI) to detect the regions of focal activities in the brain. The efficiency is evaluated by comparing with independent component analysis and principal component analysis method of fMRI.