Autism spectrum disorder (ASD) is a complex neurodevelopmental disorder with marked impairments in neural system functioning. Electroencephalography (EEG) offers a promising approach to investigate the neurophysiological basis of ASD, however, most EEG studies in ASD focus on spontaneous brain activity. Emotional processing deficits are a core feature of ASD, but related connectivity patterns remain underexplored due to challenges in data collection and analysis. This study investigates functional brain connectivity differences between children with ASD (n = 32) and typically developing (TD) children (n = 32) across five frequency bands and four connectivity indices. We designed an SVM-MRMR pipeline to classify ASD and TD children using these features. Our findings reveal that ASD children exhibit more coordinated intra-brain networks and oscillatory patterns in the high-frequency range. Additionally, they show an increased number of long-range connections in the Theta band, particularly between the left and right hemispheres. ASD children also demonstrate increased frontal lobe connectivity during positive emotions and heightened temporal lobe activity during negative emotions. Functional connectivity under positive and negative emotional clips achieved a classification accuracy exceeding 85%. These findings suggest that functional connectivity derived from portable EEG devices may serve as a potential biomarker for diagnosing and classifying ASD in real-world applications.

EEG emotion recognition is crucial in both human-machine interaction and healthcare. However, recognizing emotions across different subjects remains challenging due to individual variability. While existing multi-source domain adaptation methods have been utilized for cross-subject EEG emotion decoding, they often struggle with irrelevant or weakly relevant source domains, leading to negative transfer. Additionally, variations within subdomains are often neglected in these studies. We propose a joint domain adaptation method, Adaptive Source Joint Domain Adaptation (ASJDA) to address these issues. ASJDA utilizes an unsupervised adaptive source selection strategy to select a subset of source domains by evaluating the Jensen-Shannon divergence between the source and target domains, choosing those most relevant to the target. Subsequently, it implements joint domain adaptation with these chosen sources at both the domain and category subdomain levels. Our proposed method outperforms existing state-of-the-art methods, achieving cross-subject accuracies of 96.81% in SEED, 89.69% in SEED-IV, and 69.31% in DEAP. This work significantly advances the state of the art in EEG emotion recognition by effectively addressing the challenges of cross-subject variability.

EEG (Electroencephalography)-based emotion recognition has emerged as a crucial area of research due to its potential applications in mental health, brain-computer interfaces (BCIs), and affective computing. However, the inherent variability in EEG signals across individuals, coupled with limited dataset sizes, significantly hinders the development of robust and generalizable emotion recognition models. To overcome these challenges, we propose the Sparse Bayesian Learning with Enhanced Covariance Alignment (SBLECA) algorithm. SBLECA formulates cross-subject emotion recognition as an end-to-end decoding problem, integrating spatiotemporal filtering and classification within a sparse Bayesian learning (SBL) framework. Crucially, SBLECA incorporates a novel covariance alignment technique to mitigate inter-subject variability in EEG patterns. Rigorous evaluations on two publicly available emotion datasets demonstrate that SBLECA consistently outperforms state-of-the-art methods. Furthermore, SBLECA offers valuable insights into the neural correlates of emotion through interpretable visualizations of learned spatial and temporal filters. SBLECA holds promise as a valuable EEG decoding tool to advance the development and translation of neurotechnologies and biomarkers for brain disorders.

Multimodal Emotion Recognition in Conversations (MERC) is an important topic in human-computer interaction. In the MERC task, conversations exhibit dynamic emotional dependency, including inter-speaker and intra-speaker emotional dependency, both are vital in understanding the content. However, current research primarily integrates these two emotional dependencies into one unified module, limiting the accuracy of MERC. In this paper, we propose a dynamic emotion-dependent network with relational subgraph interaction named DEDNet. DEDNet introduces relational subgraphs to separately model two emotional dependencies, enabling structured learning paths for utterances based on distinct emotional dependency types. Specifically, nodes indicate the utterances at different moments in the conversation, while edges define the emotional dependency and temporal relationships between nodes. To explicitly capture the differences between these two emotional dependencies, distinct subgraphs are designed for comprehensive representations. Furthermore, we propose an incremental interactive strategy, sequentially leveraging two emotional dependencies to learn the changes in dependency relationships. We find that modeling inter-speaker emotional dependency can better identify negative emotions and modeling intra-speaker emotional dependency can better recognize positive emotions. Experimental results demonstrate that our model outperforms current state-of-the-art methods on three benchmark datasets, IEMOCAP, MELD and DailyDialog.

Emotion recognition from EEG signals offers significant advantages in affective computing, as EEG more accurately reflects internal emotional states than other modalities, such as facial expressions or peripheral physiological signals. Modeling and capturing subtle affective changes over time is crucial for real-world applications to achieve better human-computer interaction. However, training such models usually requires segment-level emotion labels, which are costly and may not be feasible. Assigning the overall label to all EEG segments within a trial can lead to inaccurate model training and degraded performance, as emotions evolve continuously. This highlights the need for models capable of learning from trial-wise emotion labels while capturing temporal dynamics of emotional responses within each segment because trial-wise post-stimulus labels are more accessible. To this end, we propose EmotionMIL, an end-to-end EEG-based emotion recognition framework that leverages recent advances in deep multiple instance learning (MIL). This framework enables robust emotion recognition from weakly labeled EEG signals and identifies the most prominent emotional responses. EmotionMIL captures the temporal dynamics of emotions using a retentive self-attention mechanism, which adaptively assigns weights to EEG segments based on their relevance in predicting the overall emotion label. A pseudo-bag augmentation strategy is also introduced to enhance the model's generalization ability by generating additional pseudo-bags from the original ones. Evaluated on three benchmark datasets—DEAP, DREAMER, and SEED—EmotionMIL outperforms state-of-the-art non-MIL and MIL models in both subject-dependent and subject-independent tasks, achieving superior accuracy and F1-score. Ablation study further validates the model design, while visualization results demonstrate that EmotionMIL effectively identifies both spatial EEG patterns and temporal emotional dynamics. These findings underscore EmotionMIL's potential for robust, interpretable emotion recognition, paving the way for real-world applications in emotion-aware systems. The code is available athttps://github.com/yuty2009/emotionmil.

Electroencephalography (EEG) source imaging (ESI) methods aim to reconstruct cortical sources from scalp EEG signals, a crucial task for understanding the normal brain as well as brain disorders. Traditional model-driven ESI methods face challenges in real-time reconstruction, while deep neural network (DNN)-based ESI methods often struggle with generalization to new data. To address these issues, we propose ADMM-ESINet, a novel deep unfolding neural network for robust and efficient reconstruction of EEG extended sources. ADMM-ESINet leverages a structured sparsity constraint within a regularization framework and employs the Alternating Direction Method of Multipliers (ADMM) to achieve iterative solutions. By unrolling the ADMM algorithm into a cascaded network architecture, ADMM-ESINet effectively integrates prior knowledge, enabling end-to-end, real-time ESI. Crucially, both the regularization parameters and the spatial transform operator are learned directly from the training data. Numerical results demonstrate that ADMM-ESINet surpasses traditional DNN-based methods in generalization ability and accurately reconstructs the location, extent, and temporal dynamics of extended sources, establishing ADMM-ESINet as a promising method for real-time ESI.

OBJECTIVE: Accurate decoding of electroencephalogram (EEG) signals has become more significant for the brain-computer interface (BCI). Specifically, motor imagery and motor execution (MI/ME) tasks enable the control of external devices by decoding EEG signals during imagined or real movements. However, accurately decoding MI/ME signals remains a challenge due to the limited utilization of temporal information and ineffective feature selection methods. METHODS: This paper introduces DMSACNN, an end-to-end deep multiscale attention convolutional neural network for MI/ME-EEG decoding. DMSACNN incorporates a deep multiscale temporal feature extraction module to capture temporal features at various levels. These features are then processed by a spatial convolutional module to extract spatial features. Finally, a local and global feature fusion attention module is utilized to combine local and global information and extract the most discriminative spatiotemporal features. MAIN RESULTS: DMSACNN achieves impressive accuracies of 78.20%, 96.34% and 70.90% for hold-out analysis on the BCI-IV-2a, High Gamma and OpenBMI datasets, respectively, outperforming most of the state-of-the-art methods. CONCLUSION AND SIGNIFICANCE: These results highlight the potential of DMSACNN in robust BCI applications. Our proposed method provides a valuable solution to improve the accuracy of the MI/ME-EEG decoding, which can pave the way for more efficient and reliable BCI systems.

Electroencephalography (EEG)-based emotion recognition plays a key role in the development of affective brain-computer interfaces (BCIs). However, emotions are complex and extracting salient EEG features underlying distinct emotional states is inherently limited by low signal-to-noise ratio (SNR) and low spatial resolution of practical EEG data, which is further compounded by the lack of effective spatio-temporal filter optimization approaches for generic EEG features. To address these challenges, this study proposes a set of neural networks termed the Filter-Bank Spatio-Temporal Convolutional Networks (FBSTCNets) for performing end-to-end multi-class emotion recognition via robust extraction of power and/or connectivity features from EEG. First, a filter bank is employed to construct a multiview spectral representation of EEG data. Next, a temporal convolutional layer, followed by a depth-wise spatial convolutional layer, performs spatio-temporal filtering, transforming EEG into latent signals with higher SNR. A feature extraction layer then extracts power and/or connectivity features from the latent signals. Finally, a fully connected layer with a cropped decoding strategy predicts the emotional state. Experimental results on two public emotion EEG datasets, SEED and SEED-IV, demonstrate that FBSTCNets outperform previous benchmark methods in decoding accuracy. Our approach provides a principled emotion decoding framework for designing high-performance spatio-temporal filtering networks tailored to specific EEG feature types. The FBSTCNet source code is available athttps://github.com/TimeSpacerRob/FBSTCNet.

OBJECT: Transformer-based neural networks have been applied to the electroencephalography (EEG) decoding for motor imagery (MI). However, most networks focus on applying the self-attention mechanism to extract global temporal information, while the cross-frequency coupling features between different frequencies have been neglected. Additionally, effectively integrating different neural networks poses challenges for the advanced design of decoding algorithms. METHODS: This study proposes a novel end-to-end Multi-Scale Vision Transformer Neural Network (MSVTNet) for MI-EEG classification. MSVTNet first extracts local spatio-temporal features at different filtered scales through convolutional neural networks (CNNs). Then, these features are concatenated along the feature dimension to form local multi-scale spatio-temporal feature tokens. Finally, Transformers are utilized to capture cross-scale interaction information and global temporal correlations, providing more distinguishable feature embeddings for classification. Moreover, auxiliary branch loss is leveraged for intermediate supervision to ensure the effective integration of CNNs and Transformers. RESULTS: The performance of MSVTNet was assessed through subject-dependent (session-dependent and session-independent) and subject-independent experiments on three MI datasets, i.e., the BCI competition IV 2a, 2b and OpenBMI datasets. The experimental results demonstrate that MSVTNet achieves state-of-the-art performance in all analyses. CONCLUSION: MSVTNet shows superiority and robustness in enhancing MI decoding performance.

Electromagnetic source imaging (ESI) requires solving a highly ill-posed inverse problem. To seek a unique solution, traditional ESI methods impose various forms of priors that may not accurately reflect the actual source properties, which may hinder their broad applications. To overcome this limitation, in this article, a novel data-synthesized spatiotemporally convolutional encoder-decoder network (DST-CedNet) method is proposed for ESI. The DST-CedNet recasts ESI as a machine learning problem, where discriminative learning and latent-space representations are integrated in a CedNet to learn a robust mapping from the measured electroencephalography/magnetoencephalography (E/MEG) signals to the brain activity. In particular, by incorporating prior knowledge regarding dynamical brain activities, a novel data synthesis strategy is devised to generate large-scale samples for effectively training CedNet. This stands in contrast to traditional ESI methods where the prior information is often enforced via constraints primarily aimed for mathematical convenience. Extensive numerical experiments as well as analysis of a real MEG and epilepsy EEG dataset demonstrate that the DST-CedNet outperforms several state-of-the-art ESI methods in robustly estimating source signals under a variety of source configurations.

Decoding brain activity from non-invasive electroencephalography (EEG) is crucial for brain-computer interfaces (BCIs) and the study of brain disorders. Notably, end-to-end EEG decoding has gained widespread popularity in recent years owing to the remarkable advances in deep learning research. However, many EEG studies suffer from limited sample sizes, making it difficult for existing deep learning models to effectively generalize to highly noisy EEG data. To address this fundamental limitation, this paper proposes a novel end-to-end EEG decoding algorithm that utilizes a low-rank weight matrix to encode both spatio-temporal filters and the classifier, all optimized under a principled sparse Bayesian learning (SBL) framework. Importantly, this SBL framework also enables us to learn hyperparameters that optimally penalize the model in a Bayesian fashion. The proposed decoding algorithm is systematically benchmarked on five motor imagery BCI EEG datasets ( N=192) and an emotion recognition EEG dataset ( N=45), in comparison with several contemporary algorithms, including end-to-end deep-learning-based EEG decoding algorithms. The classification results demonstrate that our algorithm significantly outperforms the competing algorithms while yielding neurophysiologically meaningful spatio-temporal patterns. Our algorithm therefore advances the state-of-the-art by providing a novel EEG-tailored machine learning tool for decoding brain activity.

Emotion regulation plays a vital role in human beings daily lives by helping them deal with social problems and protects mental and physical health. However, objective evaluation of the efficacy of emotion regulation and assessment of the improvement in emotion regulation ability at the individual level remain challenging. In this study, we leveraged neurofeedback training to design a real-time EEG-based brain-computer interface (BCI) system for users to effectively regulate their emotions. Twenty healthy subjects performed 10 BCI-based neurofeedback training sessions to regulate their emotion towards a specific emotional state (positive, negative, or neutral), while their EEG signals were analyzed in real time via machine learning to predict their emotional states. The prediction results were presented as feedback on the screen to inform the subjects of their immediate emotional state, based on which the subjects could update their strategies for emotion regulation. The experimental results indicated that the subjects improved their ability to regulate these emotions through our BCI neurofeedback training. Further EEG-based spectrum analysis revealed how each emotional state was related to specific EEG patterns, which were progressively enhanced through long-term training. These results together suggested that long-term EEG-based neurofeedback training could be a promising tool for helping people with emotional or mental disorders.

Simultaneously estimating brain source activity and noise has long been a challenging task in electromagnetic brain imaging using magneto- and electroencephalography. The problem is challenging not only in terms of solving the NP-hard inverse problem of reconstructing unknown brain activity across thousands of voxels from a limited number of sensors, but also for the need to simultaneously estimate the noise and interference. We present a generative model with an augmented leadfield matrix to simultaneously estimate brain source activity and sensor noise statistics in electromagnetic brain imaging (EBI). We then derive three Bayesian inference algorithms for this generative model (expectation-maximization (EBI-EM), convex bounding (EBI-Convex) and fixed-point (EBI-Mackay)) to simultaneously estimate the hyperparameters of the prior distribution for brain source activity and sensor noise. A comprehensive performance evaluation for these three algorithms is performed. Simulations consistently show that the performance of EBI-Convex and EBI-Mackay updates is superior to that of EBI-EM. In contrast to the EBI-EM algorithm, both EBI-Convex and EBI-Mackay updates are quite robust to initialization, and are computationally efficient with fast convergence in the presence of both Gaussian and real brain noise. We also demonstrate that EBI-Convex and EBI-Mackay update algorithms can reconstruct complex brain activity with only a few trials of sensor data, and for resting-state data, achieving significant improvement in source reconstruction and noise learning for electromagnetic brain imaging.

Reconstructing complex brain source activity at a high spatiotemporal resolution from magnetoencephalography (MEG) or electroencephalography (EEG) remains a challenging problem. Adaptive beamformers are routinely deployed for this imaging domain using the sample data covariance. However adaptive beamformers have long been hindered by 1) high degree of correlation between multiple brain sources, and 2) interference and noise embedded in sensor measurements. This study develops a novel framework for minimum variance adaptive beamformers that uses a model data covariance learned from data using a sparse Bayesian learning algorithm (SBL-BF). The learned model data covariance effectively removes influence from correlated brain sources and is robust to noise and interference without the need for baseline measurements. A multiresolution framework for model data covariance computation and parallelization of the beamformer implementation enables efficient high-resolution reconstruction images. Results with both simulations and real datasets indicate that multiple highly correlated sources can be accurately reconstructed, and that interference and noise can be sufficiently suppressed. Reconstructions at 2-2.5mm resolution ( ∼ 150K voxels) are possible with efficient run times of 1-3 minutes. This novel adaptive beamforming algorithm significantly outperforms the state-of-the-art benchmarks. Therefore, SBL-BF provides an effective framework for efficiently reconstructing multiple correlated brain sources with high resolution and robustness to interference and noise.

Tracking the 6-degree-of-freedom (6D) object pose in video sequences is gaining attention because it has a wide application in multimedia and robotic manipulation. However, current methods often perform poorly in challenging scenes, such as incorrect initial pose, sudden re-orientation, and severe occlusion. In contrast, we present a robust 6D object pose tracking method with a novel hierarchical feature fusion network, refer it as HFF6D, which aims to predict the object’s relative pose between adjacent frames. Instead of extracting features from adjacent frames separately, HFF6D establishes sufficient spatial-temporal information interaction between adjacent frames. In addition, we propose a novel subtraction feature fusion (SFF) module with attention mechanism to leverage feature subtraction during feature fusion. It explicitly highlights the feature differences between adjacent frames, thus improving the robustness of relative pose estimation in challenging scenes. Besides, we leverage data augmentation technology to make HFF6D be used more effectively in the real world by training only with synthetic data, thereby reducing manual effort in data annotation. We evaluate HFF6D on the well-known YCB-Video and YCBInEOAT datasets. Quantitative and qualitative results demonstrate that HFF6D outperforms state-of-the-art (SOTA) methods in both accuracy and efficiency. Moreover, it is also proved to achieve high-robustness tracking under the above-mentioned challenging scenes.

The phenomenon of increasing accidents caused by reduced vigilance does exist. In the future, the high accuracy of vigilance estimation will play a significant role in public transportation safety. We propose a multimodal regression network that consists of multichannel deep autoencoders with subnetwork neurons (MCDAE$_{sn}$). After we define two thresholds of “0.35” and “0.70” from the percentage of eye closure, the output values are in the continuous range of 0–0.35, 0.36–0.70, and 0.71–1 representing the awake state, the tired state, and the drowsy state, respectively. To verify the efficiency of our strategy, we first applied the proposed approach to a single modality. Then, for the multimodality, since the complementary information between forehead electrooculography and electroencephalography features, we found the performance of the proposed approach using features fusion significantly improved, demonstrating the effectiveness and efficiency of our method.

Accurate electroencephalogram (EEG) pattern decoding for specific mental tasks is one of the key steps for the development of brain-computer interface (BCI), which is quite challenging due to the considerably low signal-to-noise ratio of EEG collected at the brain scalp. Machine learning provides a promising technique to optimize EEG patterns toward better decoding accuracy. However, existing algorithms do not effectively explore the underlying data structure capturing the true EEG sample distribution and, hence, can only yield a suboptimal decoding accuracy. To uncover the intrinsic distribution structure of EEG data, we propose a clustering-based multitask feature learning algorithm for improved EEG pattern decoding. Specifically, we perform affinity propagation-based clustering to explore the subclasses (i.e., clusters) in each of the original classes and then assign each subclass a unique label based on a one-versus-all encoding strategy. With the encoded label matrix, we devise a novel multitask learning algorithm by exploiting the subclass relationship to jointly optimize the EEG pattern features from the uncovered subclasses. We then train a linear support vector machine with the optimized features for EEG pattern decoding. Extensive experimental studies are conducted on three EEG data sets to validate the effectiveness of our algorithm in comparison with other state-of-the-art approaches. The improved experimental results demonstrate the outstanding superiority of our algorithm, suggesting its prominent performance for EEG pattern decoding in BCI applications.

Achieving high classification performance in electroencephalogram (EEG)-based brain-computer interfaces (BCIs) often entails a large number of channels, which impedes their use in practical applications. Despite the previous efforts, it remains a challenge to determine the optimal subset of channels in a subject-specific manner without heavily compromising the classification performance. In this article, we propose a new method, called spatiotemporal-filtering-based channel selection (STECS), to automatically identify a designated number of discriminative channels by leveraging the spatiotemporal information of the EEG data. In STECS, the channel selection problem is cast under the framework of spatiotemporal filter optimization by incorporating a group sparsity constraints, and a computationally efficient algorithm is developed to solve the optimization problem. The performance of STECS is assessed on three motor imagery EEG datasets. Compared with state-of-the-art spatiotemporal filtering algorithms using full EEG channels, STECS yields comparable classification performance with only half of the channels. Moreover, STECS significantly outperforms the existing channel selection methods. These results suggest that this algorithm holds promise for simplifying BCI setups and facilitating practical utility.

With the rapid development of the Internet, social media is affecting and changing people’s lives. The research of network community structure based on a large number of complex network data sets is increasingly popular. Due to the large scale of existing social network data and privacy issues, it is hard to analyze the entire network data directly. Therefore, a reliable and effective network sampling method is very important for the actual estimation of online social networks attributes. Existing network sampling methods like metropolis–hasting random walk (MHRW) can obtain unbiased sample sets from relatively large-scale social networks such as Facebook and describe the key features of the original network. Moreover, MHRW uses a proposed distribution function for sampling control, which can guarantee a well-balanced nature of resulting Markov chain. However, MHRW has the defect of partial graph over entry. In this paper, we proposed a new hybrid jump (HJ) sample by introducing an HJ strategy into MHRW during the sampling progress. First, we use a breadth-first search to obtain a data set without repeated node quickly from a list of jump nodes. Moreover, we adopted uniform sample (UNI) to get the average degree of the original network. Then, a 3-D average degree distribution model is designed to determine the optimal value of the jump parameter in HJ. Finally, we set the condition to execute the HJ strategy in each step of sampling progress. The experimental results demonstrate the performance of HJ is better than the other representation sampling methods both in strong-tie networks and weak-tie networks.

Recent years have witnessed brain-computer interface (BCI) as a promising technology for integrating human intelligence and machine intelligence. Currently, event-related potential (ERP)-based BCI is an important branch of noninvasive electroencephalogram (EEG)-based BCIs. Extracting ERPs from a limited number of trials remains challenging due to their low signal-to-noise ratio (SNR) and low spatial resolution caused by volume conduction. In this paper, we propose a probabilistic model for trial-by-trial concatenated EEG, in which the concatenated ERPs are expressed as a linear combination of a set of discrete sine and cosine bases. The bases are simply determined by the data length of a single trial. A sparse prior on the rank of the spatio-spectral pattern matrix is introduced into the model to allow the number of components to be automatically determined. A maximum posterior estimation algorithm based on cyclic descent is then developed to estimate the spatiospectral patterns. A spatial filter can then be obtained by maximizing the SNR of the ERP components. Experiments on both synthetic data and real N170 ERP from 13 subjects were conducted to test the efficacy and efficiency of the algorithm. The results showed that the proposed algorithm can estimate the ERPs more accurately than the several state-of-the-art algorithms.

Despite rapid advances in the study of brain–computer interfaces (BCIs) in recent decades, two fundamental challenges, namely, improvement of target detection performance and multidimensional control, continue to be major barriers for further development and applications. In this paper, we review the recent progress in multimodal BCIs (also called hybrid BCIs), which may provide potential solutions for addressing these challenges. In particular, improved target detection can be achieved by developing multimodal BCIs that utilize multiple brain patterns, multimodal signals, or multisensory stimuli. Furthermore, multidimensional object control can be accomplished by generating multiple control signals from different brain patterns or signal modalities. Here, we highlight several representative multimodal BCI systems by analyzing their paradigm designs, detection/control methods, and experimental results. To demonstrate their practicality, we report several initial clinical applications of these multimodal BCI systems, including awareness evaluation/detection in patients with disorder of consciousness (DOC). As an evolving research area, the study of multimodal BCIs is increasingly requiring more synergetic efforts from multiple disciplines for the exploration of the underlying brain mechanisms, the design of new effective paradigms and means of neurofeedback, and the expansion of the clinical applications of these systems.

For M/EEG-based distributed source imaging, it has been established that the L2-norm-based methods are effective in imaging spatially extended sources, whereas the L1-norm-based methods are more suited for estimating focal and sparse sources. However, when the spatial extents of the sources are unknown a priori, the rationale for using either type of methods is not adequately supported. Bayesian inference by exploiting the spatio-temporal information of the patch sources holds great promise as a tool for adaptive source imaging, but both computational and methodological limitations remain to be overcome. In this paper, based on state-space modeling of the M/EEG data, we propose a fully data-driven and scalable algorithm, termed STRAPS, for M/EEG patch source imaging on high-resolution cortices. Unlike the existing algorithms, the recursive penalized least squares (RPLS) procedure is employed to efficiently estimate the source activities as opposed to the computationally demanding Kalman filtering/smoothing. Furthermore, the coefficients of the multivariate autoregressive (MVAR) model characterizing the spatial-temporal dynamics of the patch sources are estimated in a principled manner via empirical Bayes. Extensive numerical experiments demonstrate STRAPS's excellent performance in the estimation of locations, spatial extents and amplitudes of the patch sources with varying spatial extents.

Common spatial patterns (CSP) is a well-known spatial filtering algorithm for multichannel electroencephalogram (EEG) analysis. In this paper, we cast the CSP algorithm in a probabilistic modeling setting. Specifically, probabilistic CSP (P-CSP) is proposed as a generic EEG spatio-temporal modeling framework that subsumes the CSP and regularized CSP algorithms. The proposed framework enables us to resolve the overfitting issue of CSP in a principled manner. We derive statistical inference algorithms that can alleviate the issue of local optima. In particular, an efficient algorithm based on eigendecomposition is developed for maximum a posteriori (MAP) estimation in the case of isotropic noise. For more general cases, a variational algorithm is developed for group-wise sparse Bayesian learning for the P-CSP model and for automatically determining the model size. The two proposed algorithms are validated on a simulated data set. Their practical efficacy is also demonstrated by successful applications to single-trial classifications of three motor imagery EEG data sets and by the spatio-temporal pattern analysis of one EEG data set recorded in a Stroop color naming task.

For hyperspectral image (HSI) classification, it is challenging to adopt the methodology of sparse-representation-based classification. In this paper, we first propose an l1-minimization-based spectral-spatial classification method for HSIs via a spatial translation-invariant wavelet (STIW)-based sparse representation (STIW-SR), wherein both the spectrum dictionary and the analyzed signal are formed with STIW features. Due to the capability of a STIW to reduce both the observation noise and the spatial nonstationarity while maintaining the ideal spectra, which is proved with our signal-interference-noise spectrum model involved, it is expected that the pixels in the same class congregate in a lower dimensional subspace, and the separations among class-specific subspaces are enhanced, thus yielding a highly discriminative sparse representation. Then, we develop an approach to evaluate the sparsity recoverability of an l1-minimization on HSIs in a probabilistic framework. This approach takes into account not only the recovery probability under the given support length of the l0-norm solution but also the apriori probability of the support length; consequently, it overcomes the inability of traditional mutual/cumulative coherence conditions to address high-coherence HSIs. This paper reveals that the higher sparsity recoverability of a STIW-SR leads to its higher classification accuracy and that the increasing coherence does not necessarily lead to a reduced sparsity recovery probability, and this paper verifies the connection between l0and l1-minimizations on HSIs. Experimental results from realworld HSIs suggest that our classification method significantly outperforms several representative spectral-spatial classifiers and support vector machines.

Learning optimal spatio-temporal filters is a key to feature extraction for single-trial electroencephalogram (EEG) classification. The challenges are controlling the complexity of the learning algorithm so as to alleviate the curse of dimensionality and attaining computational efficiency to facilitate online applications, e.g., brain-computer interfaces (BCIs). To tackle these barriers, this paper presents a novel algorithm, termed regularized spatio-temporal filtering and classification (RSTFC), for single-trial EEG classification. RSTFC consists of two modules. In the feature extraction module, an l2 -regularized algorithm is developed for supervised spatio-temporal filtering of the EEG signals. Unlike the existing supervised spatio-temporal filter optimization algorithms, the developed algorithm can simultaneously optimize spatial and high-order temporal filters in an eigenvalue decomposition framework and thus be implemented highly efficiently. In the classification module, a convex optimization algorithm for sparse Fisher linear discriminant analysis is proposed for simultaneous feature selection and classification of the typically high-dimensional spatio-temporally filtered signals. The effectiveness of RSTFC is demonstrated by comparing it with several state-of-the-arts methods on three brain-computer interface (BCI) competition data sets collected from 17 subjects. Results indicate that RSTFC yields significantly higher classification accuracies than the competing methods. This paper also discusses the advantage of optimizing channel-specific temporal filters over optimizing a temporal filter common to all channels.

In numerous neuroscience studies, multichannel EEG data are often recorded over multiple trial periods under the same experimental condition. To date, little effort is aimed to learn spatial patterns from EEG data to account for trial-to-trial variability. In this paper, a hierarchical Bayesian framework is introduced to model inter-trial source variability while extracting common spatial patterns under multiple experimental conditions in a supervised manner. We also present a variational Bayesian algorithm for model inference, by which the number of sources can be determined effectively via automatic relevance determination (ARD). The efficacy of the proposed learning algorithm is validated with both synthetic and real EEG data. Using two brain-computer interface (BCI) motor imagery data sets we show the proposed algorithm consistently outperforms the common spatial patterns (CSP) algorithm while attaining comparable performance with a recently proposed discriminative approach.