Motor imagery (MI) is a popular noninvasive brain computer interface (BCI) paradigm, yet its decoding accuracy remains hindered by the inherent nonstationarity and low signal-to-noise ratio of electroencephalogram (EEG) signals. Current decoding frameworks often fail to fully exploit the intricate spatial-temporal dependencies, leading to suboptimal feature representation and the omission of latent discriminative cues. To address these challenges, we introduce a deep neural network-powered multifaceted strategy (DPMS-Net) model, a novel approach that employs dynamic convolution to unearth effective discriminative cues across multiple dimensions, including the temporal, spatial, and frequency domains. This model synergizes channel and temporal attention mechanisms to adeptly capture the salient features of EEG signals across diverse spatial-temporal dimensions, thereby mitigating the risk of omitting critical information. Furthermore, we introduce a spectral-domain analysis component that unearths subtle oscillatory signatures hidden within the EEG spectrum, providing enriched evidence for classification. We evaluated the performance of DPMS-Net on two publicly available datasets and a self-collected dataset from stroke patients. On the BCI Competition IV 2a and BCI Competition IV 2b datasets, DPMS-Net achieved subject-dependent classification accuracies of 83.93% and 88.38%, respectively, alongside subject-independent classification accuracies of 65.88% and 76.01%. In the stroke patient dataset, DPMS-Net attained a subject-dependent classification accuracy of 67.67% and a subject-independent classification accuracy of 57.58%. Experimental results indicate that DPMS-Net possesses efficient decoding capabilities and robust stability, reflecting its potential for deployment in neurorehabilitation BCI systems.

Steady state visual evoked potential (SSVEP)-based brain-computer interfaces have been widely studied for their fast response speeds and high information transfer rates. However, how to fully utilize the potential information of existing subjects to realize the mining of common information among different subjects and then realize the information migration in a small amount of data scenarios is a difficult problem faced by current research. In order to solve the above problems, this study proposes a deep neural network based on the pyramid squeeze attention (PSA-DNN) mechanism to enhance the performance of SSVEP-BCI through common information migration. Specifically, the band-pass filtered EEG signals were first Fourier transformed to obtain the frequency domain information; subsequently, the frequency domain information is input into a deep neural network, followed by a spatial convolution step to extract spatial domain information. In order to further enhance the quality of information extraction, a pyramid attention module is introduced into the network to realize the enhancement of frequency domain and spatial domain information. Time domain information from the EEG signals is then mined using temporal convolution. Finally, the full connectivity layer is used to output the recognition results. The model is trained in a three-stage stepped approach for SSVEP target recognition. The first stage uses data from all participants in the training set for common information learning and transfers the model parameters trained in the first stage to the network model in the second stage. In the second stage, some of the information from participants in the test set is used for fine-tuning and to mine personalized information from these new participants. The third stage uses the remaining data from participants in the test set to produce classification results. The proposed method is systematically evaluated using the Benchmark and BETA datasets, where it demonstrates favorable performance compared to established baselines. These findings contribute theoretical insights and methodological References for the application of SSVEP-based brain-computer interfaces in real-world scenarios.

Motor imagery (MI) is one of the most widely used paradigms in brain–computer interfaces (BCIs), known for its ability to trigger changes in brain activity without the need for an external “cue” stimulus. This unique characteristic has attracted significant attention from neuroscientists and researchers in fundamental science. However, compared toP300 and steady-state visual evoked potential (SSVEP), neural activity related to MI tends to be less stable and exhibits substantial variability between individuals. Consequently, accurately decoding MI, using both traditional machine learning and deep learning, has proven to be a considerable challenge. Moreover, given the difficulty of acquiring electroencephalography (EEG) data and the high data demands of deep learning, enhancing the accuracy of MI decoding with limited sample sizes remains a pressing issue that urgently needs to be addressed. This article addresses the challenges mentioned above by introducing a novel deep neural network designed for accurate MI decoding, which is designed to be effective with both small-sample sizes and larger datasets. This network, named the multiscale pooling spatial–temporal attention network (MPSTANet), integrates mix pooling techniques with spatial–temporal attention mechanisms. MPSTANet first employs local and global spatial attention, along with multiscale temporal attention, to thoroughly extract spatial–temporal information from EEG signals. Next, MPSTANet utilizes feature fusion and the proposed mix pooling technique to preserve as much of the extracted spatial–temporal information as possible. Finally, channel interaction attention (CIA) and 3-D weight attention (3-DWA) are employed to recalibrate the weights of the fused channels and spatial–temporal features, respectively. To validate the performance of our proposed MPSTANet model, we conducted experiments on four public datasets, including both small-sample sizes and subject-independent scenarios. MPSTANet achieved cross-session decoding accuracies of 84.82%, 72.92%, 88.20%, and 46.54% on the BCI Competition IV 2a dataset, the Open BMI dataset, the BCI Competition IV 2b dataset, and the PhysioNet dataset, respectively. Furthermore, MPSTANet demonstrated a significant lead compared to other deep learning models in both small-sample and subject-independent experiments. These results demonstrate the robustness of MPSTANet in MI decoding and its promising potential for BCI applications.

Brain-computer interface (BCI) technology enables the control of external devices by recognizing user intentions. Steady-state visual evoked potential (SSVEP)-based BCI technology has been widely applied in the field of Internet of Things (IoT) device control, including smart healthcare, smart homes, and robotics, and has achieved significant results. However, as the field of BCI-based IoT device control is still in its development stage, there remains considerable room for improvement in terms of accuracy, efficiency, and cost. Therefore, enhancing the classification accuracy of SSVEP decoding using a short time window, reducing both human and material costs, and improving work efficiency are crucial for the theoretical research and engineering applications of BCI technology in IoT device control. Based on this, we propose a novel approach to address the challenge of high-accuracy feature extraction within brief timeframes. Our approach integrates a multiscale convolutional neural network with a squeeze excitation module (SEMSCNN). This fusion leverages convolutional neural networks (CNNs)’ local feature learning capacity and the advantageous feature importance distinction offered by the squeeze excitation mechanism. First, the electroencephalogram signals are band-pass filtered into distinct frequency bands and frequency band and channel features are extracted by a two-layer convolution. Then, temporal features are extracted via a multibranch convolution of different scales. Finally, the squeeze and excitation (SE) module is introduced to learn the interdependence between features to improve the quality of the extracted features. The first stage of training exploits statistical commonalities across research participants by learning the global model, and the second stage fine-tunes each participant’s features separately by exploiting participant-specific differences in features. We evaluate our SEMSCNN model on two large public datasets, Benchmark and BETA, and we compare our model to other state-of-the-art models in order to evaluate the effectiveness of our proposed network. Our experimental results indicate that our method effectively improves the accuracy of target recognition and information transfer rate under short-duration stimuli, showing a significant advantage compared to other baseline methods. This provides a broad prospect for the practical application of BCIs in the field of IoT.

Event-related potentials (ERPs) reflect neurophysiological changes of the brain in response to external events and their associated underlying complex spatiotemporal feature information is governed by ongoing oscillatory activity within the brain. Deep learning methods have been increasingly adopted for ERP-based brain-computer interfaces (BCIs) due to their excellent feature representation abilities, which allow for deep analysis of oscillatory activity within the brain. Features with higher spatiotemporal frequencies usually represent detailed and localized information, while features with lower spatiotemporal frequencies usually represent global structures. Mining EEG features from multiple spatiotemporal frequencies is conducive to obtaining more discriminative information. A multiscale feature fusion octave convolution neural network (MOCNN) is proposed in this article. MOCNN divides the ERP signals into high-, medium- and low-frequency components corresponding to different resolutions and processes them in different branches. By adding mid- and low-frequency components, the feature information used by MOCNN can be enriched, and the required amount of calculations can be reduced. After successive feature mapping using temporal and spatial convolutions, MOCNN realizes interactive learning among different components through the exchange of feature information among branches. Classification is accomplished by feeding the fused deep spatiotemporal features from various components into a fully connected layer. The results, obtained on two public datasets and a self-collected ERP dataset, show that MOCNN can achieve state-of-the-art ERP classification performance. In this study, the generalized concept of octave convolution is introduced into the field of ERP-BCI research, which allows effective spatiotemporal features to be extracted from multiscale networks through branch width optimization and information interaction at various scales.

Optimal feature extraction for multi-category motor imagery brain-computer interfaces (MI-BCIs) is a research hotspot. The common spatial pattern (CSP) algorithm is one of the most widely used methods in MI-BCIs. However, its performance is adversely affected by variance in the operational frequency band and noise interference. Furthermore, the performance of CSP is not satisfactory when addressing multi-category classification problems. In this work, we propose a fusion method combining Filter Banks and Riemannian Tangent Space (FBRTS) in multiple time windows. FBRTS uses multiple filter banks to overcome the problem of variance in the operational frequency band. It also applies the Riemannian method to the covariance matrix extracted by the spatial filter to obtain more robust features in order to overcome the problem of noise interference. In addition, we use a One-Versus-Rest support vector machine (OVR-SVM) model to classify multi-category features. We evaluate our FBRTS method using BCI competition IV dataset 2a and 2b. The experimental results show that the average classification accuracy of our FBRTS method is 77.7% and 86.9% in datasets 2a and 2b respectively. By analyzing the influence of the different numbers of filter banks and time windows on the performance of our FBRTS method, we can identify the optimal number of filter banks and time windows. Additionally, our FBRTS method can obtain more distinctive features than the filter banks common spatial pattern (FBCSP) method in two-dimensional embedding space. These results show that our proposed method can improve the performance of MI-BCIs.

The common spatial patterns (CSP) algorithm is one of the most frequently used and effective spatial filtering methods for extracting relevant features for use in motor imagery brain-computer interfaces (MI-BCIs). However, the inherent defect of the traditional CSP algorithm is that it is highly sensitive to potential outliers, which adversely affects its performance in practical applications. In this work, we propose a novel feature optimization and outlier detection method for the CSP algorithm. Specifically, we use the minimum covariance determinant (MCD) to detect and remove outliers in the dataset, then we use the Fisher score to evaluate and select features. In addition, in order to prevent the emergence of new outliers, we propose an iterative minimum covariance determinant (IMCD) algorithm. We evaluate our proposed algorithm in terms of iteration times, classification accuracy and feature distribution using two BCI competition datasets. The experimental results show that the average classification performance of our proposed method is 12% and 22.9% higher than that of the traditional CSP method in two datasets ([Formula: see text]), and our proposed method obtains better performance in comparison with other competing methods. The results show that our method improves the performance of MI-BCI systems.

The common spatial pattern (CSP) algorithm is a well-recognized spatial filtering method for feature extraction in motor imagery (MI)-based brain-computer interfaces (BCIs). However, due to the influence of nonstationary in electroencephalography (EEG) and inherent defects of the CSP objective function, the spatial filters, and their corresponding features are not necessarily optimal in the feature space used within CSP. In this work, we design a new feature selection method to address this issue by selecting features based on an improved objective function. Especially, improvements are made in suppressing outliers and discovering features with larger interclass distances. Moreover, a fusion algorithm based on the Dempster-Shafer theory is proposed, which takes into consideration the distribution of features. With two competition data sets, we first evaluate the performance of the improved objective functions in terms of classification accuracy, feature distribution, and embeddability. Then, a comparison with other feature selection methods is carried out in both accuracy and computational time. Experimental results show that the proposed methods consume less additional computational cost and result in a significant increase in the performance of MI-based BCI systems.

Brain-computer music interfaces (BCMIs) may be used to modulate affective states, with applications in music therapy, composition, and entertainment. However, for such systems to work they need to be able to reliably detect their user's current affective state. We present a method for personalised affective state detection for use in BCMI. We compare it to a population-based detection method trained on 17 users and demonstrate that personalised affective state detection is significantly (p <; 0.01) more accurate, with average improvements in accuracy of 10.2 percent for valence and 9.3 percent for arousal. We also compare a hybrid BCMI (a BCMI that combines physiological signals with neurological signals) to a conventional BCMI design (one based upon the use of only EEG features) and demonstrate that the hybrid design results in a significant (p <; 0.01) 6.2 percent improvement in performance for arousal classification and a significant (p <; 0.01) 5.9 percent improvement for valence classification.

Affectively driven algorithmic composition (AAC) is a rapidly growing field that exploits computer-aided composition in order to generate new music with particular emotional qualities or affective intentions. An AAC system was devised in order to generate a stimulus set covering nine discrete sectors of a two-dimensional emotion space by means of a 16-channel feed-forward artificial neural network. This system was used to generate a stimulus set of short pieces of music, which were rendered using a sampled piano timbre and evaluated by a group of experienced listeners who ascribed a two-dimensional valence-arousal coordinate to each stimulus. The underlying musical feature set, initially drawn from the literature, was subsequently adjusted by amplifying or attenuating the quantity of each feature in order to maximize the spread of stimuli in the valence-arousal space before a second listener evaluation was conducted. This process was repeated a third time in order to maximize the spread of valence-arousal coordinates ascribed to the generated stimulus set in comparison to a spread taken from an existing prerated database of stimuli, demonstrating that this prototype AAC system is capable of creating short sequences of music with a slight improvement on the range of emotion found in a stimulus set comprised of real-world, traditionally composed musical excerpts.

Brain-computer music interfaces (BCMI) provide a method to modulate an individuals affective state via the selection or generation of music according to their current affective state. Potential applications of such systems may include entertainment of therapeutic applications. We outline a proposed design for such a BCMI and seek a method for automatically differentiating different music induced affective states. Band-power features are explored for use in automatically identifying music-induced affective states. Additionally, a linear discriminant analysis classifier and a support vector machine are evaluated with respect to their ability to classify music induced affective states from the electroencephalogram recorded during a BCMI calibration task. Accuracies of up to 79.5% (p <; 0.001) are achieved with the support vector machine.

Affective algorithmic composition is a growing field that combines perceptually motivated affective computing strategies with novel music generation. This article presents work toward the development of one application. The long-term goal is to develop a responsive and adaptive system for inducing affect that is both controlled and validated by biophysical measures. Literature documenting perceptual responses to music identifies a variety of musical features and possible affective correlations, but perceptual evaluations of these musical features for the purposes of inclusion in a music generation system are not readily available. A discrete feature, rhythmic density (a function of note duration in each musical bar, regardless of tempo), was selected because it was shown to be well-correlated with affective responses in existing literature. A prototype system was then designed to produce controlled degrees of variation in rhythmic density via a transformative algorithm. A two-stage perceptual evaluation of a stimulus set created by this prototype was then undertaken. First, listener responses from a pairwise scaling experiment were analyzed via Multidimensional Scaling Analysis (MDS). The statistical best-fit solution was rotated such that stimuli with the largest range of variation were placed across the horizontal plane in two dimensions. In this orientation, stimuli with deliberate variation in rhythmic density appeared farther from the source material used to generate them than from stimuli generated by random permutation. Second, the same stimulus set was then evaluated according to the order suggested in the rotated two-dimensional solution in a verbal elicitation experiment. A Verbal Protocol Analysis (VPA) found that listener perception of the stimulus set varied in at least two commonly understood emotional descriptors, which might be considered affective correlates of rhythmic density. Thus, these results further corroborate previous studies wherein musical parameters are monitored for changes in emotional expression and that some similarly parameterized control of perceived emotional content in an affective algorithmic composition system can be achieved and provide a methodology for evaluating and including further possible musical features in such a system. Some suggestions regarding the test procedure and analysis techniques are also documented here.

The use of statistical models and statistical inference for characterizing the interplay between brain structures and human behavior (functional brain mapping) is common in neuroscience. Statistical methods, however, require the availability of sufficiently large data sets. As a result, experimental paradigms used to collect behavioral trials from individuals are data centered and not user centered. This means that experimental paradigms are tuned to collect as many trials as possible, are generally rather demanding, and are not always motivating or engaging for individuals. Subject cooperation and their compliance with the task may decrease over time. Whenever possible, paradigms are designed to control for factors such as fatigue, attention, and motivation. In this paper, we propose the use of the Kinect motion tracking sensor (Microsoft, Inc., Redmond, WA, USA) in a game-based paradigm for noninvasive electroencephalogram (EEG)-based functional motor mapping. Results from an experimental study with able-bodied subjects playing a virtual ball game suggest that the Kinect sensor is useful for isolating specific movements during the interaction with the game, and that the computed EEG patterns for hand and feet movements are in agreement with results described in the literature.