Electroencephalogram (EEG) signals are inherently non-stationary and exhibit significant inter-subject variability, leading to pronounced cross-subject distribution shifts that hinder accurate emotion recognition. Although graph convolutional networks (GCNs) and domain adaptation (DA) methods have made progress in mitigating individual differences, existing approaches still face two fundamental limitations: (1) traditional GCNs rely on static functional connectivity graphs, which fail to capture the dynamic temporal evolution of neural interactions during emotional processes, and (2) most DA-based methods only emphasize global feature alignment while overlooking emotion-specific semantic structures, thereby impairing both fine-grained discriminability and cross-subject generalization. To overcome these challenges, we propose the Prototypical Contrastive Learning with Temporal Dynamic Graph Convolutional Network (PCL-TDGCN) for EEG-based emotion recognition. Specifically, we construct an adaptive global EEG pattern memory mechanism to model temporally dynamic brain networks, thereby facilitating spatiotemporal neural interactions essential for emotion representation learning. Furthermore, we design a prototypical contrastive learning strategy that incorporates: (i) intra-domain contrastive learning to enhance the discriminability of emotional state representations, and (ii) inter-domain contrastive learning to mitigate distribution shifts across domains via semantic-aware prototypical alignment. Extensive experiments on three public datasets demonstrate that the proposed PCL-TDGCN outperforms state-of-the-art methods, achieving accuracy improvements of 1.08% (SEED), 6.53% (HIED), and 0.98% (SEED-IV) in subject-dependent experiments, and 1.08% (SEED), 7.51% (HIED), and 1.99% (SEED-IV) in subject-independent scenarios, respectively.

The interaction of the brain's decision-making and feedback stages is crucial for guiding human behavior. Previous studies mainly focused on the interaction immediately after the feedback, resulting in a limited understanding of brain communication dynamics during the interaction process. This study examined the communication dynamics of the brain network during decision-feedback interaction under various feedback conditions by employing a newly developed activation network approach to reveal its underlying neural mechanism. Thirty participants completed a decision-feedback task that involved a sequence of cue-induced predictions with highly predictable, somewhat predictable, and unpredictable feedback conditions. We constructed the activation network for all experimental stages using source-level EEG data in the alpha band. Notably, the brain exhibited the highest communication efficiency ($p < 0.05$) in receiving and integrating feedback with decision-making information during the feedback stage. Furthermore, the network-behavior correlations indicated that the brain tends to evaluate unexpected feedback under highly predictable conditions and expected feedback under unpredictable conditions, suggesting distinct neural strategies of the decision-feedback interaction process. Finally, we decoded the optimization process of decision-feedback interaction across the entire task. Although network correlations between the decision and feedback stages decreased over time (high predictable: $r = -0.447$, $p = 0.001$; unpredictable: $r = -0.305$, $p = 0.032$), classification accuracy significantly improved (${r = -0.448}$, $p = 0.010$, best accuracy: 86.667% ) under the highly predictable condition, corresponding with enhanced prediction behavior. These results indicate the optimization process of the cognitive resources allocation that supports more efficient interaction and improved predictive performance. Our findings advance the understanding of the mechanisms of decision-feedback interaction.

Steady-state visual evoked potential-based brain-computer interfaces (SSVEP-BCIs) hold significant promise for enabling high-speed human-computer interaction in real-world scenarios. However, existing frequency-domain decoding methods treat frequency spectrum features (the real and imaginary spectrum features) as a single feature without considering their unique spatial and spectral characteristics, resulting in insufficient generalizable features and limited classification accuracy in cross-subject scenarios. To address this issue, we propose a Dual-Branch Attention-Based Frequency Domain Network (DB-AFDNet) to independently decode real and imaginary spectral components, aiming to acquire more discriminative and generalizable features for cross-subject applications. Specifically, we construct inter-branch attention similarity constraints to encourage the two branches to have similar attention properties, promoting to learn the consensus characteristics in the dual branches. Furthermore, we propose intra-branch orthogonality constraints to explore branch-specific discriminative features to learn generalizable features. Experimental studies on two public datasets, the Benchmark and Beta datasets, demonstrate that DB-AFDNet outperforms state-of-the-art methods in cross-subject classification, achieving a relative improvement of 1.36$\%$ and 1.45$\%$, respectively.

This paper presents MindSpeak, a real-time brain-computer interface (BCI) system for recording, processing, and decoding silent speech to enable online multimodal communication between the human brain and a computer, involving both noninvasive multichannel EEG signals and text output. To enable hand-free and brain-only control, our system incorporates steady-state visual evoked potential (SSVEP) for users to select incomplete sentences from a predefined pool and confirm the correctness of decoded words. An intuitive graphical interface is designed for natural communication. We evaluate the effectiveness of our real-time BCI system, which achieves 77.3% accuracy in decoding silent speech and 98.9% accuracy in SSVEP-based selection and confirmation of correct sentences. Unlike existing BCI systems, the presented MindSpeak system significantly expands the application scope of existing BCI systems by enabling users to express complete thoughts through a fully BCI-controlled interactive interface. Our demonstration video is on: https://youtu.be/B1wt1dmCCrg.

Motor imagery (MI) engages a broad network of brain regions to imagine a specific action. Investigating the mechanism of brain network reorganization during MI after spinal cord injury (SCI) is crucial because it reflects overall brain activity. Using electroencephalogram (EEG) data from SCI patients, we conducted EEG-based coherence analysis to examine different brain network reorganizations across different frequency bands, from resting to MI. Furthermore, we introduced a consistency calculation-based residual graph convolution (C-ResGCN) classification algorithm. The results show that the [Formula: see text]- and [Formula: see text]-band connectivity weakens, and brain activity decreases during the MI task compared to the resting state. In contrast, the [Formula: see text]-band connectivity increases in motor regions while the default mode network activity declines during MI. Our C-ResGCN algorithm showed excellent performance, achieving a maximum classification accuracy of 96.25%, highlighting its reliability and stability. These findings suggest that brain reorganization in SCI patients reallocates relevant brain resources from the resting state to MI, and effective network reorganization correlates with improved MI performance. This study offers new insights into the mechanisms of MI and potential biomarkers for evaluating rehabilitation outcomes in patients with SCI.

Stroke, an abrupt cerebrovascular ailment resulting in brain tissue damage, has prompted the adoption of motor imagery (MI)-based brain–computer interface (BCI) systems in stroke rehabilitation. However, analyzing electroencephalogram (EEG) signals from stroke patients poses challenges. To address the issues of low accuracy and efficiency in EEG classification, particularly involving MI, the study proposes a residual graph convolutional network (M-ResGCN) framework based on the modified S-transform (MST), and introduces the self-attention mechanism into residual graph convolutional network (ResGCN). This study uses MST to extract EEG time-frequency domain features, derives spatial EEG features by calculating the absolute Pearson correlation coefficient (aPcc) between channels, and devises a method to construct the adjacency matrix of the brain network using aPcc to measure the strength of the connection between channels. Experimental results involving 16 stroke patients and 16 healthy subjects demonstrate significant improvements in classification quality and robustness across tests and subjects. The highest classification accuracy reached 94.91% and a Kappa coefficient of 0.8918. The average accuracy and F1 scores from 10 times 10-fold cross-validation are 94.38% and 94.36%, respectively. By validating the feasibility and applicability of brain networks constructed using the aPcc in EEG signal analysis and feature encoding, it was established that the aPcc effectively reflects overall brain activity. The proposed method presents a novel approach to exploring channel relationships in MI-EEG and improving classification performance. It holds promise for real-time applications in MI-based BCI systems.

Due to the inherent non-stationarity and individual differences present in electroencephalogram (EEG) signals, developing a generalizable model that performs well on new subjects is challenging in EEG-based emotion recognition. Most existing domain adaptation (DA) methods typically mitigate these discrepancies by aligning the marginal distributions of domain feature representations. However, when there is a significant difference in the class-conditional distribution between domain features and labels, the domain-invariant features learned by aligning marginal distributions may have limited discriminative ability for unlabeled target instances or even prove counterproductive. To address this issue, we propose a Neighborhood Semantic Aware Learning-based Dynamic Graph Attention Convolution (NSAL-DGAT) approach that learns target semantic information by considering the inter-domain semantic topological structure, thereby improving classifier adaptation for target instances. Specifically, the proposed NSAL framework is designed to capitalize on the insight that after domain feature alignment, some target samples and their neighboring source samples exhibit similar semantics. By leveraging the neighborhood topological structure, we extract and incorporate semantic target features to train a more transferable classifier. Besides, we implement an entropy weighting mechanism to emphasize representative target semantic information, encouraging target instances to prioritize high-confidence individuals within the source neighborhood. We have conducted extensive experiments on the public SEED dataset and our collected the Hearing-Impaired EEG Dataset (HIED). The experimental results underscore the efficacy of our proposed NSAL-DGAT approach, showcasing state-of-the-art accuracy in subject-dependent as well as subject-independent scenarios. The source code is available at https://github.com/YYingDL/NSAL-DGAT.

Objective: Because of the non-stationary nature of electroencephalogram (EEG) signals, traditional non-invasive brain-computer interfaces (BCIs) usually only produce discrete commands, limiting their ability to control external devices continuously. This study proposes a novel BCI control strategy mapping multiple discrete commands to continuous motion, enabling real-time manipulation of a drone in four degrees of freedom (DOF).Methods: Our strategy used the fast steady state visual evoked potential (SSVEP) encoding and decoding method to convert user intentions into the drone’s flight status in near real-time. Simultaneously, the drone’s live video was embedded into the SSVEP stimuli, providing users with a first-person perspective control experience.Results: In drone control experiments, participants successfully maneuvered the drone through complex path-following tasks in simulated and physical scenarios. The mean flight trajectory bias ratio was measured as 0.81, with a mean flight smoothness of -3.31 (measured by spectral arc length) and mean Fitts’s throughput of 9.18 bits/min. Notably, the brain-to-hand ratio (BHR) for all metrics approached 1, indicating that our non-invasive control system achieved comparable performance to manual control systems.Conclusion: These results suggest the effectiveness of our proposed BCI control strategy that maps discrete commands to continuous motion and extends the capabilities of non-invasive BCIs in continuous control scenarios.Significance: This study significantly advances the applications of BCI and propels human-machine interaction towards a more direct realm.

Sleep quality is critical for human well-being. Lack of sleep and poor sleep quality impair daily cognitive functions and health. While stress has been recognized as a detrimental factor on sleep quality, the relationship between pre-sleep stress level, resting EEG and subsequent sleep structure remains to be explored. This study presents a novel approach that evaluates pre-sleep stress levels using a 2-channel EEG to predict slow-wave sleep (SWS) deficiency. We recorded forehead EEG immediately before sleep onset, then utilized power spectra and entropy analysis to extract stress-related neurological features, including beta/delta correlation, alpha asymmetry, fuzzy entropy (FuzzEn), and spectral entropy (SpEn). We found that individuals with SWS deficiency exhibited signs of stress, such as a robust beta/delta correlation, higher alpha asymmetry, and increased FuzzEn. Conversely, individuals with ample SWS displayed weak beta/delta correlation and reduced FuzzEn in EEG recordings. Finally, we tested the robustness of the selected neuro markers with two supervised learning models and found that the selected markers predict SWS deficiency with an accuracy above 70%. Our study demonstrated that stress-related neurological markers derived from pre-sleep EEG can effectively predict SWS deficiency. The proposed method can be integrated with a portable EEG device and sleep-improving interventions to develop a personalized sleep-improvement solution.

Stroke, a sudden cerebrovascular ailment resulting from brain tissue damage, has prompted the use of motor imagery (MI)-based Brain-Computer Interface (BCI) systems in stroke rehabilitation. However, analyzing EEG signals from stroke patients is challenging because of their low signal-to-noise ratio and high variability. Therefore, we propose a novel approach that combines the modified S-transform (MST) and a dense graph convolutional network (DenseGCN) algorithm to enhance the MI-BCI performance across time, frequency, and space domains. MST is a time-frequency analysis method that efficiently concentrates energy in EEG signals, while DenseGCN is a deep learning model that uses EEG feature maps from each layer as inputs for subsequent layers, facilitating feature reuse and hyper-parameters optimization. Our approach outperforms conventional networks, achieving a peak classification accuracy of 90.22% and an average information transfer rate (ITR) of 68.52 bits per minute. Moreover, we conduct an in-depth analysis of the event-related desynchronization/event-related synchronization (ERD/ERS) phenomenon in the deep-level EEG features of stroke patients. Our experimental results confirm the feasibility and efficacy of the proposed approach for MI-BCI rehabilitation systems.

Brain-computer interface (BCI) actively translates the brain signals into executable actions by establishing direct communication between the human brain and external devices. Recording brain activity through electroencephalography (EEG) is generally contaminated with both physiological and nonphysiological artifacts, which significantly hinders the BCI performance. Artifact subspace reconstruction (ASR) is a well-known statistical technique that automatically removes artifact components by determining the rejection threshold based on the initial reference EEG segment in multichannel EEG recordings. In real-world applications, the fixed threshold may limit the efficacy of the artifact correction, especially when the quality of the reference data is poor. This study proposes an adaptive online ASR technique by integrating the Hebbian/anti-Hebbian neural networks into the ASR algorithm, namely, principle subspace projection ASR (PSP-ASR) and principal subspace whitening ASR (PSW-ASR) that segmentwise self-organize the artifact subspace by updating the synaptic weights according to the Hebbian and anti-Hebbian learning rules. The effectiveness of the proposed algorithm is compared to the conventional ASR approaches on benchmark EEG dataset and three BCI frameworks, including steady-state visual evoked potential (SSVEP), rapid serial visual presentation (RSVP), and motor imagery (MI) by evaluating the root-mean-square error (RMSE), the signal-to-noise ratio (SNR), the Pearson correlation, and classification accuracy. The results demonstrated that the PSW-ASR algorithm effectively removed the EEG artifacts and retained the activity-specific brain signals compared to the PSP-ASR, standard ASR (Init-ASR), and moving-window ASR (MW-ASR) methods, thereby enhancing the SSVEP, RSVP, and MI BCI performances. Finally, our empirical results from the PSW-ASR algorithm suggested the choice of an aggressive cutoff range of c = 1-10 for activity-specific BCI applications and a moderate range of for the benchmark dataset and general BCI applications.

Despite advances in neuroscience, the mechanisms by which human brain resolve optical image formation through relational reasoning remain unclear, particularly its relationships with task difficulty. Therefore, this study explores the underlying brain dynamics involved in optical image formation tasks at various difficulty levels, including those with a single convex lens and a single mirror. Compared to single convex lens relational reasoning with high task difficulty, the single mirror relational reasoning exhibited significantly higher response accuracy and shorter latency. As compared to single mirror tasks, single convex tasks exhibited greater frontal midline theta augmentation and right parietal alpha suppression during phase I and earlier phase II, and augmentation of frontal midline theta, right parietal-occipital alpha, and left mu alpha suppression during late phase II. Moreover, the frontal midline theta power in late phase II predicts the likelihood of solving single convex tasks the best, while the parietal alpha power in phase I is most predictive. In addition, frontal midline theta power exhibited stronger synchronization with right parietal alpha, right occipital alpha, and mu alpha power when solving single convex tasks than single mirror tasks. In summary, having stronger brain dynamics and coordination is vital for achieving optical image formation with greater difficulty.

Wireless electroencephalography (EEG) systems have been attracting increasing attention in recent times. Both the number of articles discussing wireless EEG and their proportion relative to general EEG publications have increased over years. These trends indicate that wireless EEG systems could be more accessible to researchers and the research community has recognized the potential of wireless EEG systems. To explore the development and diverse applications of wireless EEG systems, this review highlights the trends in wearable and wireless EEG systems over the past decade and compares the specifications and research applications of the major wireless systems marketed by 16 companies. For each product, five parameters (number of channels, sampling rate, cost, battery life, and resolution) were assessed for comparison. Currently, these wearable and portable wireless EEG systems have three main application areas: consumer, clinical, and research. To address this multitude of options, the article also discussed the thought process to find a suitable device that meets personalization and use cases specificities. These investigations suggest that low-price and convenience are key factors for consumer applications, wireless EEG systems with FDA or CE-certification may be more suitable for clinical settings, and devices that provide raw EEG data with high-density channels are important for laboratory research. This article presents an overview of the current state of the wireless EEG systems specifications and possible applications and serves as a guide point as it is expected that more influential and novel research will cyclically promote the development of such EEG systems.

Motor imagery (MI) based brain-computer interfaces (BCIs) have been studied as applications for the improving rehabilitation and recovery, as well as augmenting existing function. MI BCI systems typically provide feedback in an egocentric rather than an allocentric reference frame. This study aims to see if presenting stimuli in an allocentric reference frame is comparable to presenting egocentric stimuli. We used dynamic visual stimuli in egocentric and allocentric reference frames to induce motor imagery in a virtual reality (VR) environment. Eight participants imagined grasping actions with their left and right hands while observing egocentric or allocentric stimuli. The allocentric and egocentric reference frame tasks had comparable inter-rater agreement and precision, indicating that allocentric visual feedback is as effective as egocentric one for MI BCI.

In recent years, the use of bio-sensing signals such as electroencephalogram (EEG), electrocardiogram (ECG), etc. have garnered interest towards applications in affective computing. The parallel trend of deep-learning has led to a huge leap in performance towards solving various vision-based research problems such as object detection. Yet, these advances in deep-learning have not adequately translated into bio-sensing research. This work applies novel deep-learning-based methods to various bio-sensing and video data of four publicly available multi-modal emotion datasets. For each dataset, we first individually evaluate the emotion-classification performance obtained by each modality. We then evaluate the performance obtained by fusing the features from these modalities. We show that our algorithms outperform the results reported by other studies for emotion/valence/arousal/liking classification on DEAP and MAHNOB-HCI datasets and set up benchmarks for the newer AMIGOS and DREAMER datasets. We also evaluate the performance of our algorithms by combining the datasets and by using transfer learning to show that the proposed method overcomes the inconsistencies between the datasets. Hence, we do a thorough analysis of multi-modal affective data from more than 120 subjects and 2,800 trials. Finally, utilizing a convolution-deconvolution network, we propose a new technique towards identifying salient brain regions corresponding to various affective states.

Learning from subject's calibration data can significantly improve the performance of a steady-state visually evoked potential (SSVEP)-based brain-computer interface (BCI), for example, the state-of-the-art target recognition methods utilize the learned subject-specific and stimulus-specific model parameters. Unfortunately, when dealing with new stimuli or new subjects, new calibration data must be acquired, thus requiring laborious calibration sessions, which becomes a major challenge in developing high-performance BCIs for real-life applications. This study investigates the feasibility of transferring the model parameters (i.e., the spatial filters and the SSVEP templates) across two different groups of visual stimuli in SSVEP-based BCIs. According to our exploration, we can extract a common spatial filter from the spatial filters across different stimulus frequencies and a common impulse response from the SSVEP templates across different neighboring stimulus frequencies, in which the common spatial filter is considered as the transferred spatial filter and the common impulse response is utilized to reconstruct the transferred SSVEP template according to the theory that an SSVEP is a superposition of the impulse responses. Then, we develop a transfer learning canonical correlation analysis (tlCCA) incorporating the transferred model parameters. For evaluation, we compare the recognition performance of the calibration-free, the calibration-based, and the proposed tlCCA on an SSVEP data set with 60 subjects. Experiment results prove that the spatial filters share commonality across different frequencies and the impulse responses share commonality across neighboring frequencies. More importantly, the tlCCA performs significantly better than the calibration-free algorithms, comparably to the calibration-based algorithm. Note to Practitioners-This work is motivated by the long calibration time problem in using an steady-state visually evoked potential (SSVEP)-based brain-computer interface (BCI) because most state-of-the-art frequency recognition methods consider merely the situation that the calibration data and the test data are from the same subject and the same visual stimulus. This article assumes that the model parameters share the stimulus-nonspecific knowledge in a limited stimulus frequency range, and thus, the subject's old calibration data can be reused to learn new model parameters for new visual stimuli. First, the model parameters can be decomposed into the stimulus-nonspecific knowledge (or subject-specific knowledge) and stimulus-specific knowledge. Second, the new model parameters can be generated via transferring the knowledge across stimulus frequencies. Then, a new recognition algorithm is developed using the transferred model parameters. Experiment results validate the assumptions, and moreover, the proposed scheme could be extended to other scenarios, such as when facing new subjects, or adopting new signal acquisition equipment, which would be helpful to the future development of zero-calibration SSVEP-based BCIs for real-life healthcare applications.

Brain-Computer interfaces (BCIs) enhance the capability of human brain activities to interact with the environment.Recent advancements in technology and machine learning algorithms have increased interest in electroencephalographic (EEG)-based BCI applications.EEG-based intelligent BCI systems can facilitate continuous monitoring of fluctuations in human cognitive states under monotonous tasks, which is both beneficial for people in need of healthcare support and general researchers in different domain areas.In this review, we survey the recent literature on EEG signal sensing technologies and computational intelligence approaches in BCI applications, compensating for the gaps in the systematic summary of the past five years.Specifically, we first review the current status of BCI and signal sensing technologies for collecting reliable EEG signals.Then, we demonstrate state-of-the-art computational intelligence techniques, including fuzzy models and transfer learning in machine learning and deep learning algorithms, to detect, monitor, and maintain human cognitive states and task performance in prevalent applications.Finally, we present a couple of innovative BCI-inspired healthcare applications and discuss future research directions in EEG-based BCI research.!

Recently, the advances in passive brain-computer interfaces (BCIs) based on electroencephalogram (EEG) have shed light on real-world neuromonitoring technologies. However, human variability in the EEG activities hinders the development of practical applications of EEG-based BCI. To tackle this problem, many transfer-learning techniques perform supervised calibration. This kind of calibration approach requires task-relevant data, which is impractical in real-life scenarios such as drowsiness during driving. This study presents a transfer-learning framework for EEG decoding based on the low-dimensional representations of subjects learned from the pre-trial EEG. Tensor decomposition was applied to the pre-trial EEG of subjects to extract the underlying characteristics in subject, spatial, and spectral domains. Then, the proposed framework assessed the characteristics to obtain the low-dimensional subject representations such that the subjects with similar brain dynamics can be identified. This method can leverage the existing data from other users, and a small number of data from a rapid, non-task, unsupervised calibration from a new user to build an accurate BCI. Our results demonstrated that, in terms of prediction accuracy, the proposed low-dimensional subject representation-based transfer learning (LDSR-TL) framework outperformed the random selection, and the Riemannian manifold approach in cognitive-state tracking, while requiring fewer training data. The results can greatly improve the practicability, and usability of EEG-based BCI in the real world.

Combing brain-computer interfaces (BCI) and virtual reality (VR) is a novel technique in the field of medical rehabilitation and game entertainment. However, the limitations of BCI such as a limited number of action commands and low accuracy hinder the widespread use of BCI-VR. Recent studies have used hybrid BCIs that combine multiple BCI paradigms and/or the multi-modal biosensors to alleviate these issues, which may become the mainstream of BCIs in the future. The main purpose of this review is to discuss the current status of multi-modal BCI-VR. This study first reviewed the development of the BCI-VR, and explored the advantages and disadvantages of incorporating eye tracking, motor capture, and myoelectric sensing into the BCI-VR system. Then, this study discussed the development trend of the multi-modal BCI-VR, hoping to provide a pathway for further research in this field.

Electroencephalography (EEG)-based emotion recognition has advanced the field in affective computing and has enabled applications in human-computer interactions. Despite significant progress has been made in decoding emotion using supervised machine-learning methods, few studies applied data-driven, unsupervised approaches to explore the underlying EEG dynamics during an emotion experiment and examine how such dynamics correlate with subjective reports of emotion. This study employs the adaptive mixture independent component analysis (AMICA), an unsupervised approach, to EEG data from the DEAP dataset where 32 subjects watched emotional videos. Empirical results showed that AMICA could learn distinct models that separated EEG date collected in the emotion experiment. The identified changes in EEG patterns were weakly-correlated with the four reported emotion scales, indicating the underlying EEG dynamics partially reflected the emotional activities as well as the emotion-irrelevant brain dynamics. Further, the correlations between EEG dynamics and individuals' subjective emotional ratings were significantly higher than those between the EEG and the average ratings from online raters. Finally, building an emotion-decoding model based on the EEG dynamics revealed a significantly better classification performance for valence ratings compared to arousal. This study demonstrated the use of AMICA in characterizing the EEG dynamics in emotion experiments and provided insight into the relationship between EEG and the reported emotional experiences. The unsupervised learning approach can be applied to studying emotion and other confounding factors such as emotion irrelevant EEG artifacts, thereby improving the performance of emotion decoding for EEG-based affective computing.

This paper presents a hardware-oriented memory-limited online FastICA algorithm and its hardware architecture and implementation for eight-channel electroencephalogram (EEG) signal separation. The online algorithm integrates the data overlapping, garbage detection, channel permutation, and momentum-controlled weight update schemes to stabilize the order of the decomposed source signals across time. This study also realizes the algorithm into a hardware architecture and implementation with a core area of 1.469x1.469 mm2in a TSMC 90 nm process. The resulting power dissipation for eight-channel EEG signal separation is 65 mW@100 MHz at 1V.

Recently game-based brain-computer interface (BCI) systems using electroencephalography (EEG) has been gaining popularity, providing a sophisticated experience to its users. Here we present such a novel hybrid system based on rapid serial visual presentation (RSVP) in conjunction with steady-state visual evoked potentials (SSVEP). Based on a matching computer game Jewel Quest a game is designed wherein a sequence of jewel images containing rare targets (<; 3%) in an RSVP paradigm is presented on a display at four distinct locations each flickering at different rates (4, 5, 6 and 7 Hz). A score is awarded upon successful detection of target image from neural signals. During real-time implementation to achieve higher classification speeds, EEG signals were epoched at the onset of each image, creating a high degree of class overlap and imbalance. Given these challenges in our EEG datasets, we present classifiers that can classify single-trial EEG epochs at the onset of target image presentation accurately. Initial results from 14 subjects indicate Hidden Markov Model (HMM) with Dirichlet emission probabilities provide ~1% higher, on average, the area under the precision-recall curve (AUC-PR) compared to the ensemble technique Bagging, commonly used to handle class imbalance.

Recent studies have shown that integrating individualized templates into a template-matching target identification method could significantly improve the performance of a steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI). However, collecting the template (or calibration) data for each individual can be time-consuming and laborious. This issue can be alleviated by employing phase-coded visual stimuli because phase information could be discriminated by using templates synthesized from the template induced by a visual stimulus. Minimizing phase intervals between two adjacent visual stimuli could increase the number of stimuli without increasing the calibration cost. Nonetheless, no study has investigated the effects of the phase interval on the classification performance. This study compared the classification accuracy of SSVEPs with five different phase intervals (0.1 π, 0.2 π, 0.3 π, 0.4 π, and 0.5 π) using synthesized individual templates with task-related component analysis (TRCA)-based spatial filtering. From a public 12-class SSVEP dataset, phase-adjusted SSVEP data were created by adding time shifts according to the five phase intervals. The classification results showed that the accuracy was sufficiently high when the phase intervals were over 0.3 π, suggesting the use of up to six phase-shifted visual stimuli at a given frequency.

High-speed steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs) have been developed to enable the communications between the human brain and external environments. One of the major issues in the real-world applications of SSVEP-BCIs is the laborious and time-consuming calibration process, triggering the development of transfer-learning approaches to leverage existing data from other users. A comprehensive investigation on the inter-and intra-subject variability in SSVEP data is thus needed to provide insight for designing future transfer-learning frameworks for SSVEP-BCIs. We hereby present the first study that systematically and quantitatively assesses the variability in SSVEP data, where the sources of inter-and intra-subject variability at low-and high-frequency range were identified using Fisher's discriminant ratios (FDRs). The insights gained from this work could drive the future developments of transfer-learning approaches to minimize the calibration efforts in high-speed SSVEP BCIs.

This study explored the electroencephalography (EEG) dynamics during a chemistry-related decision-making task and further examined whether the correctness of the decision-making performance could be reflected by EEG activity. A total of 66 undergraduate students' EEG were collected while they participated in a chemistry-related decision-making task in which they had to retrieve the relevant chemistry concepts in order to make correct decisions for each task item. The results showed that it was only in the anterior cingulate cortex (ACC) cluster that distinct patterns in EEG dynamics were displayed for the correct and incorrect responses. The logistic regression results indicated that ACC theta power from 300[Formula: see text]ms to 250[Formula: see text]ms before stimulus onset was the most informative factor for estimating the likelihood of making correct decisions in the chemistry-related decision-making task, while it was the ACC low beta power from 150[Formula: see text]ms to 250[Formula: see text]ms after stimulus onset. The results suggested that the ACC theta augmentation before the stimulus onset serves to actively maintain the relevant information for retrieval from long-term memory, while the ACC low beta augmentation after the stimulus onset may serve the function of mapping the encoded stimulus onto the relevant criteria that the given participant has held within his or her mind to guide the decision-making responses.

Electroencephalogram (EEG) signals are frequently used in brain-computer interfaces (BC!s), but they are easily contaminated by artifacts and noise, so preprocessing must be done before they are fed into a machine learning algorithm for classification or regression. Spatial filters have been widely used to increase the signal-to-noise ratio of EEG for BC! classification problems, but their applications in BC! regression problems have been very limited. This paper proposes two common spatial pattern (CSP) filters for EEG-based regression problems in BC!, which are extended from the CSP filter for classification, by using fuzzy sets. Experimental results on EEG-based response speed estimation from a large-scale study, which collected 143 sessions of sustained-attention psychomotor vigilance task data from 17 subjects during a 5-month period, demonstrate that the two proposed spatial filters can significantly increase the EEG signal quality. When used in LASSO and k-nearest neighbors regression for user response speed estimation, the spatial filters can reduce the root-mean-square estimation error by 10.02-19.77%, and at the same time increase the correlation to the true response speed by 19.39-86.47%.

Research has indicated that fatigue is a critical factor in cognitive lapses because it negatively affects an individual's internal state, which is then manifested physiologically. This study explores neurophysiological changes, measured by electroencephalogram (EEG), due to fatigue. This study further demonstrates the feasibility of an online closed-loop EEG-based fatigue detection and mitigation system that detects physiological change and can thereby prevent fatigue-related cognitive lapses. More importantly, this work compares the efficacy of fatigue detection and mitigation between the EEG-based and a nonEEG-based random method. Twelve healthy subjects participated in a sustained-attention driving experiment. Each participant's EEG signal was monitored continuously and a warning was delivered in real-time to participants once the EEG signature of fatigue was detected. Study results indicate suppression of the alpha- and theta-power of an occipital component and improved behavioral performance following a warning signal; these findings are in line with those in previous studies. However, study results also showed reduced warning efficacy (i.e. increased response times (RTs) to lane deviations) accompanied by increased alpha-power due to the fluctuation of warnings over time. Furthermore, a comparison of EEG-based and nonEEG-based random approaches clearly demonstrated the necessity of adaptive fatigue-mitigation systems, based on a subject's cognitive level, to deliver warnings. Analytical results clearly demonstrate and validate the efficacy of this online closed-loop EEG-based fatigue detection and mitigation mechanism to identify cognitive lapses that may lead to catastrophic incidents in countless operational environments.

This study proposes a flexible method to track and quantify the ac-coupled gain variations in electrode to tissue interface (ETI) of adhesive/reusable electrodes. Particularly, this study focuses on the effects of multiplicative motion artifacts (MMA) on ETIs and proposes a continuously monitoring technique to assess ETI multiplicative variations. The proposed method only requires one additional channel of self electrocardiogram (self-ECG). Both gel-covered and foam-based adhesive electrodes were tested. Empirical results show that the ETI multiplicative variations can be characterized by the proposed convenient method. Meanwhile, we successfully demonstrate different characteristics for two types of electrodes under five motion postures including suppress motion, fist/stretch palm, twist, bent inward and arm lift, Furthermore, The long-term degradation experiment illustrates that the average characteristics variation of ETI is less than 6% within 3-day of use.

On the pathway from laboratory settings to real world environment, a major challenge on the development of a robust electroencephalogram (EEG)-based brain-computer interface (BCI) is to collect a significant amount of informative training data from each individual, which is labor intensive and time-consuming and thereby significantly hinders the applications of BCIs in real-world settings. A possible remedy for this problem is to leverage existing data from other subjects. However, substantial inter-subject variability of human EEG data could deteriorate more than improve the BCI performance. This study proposes a new transfer learning (TL)-based method that exploits a subject's pilot data to select auxiliary data from other subjects to enhance the performance of an EEG-based BCI for drowsiness detection. This method is based on our previous findings that the EEG correlates of drowsiness were stable within individuals across sessions and an individual's pilot data could be used as calibration/training data to build a robust drowsiness detector. Empirical results of this study suggested that the feasibility of leveraging existing BCI models built by other subjects' data and a relatively small amount of subject-specific pilot data to develop a BCI that can outperform the BCI based solely on the pilot data of the subject.

This study explores electroencephalography (EEG) brain dynamics associated with mathematical problem solving. EEG and solution latencies (SLs) were recorded as 11 neurologically healthy volunteers worked on intellectually challenging math puzzles that involved combining four single-digit numbers through basic arithmetic operators (addition, subtraction, division, multiplication) to create an arithmetic expression equaling 24. Estimates of EEG spectral power were computed in three frequency bands - θ (4-7 Hz), α (8-13 Hz) and β (14-30 Hz) - over a widely distributed montage of scalp electrode sites. The magnitude of power estimates was found to change in a linear fashion with SLs - that is, relative to a base of power spectrum, theta power increased with longer SLs, while alpha and beta power tended to decrease. Further, the topographic distribution of spectral fluctuations was characterized by more pronounced asymmetries along the left-right and anterior-posterior axes for solutions that involved a longer search phase. These findings reveal for the first time the topography and dynamics of EEG spectral activities important for sustained solution search during arithmetical problem solving.

An augmented brain computer interface that can detect users' brain states in real-life situations has been developed using wireless EEG headsets, smart phones and ubiquitous computing services. This kind of wearable natural user interfaces will have a wide-range of potential applications in future smart environments. This paper describes its ubiquitous system architecture and introduces its enabling technologies, which include machine-to-machine publish/subscribe protocols, multi-tier fog/cloud computing infrastructure and a linked data web. Its real-time responsiveness and easiness-of-use will be demonstrated by playing a multi-player on-line BCI game EEG Tractor Beam at the Intelligent Environment Conference.

The research of electroencephalography (EEG)-based emotion classification has gained much popularity in the past few years. Researchers continue to seek an optimal machine learning-based pipeline to characterize the associations between complex spatio-spectral EEG dynamics and implicit emotional responses. However, toward a real-life application, addressing the inherent day-to-day variability in EEG signals is also of urgent importance, yet was less concerned in the literature. This study explored the day-to-day EEG variability and tested the feasibility of developing an emotion-classification pipeline that can account for such variability. The empirical results of this study showed that the use of a proper feature extraction, e.g., band-power asymmetries over the fronto-posterior regions, in conjunction with an effective artifact removal method, e.g., independent component analysis, could alleviate the impacts of inter-day variability and improve the classification performance.

Implementing a complex spelling program using a steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) remains a challenge due to difficulties in stimulus presentation and target identification. This study aims to explore the feasibility of mixed frequency and phase coding in building a high-speed SSVEP speller with a computer monitor. A frequency and phase approximation approach was developed to eliminate the limitation of the number of targets caused by the monitor refresh rate, resulting in a speller comprising 32 flickers specified by eight frequencies (8-15 Hz with a 1 Hz interval) and four phases (0°, 90°, 180°, and 270°). A multi-channel approach incorporating Canonical Correlation Analysis (CCA) and SSVEP training data was proposed for target identification. In a simulated online experiment, at a spelling rate of 40 characters per minute, the system obtained an averaged information transfer rate (ITR) of 166.91 bits/min across 13 subjects with a maximum individual ITR of 192.26 bits/min, the highest ITR ever reported in electroencephalogram (EEG)-based BCIs. The results of this study demonstrate great potential of a high-speed SSVEP-based BCI in real-life applications.

This study proposes and evaluates a recursive algorithm for incremental estimation of independent components from on-line data. The algorithm offers the convergence properties of batch independent component analysis (ICA) with incremental updates of a form similar to natural gradient (NG) on-line information maximization (Infomax). We employ recursive procedure to arrive at steady state solution given by NG Infomax. Furthermore, we propose a novel procedure to compute corrective updates on the basis of previous estimates. Implementation of this algorithm incurs linear complexity in data size, input dimensions, and number of estimated independent components. Significant gains in convergence rate over on-line natural gradient ICA are demonstrated.

The study of brain-computer interfaces (BCIs) has undergone 30 years of intense development and has grown into a rich and diverse field. BCIs are technologies that enable direct communication between the human brain and external devices. Conventionally, wet electrodes have been employed to obtain unprecedented sensitivity to high-temporal-resolution brain activity; recently, the growing availability of various sensors that can be used to detect high-quality brain signals in a wide range of clinical and everyday environments is being exploited. This development of biosensing neurotechnologies and the desire to implement them in real-world applications have led to the opportunity to develop augmented BCIs (ABCIs) in the upcoming decades. An ABCI is similar to a BCI in that it relies on biosensors that record signals from the brain in everyday environments; the signals are then processed in real time to monitor the behavior of the human. To use an ABCI as a mobile brain imaging technique for everyday, real-life applications, the sensors and the corresponding device must be lightweight and the equipment response time must be short. This study presents an overview of the wide range of biosensor approaches currently being applied to ABCIs, from their use in the laboratory to their application in clinical and everyday use. The basic principles of each technique are described along with examples of current applications of cutting-edge neuroscience research. In summary, we show that ABCI techniques continue to grow and evolve, incorporating new technologies and advances to address ever more complex and important neuroscience issues, with advancements that are envisioned to lead to a wide range of real-life applications.

Motion sickness is a common symptom that occurs when the brain receives conflicting information about the sensation of movement. Many motion sickness biomarkers have been identified, and electroencephalogram (EEG)-based motion sickness level estimation was found feasible in our previous study. This study employs genetic feature selection to find a subset of EEG features that can further improve estimation performance over the correlation-based method reported in the previous studies. The features selected by genetic feature selection were very different from those obtained by correlation analysis. Results of this study demonstrate that genetic feature selection is a very effective method to optimize the estimation of motion-sickness level. This demonstration could lead to a practical system for noninvasive monitoring of the motion sickness of individuals in real-world environments.

This paper presents a low-power VLSI implementation of a 4-channel independent component analysis (ICA) processor for portable EEG signal processing applications. The low-power scheme employed for this ICA chip is based on power gating and clock gating by utilizing Cadence common power flow (CPF) low-power methodology and also according to the characteristics of ICA training behavior using different training window sizes. The proposed low power ICA processor can separate EEG and mixed EEG-like super-Gaussian signals in real time. The chip can be operated at up to 60MHz working frequency and a maximum sampling rate of 9.394 KHz for EEG signals. The power consumption of this chip is 0.690 mW during training under the condition of 0.9V supply voltage and 10 MHz operating frequency using UMC 90nm High-Vt CMOS technology. The total chip area is 1230 × 1230 μm1.

Motion sickness is a normal response to real, perceived, or even anticipated movement. People tend to get motion sickness on a moving boat, train, airplane, car, or amusement park rides. Motion sickness occurs when the body, the inner ear, and the eyes send conflicting signals to the brain. Sensory conflict theory that came about in the 1970's has become the most widely accepted theorem of motion-sickness among scientists [1]. The theory proposed that the conflict between the incoming sensory inputs could induce motion- sickness. However, some new research studies have appeared to tackle the issue of the vestibular function in central nervous system (CNS). In the previous human subject studies, researchers attempt to confirm the brain areas involved in the conflict in multi-modal sensory systems by means of clinical or anatomical methods. Our past studies had investigated the EEG activities correlated with motion sickness in a virtual-reality based driving simulator. We found that the parietal, motor, occipital brain regions exhibited significant EEG power changes in response to vestibular and visual stimuli. Based on these experimental results, we attempt to implement an EEG-based evaluation system to estimate subject's motion sickness level upon the major EEG power spectra from these motion sickness related brain area in this study. The evaluation system can be applied to early detect the subject's motion sickness level and prevent the uncomfortable syndromes occurred in advance in our daily life.

Monitoring and predicting human cognitive state and performance using physiological signals such as Electroencephalogram (EEG) have recently gained increasing attention in the fields of brain-computer interface and cognitive neuroscience. Most previous psychophysiological studies of cognitive changes have attempted to use the same model for all subjects. However, the relatively large individual variability in EEG dynamics relating to loss of alertness suggests that for many operators, group statistics cannot be used to accurately predict changes in cognitive states. Attempts have also been made to build a subject-dependent model for each individual based on his/her pilot data to account for individual variability. However, such methods assume the cross-session variability in EEG dynamics to be negligible, which could be problematic due to electrode displacements, environmental noises, and skin-electrode impedance. Here first we show that the EEG power in the alpha and theta bands are strongly correlated with changes in the subject’s cognitive state reflected through his driving performance and hence his departure from alertness. Then under very mild and realistic assumptions we derive a model for the alert state of the person using EEG power in the alpha and theta bands. We demonstrate that deviations (computed by Mahalanobis distance) of the EEG power in the alpha and theta bands from the corresponding alert models are correlated to the changes in the driving performance. Finally, for detection of drowsiness we use a linear combination of deviations of the EEG power in the alpha band and theta band from the respective alert models that best correlates with subject’s changing level of alertness, indexed by subject’s behavioral response in the driving task. This approach could lead to a practical system for noninvasive monitoring of the cognitive state of human operators in attention-critical settings.

Electroencephalograph (EEG) recording systems offer a versatile, non-invasive window on the brain’s spatiotemporal activity for many neuroscience and clinical applications. Our research aims to improve the convenience and mobility of EEG recording by eliminating the need for conductive gel and creating sensors that fit into a scalable array architecture. The EEG drycontact electrodes are created with micro-electrical-mechanical system (MEMS) technology. Each channel of our analog signal processing front-end comes on a custom-built, dime-sized circuit board which contains an amplifier, filters, and analog-to-digital conversion. A daisy-chain configuration between boards with bitserial output reduces the wiring needed. A system consisting of seven sensors is demonstrated in a real-world setting. Consuming just 3 mW, it is suitable for mobile applications. The system achieves an input-referred noise of 0.28 μVrms in the signal band of 1 to 100 Hz, comparable to the best medical-grade systems in use. Noise behavior across the daisychain is characterized, alpha-band rhythms are detected, and an eye-blink study is demonstrated.

Neural prosthetic technologies have helped many patients by restoring vision, hearing, or movement and relieving chronic pain or neurological disorders. While most neural prosthetic systems to date have used invasive or implantable devices for patients with inoperative or malfunctioning external body parts or internal organs, a much larger population of “healthy” people who suffer episodic or progressive cognitive impairments in daily life can benefit from noninvasive neural prostheses. For example, reduced alertness, lack of attention, or poor decision-making during monotonous, routine tasks can have catastrophic consequences. This study proposes a noninvasive mobile prosthetic platform for continuously monitoring high-temporal resolution brain dynamics without requiring application of conductive gels on the scalp. The proposed system features dry microelectromechanical system electroencephalography sensors, low-power signal acquisition, amplification and digitization, wireless telemetry, online artifact cancellation, and signal processing. Its implications for neural prostheses are examined in two sample studies: 1) cognitive-state monitoring of participants performing realistic driving tasks in the virtual-reality-based dynamic driving simulator and 2) the neural correlates of motion sickness in driving. The experimental results of these studies provide new insights into the understanding of complex brain functions of participants actively performing ordinary tasks in natural body positions and situations within real operational environments.

We present a novel experimental paradigm and data analysis methodology for studying brain dynamics during sustained-attention tasks. 256-channel EEG data were recorded while subjects participated in hour-long simulated driving sessions. Every few seconds, the vehicle drifted away from the center of the left lane, and subjects were instructed to steer back to the lane center. The error of each drifting event was measured by the maximum absolute distance from the vehicle's position at deviation onset. EEG data were analyzed using independent component analysis and time-frequency analysis. An independent component with equivalent dipole sources located bilaterally in lateral occipital cortex exhibited multi-scale brain dynamics. Tonic (~20s) alpha-band power increased in high-error compared to low-error epochs, while phasic (~1s) alpha power was suppressed briefly after deviation onset, then increased strongly just before response offset. Other components also exhibited distinct tonic and/or phasic activity patterns relating to deviation onsets or response onsets.

We have developed a new approach to studying human heart activity using independent component analysis. The electrocardiogram (ECG) is an important tool in diagnosis of heart disease. However, the normal 12-lead ECG can only record limited aspects of heart's electrical signals and mostly their interpretation relies on trained and experienced medical doctors. We have performed experiments in which heart signals were recorded in high spatial resolution. Independent component analysis was applied to the recorded signals to separate distinct temporal components of the recorded signals. The separated components were further analyzed by back-projecting their activities to the surface montage to examine each component's property. Experimental results show this to be a promising approach that can be extended to build more detailed heart activity simulations

The growing number of traffic fatalities in recent years has become a serious concern to society. Accidents caused by drivers' drowsiness behind the steering wheel have a high fatality rate because of the marked decline in the drivers' abilities of perception, recognition, and vehicle control abilities while sleepy. Preventing accidents caused by drowsiness requires a technique for detecting, estimating, and predicting the level of alertness of a driver and a mechanism for maintaining his/her maximum performance. This work describes a system that combines electroencephalographic (EEG) power spectrum estimation, principal component analysis, and fuzzy neural network model to estimate/predict drivers' drowsiness level in a driving simulator. Our results demonstrated that, for the first time, it is feasible to accurately estimate task performance, accurately estimate quantitatively measured driving performance, expressed as deviation between the center of the vehicle and the center of the cruising lane, in a realistic driving simulation.

The analysis of electroencephalographic (EEG) and magnetoencephalographic (MEG) recordings is important both for basic brain research and for medical diagnosis and treatment. Independent component analysis (ICA) is an effective method for removing artifacts and separating sources of the brain signals from these recordings. A similar approach is proving useful for analyzing functional magnetic resonance brain imaging (fMRI) data. In this paper, we outline the assumptions underlying ICA and demonstrate its application to a variety of electrical and hemodynamic recordings from the human brain.

The authors investigate the use of two types of neural networks, multilayer perceptrons (MLP) and learning vector quantizers (LVQ), as applied to isolated speaker-independent vowel recognition as a typical classification task. The LVQ algorithm used is a modification called the frequency-sensitive competitive-learning (FSCL) LVQ. The performance of each of these networks for different input feature sets is evaluated and compared. The effects of different distortion measures on recognition performance are also studied. The results show that the choice of the input feature set and the distortion measure can significantly affect recognition performance. It is shown that, while both the backpropagation (BP) and FSCL-LVQ algorithms can be applied to a set of vowel-recognition tasks, the FSCL-LVQ procedure offers an advantage over the MLP approach. The FSCL-LVQ algorithm allows the use of any appropriate distortion measure for particular input features, while the BP algorithm optimizes the weights by minimizing the squared errors between the actual and desired outputs. Consequently, for some tasks, the LVQ architecture can perform more accurate classification