Accomplishing a plug-and-play steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) remains a critical challenge, due to the unsatisfying performance of calibration-free decoding algorithms.A current method called online adaptive canonical correlation analysis (OACCA) has proved efficient in enhancing calibration-free performance by self-adaptation merely with online data.However, OACCA only concerns the adaptation of spatial filters and excludes other useful adaptive procedures like individual template estimation, hindering fully exploitable model decoding and adaptation. This study proposes a new online adaptation framework termed online adaptive extended correlation analysis (OAECA) to augment the calibration-free online adaptation loop. OAECA first recalls and cleans the online trials for reliable data learning, then tunes individual templates and spatial filters for complete model updating, and finally adopts extended feature matching to improve target recognition. The simulation results on two public SSVEP datasets revealed that OAECA significantly outperformed OACCA for almost all 105 subjects, and both offline and online experiments further confirmed the effectiveness of OAECA. Particularly, OAECA achieved the highest average information transfer rate (ITR) of 202.17 bits/min in the online experiment, significantly exceeding the state-of-the-art OACCA of 177.02 bits/min. This study enhanced the calibration-free performance through comprehensive online adaptation, hopefully advancing SSVEP-based BCIs toward practical plug-and-play real-world applications.

Large instruction-set brain-computer interfaces (BCIs) allow users to issue many commands through a single interface, greatly expanding their application scope. Increasing the number of targets, however, raises encoding complexity and intensifies the trade-off between calibration time and decoding performance. We introduce a 120-target steady-state visual evoked potential (SSVEP)-BCI that pairs dual-frequency phase modulation (DFPM) with a lightweight global multi stimulus canonical correlation analysis-based spatiotemporal filtering (gmsCCA-st) method. DFPM encodes the 120 targets with only 23 low-frequency carriers by simultaneously flickering two frequency-phase tags in a checkerboard pattern, thereby mitigating the "frequency scarcity" problem and eliciting pronounced harmonic and intermodulation responses. Instead of training a separate filter for each target, gmsCCA-st learns a set of shared spatiotemporal filters from all targets. With just one calibration trial per target in the offline experiment, the system achieved a peak information transfer rate (ITR) of 326.49±55.13 bits/min. During online cue-guided spelling, the system attained 94.69±5.99% accuracy and 251.47±25.47 bits/min ITR; in free-spelling mode, accuracy was 91.72±6.89% at 176.64±23.75 bits/min. These findings demonstrate the feasibility of a high-performance 120 target SSVEP-BCI after only three minutes of calibration, overcoming the compromise among instruction-set size, calibration burden, and performance. This study therefore offers a practical pathway toward high-performance, minimal-calibration large instruction-set BCIs.

Cross-subject emotion recognition based on electroencephalography (EEG) is currently a major development direction for affective brain-computer interfaces (aBCI). Currently, researchers are focusing on using domain adversarial neural networks (DANN) to capture domain-invariant features and enhance the cross-subject generalization of models. However, current DANN in the aBCI field cannot align features across different domains by directly estimating the differences between the source and target domains, and may struggle to effectively align feature distributions of different domains. Moreover, the current mainstream cross-subject evaluation protocols can result in inflated offline performance. To address the shortcomings of DANN, we develop a novel conditional adversarial domain adaptation network, which brings about a 10% performance improvement for the model. Specifically, by introducing a domain adapter, we gradually align the distributions of different domains during training to reduce domain differences. Additionally, we incorporate a conditioning strategy in the domain discriminator to effectively align distributions of different domains. We also develop a novel evaluation method that simulates an online scenario to address the issue of inflated offline performance. Extensive comparisons with existing methods demonstrate that the proposed approach achieves state-of-the-art cross-subject emotion recognition performance, attaining 93.62% accuracy on the SEED dataset and 82.16% on SEED-IV.

The affective brain-computer interface (aBCI) facilitates the objective identification or regulation of human emotions. Current aBCI mainly relies on electroencephalography (EEG). However, research shows that emotions involve a large-scale distributed brain network. Compared to electroencephalography (EEG), functional near-infrared spectroscopy (fNIRS) offers a higher spatial resolution. It holds greater potential in capturing emotional spatial information, which may foster the development of new affective Brain-Computer Interfaces (aBCI). We proposed a novel self-attention-based deep-learning transformer language model for fNIRS cross-subject emotion recognition, which could automatically learn the emotion's spatial attention weight information with strong interpretability. Besides, we performed data augmentation by introducing the wasserstein generative adversarial networks (WGAN). Results showed: (1) We achieved 84% three-category cross-subject emotion decoding accuracy. The spatial transformer module and WGAN improved the accuracy by 12.8% and 4.3%, respectively. (2) Compared with cutting-edge fNIRS research, we led by 10% in three-category decoding accuracy. (3) Compared with cutting-edge EEG research, we lead by 28% in arousal decoding accuracy, 10% in valence decoding accuracy, and 2% in three-category decoding accuracy. (4) Besides, our approach holds the potential to uncover the brain's spatial encoding mechanism of human emotion processing, providing a new direction for building interpretable artificial intelligence models.

Objective: Because of the non-stationary nature of electroencephalogram (EEG) signals, traditional non-invasive brain-computer interfaces (BCIs) usually only produce discrete commands, limiting their ability to control external devices continuously. This study proposes a novel BCI control strategy mapping multiple discrete commands to continuous motion, enabling real-time manipulation of a drone in four degrees of freedom (DOF).Methods: Our strategy used the fast steady state visual evoked potential (SSVEP) encoding and decoding method to convert user intentions into the drone’s flight status in near real-time. Simultaneously, the drone’s live video was embedded into the SSVEP stimuli, providing users with a first-person perspective control experience.Results: In drone control experiments, participants successfully maneuvered the drone through complex path-following tasks in simulated and physical scenarios. The mean flight trajectory bias ratio was measured as 0.81, with a mean flight smoothness of -3.31 (measured by spectral arc length) and mean Fitts’s throughput of 9.18 bits/min. Notably, the brain-to-hand ratio (BHR) for all metrics approached 1, indicating that our non-invasive control system achieved comparable performance to manual control systems.Conclusion: These results suggest the effectiveness of our proposed BCI control strategy that maps discrete commands to continuous motion and extends the capabilities of non-invasive BCIs in continuous control scenarios.Significance: This study significantly advances the applications of BCI and propels human-machine interaction towards a more direct realm.

Objective: Emergency anticipation (EA) refers to the brain’s rapid perceptual, cognitive, and motor preparation in response to imminent emergencies. Timely decoding of EA can facilitate proactive responses before full behavioral execution, which is critical in real-world scenarios such as avoiding hazards or mitigating accidents. However, the cortical activation underlying the EA process has not been fully explored. This study aims to analyze the neural activity of the EA process and explore the feasibility of detecting emergency motor intention in conjunction with brain-computer interface (BCI) technology. Methods: We designed a new emergency state induction paradigm in the virtual environment, including a target task (emergency anticipation, EA) and two baseline tasks (emergency anticipation execution, EAE, visual observation, VO). A total of 31 healthy subjects were recruited for the offline experiment. The cortical responses during the EA process were quantified by analyzing event-related potential, movement-related cortical potential, and event-related spectral perturbation. Discriminative canonical pattern matching, common spatial patterns, and shrinkage linear discriminant analysis were employed to perform binary classification. Six subjects participated in the pseudo-online asynchronous experiment to valid the feasibility of identifying emergency motor intention. Results: The results showed that the cascading process associated with EA existed in both the temporal and spectral domains. Particularly, temporal domain feature demonstrated superior classification performance, with averages of 90.13% (>80% chance level). The pseudo-online evaluation showed that the system response time with an average of 257.12 ms, which was 35 ms faster than the behavioral response. Significance: Our work demonstrated the cascading process of perceptual recognition, cognitive evaluation, and motor preparation during the EA processes and provided preliminary evidence supporting the feasibility of detecting emergency motor intentions. These findings lay a theoretical foundation for extending the application of BCI technology to rapid control scenarios.

Recently, functional magnetic resonance imaging (fMRI)-based brain networks have been shown to be an effective diagnostic tool with great potential for accurately detecting autism spectrum disorders (ASD). Meanwhile, the successful use of graph convolution networks (GCNs) methods based on fMRI information has improved the classification accuracy of ASD. However, many graph convolution-based methods do not fully utilize the topological information of the brain functional connectivity network (BFCN) or ignore the effect of non-imaging information. Therefore, we propose a hierarchical graph embedding model that leverage both the topological information of the BFCN and the non-imaging information of the subjects to improve the classification accuracy. Specifically, our model first use the Infomax Module to automatically identify embedded features in regions of interests (ROIs) in the brain. Then, these features, along with non-imaging information, is used to construct a population graph model. Finally, we design a graph convolution framework to propagate and aggregate the node features and obtain the results for ASD detection. Our model takes into account both the significance of the BFCN to individual subjects and relationships between subjects in the population graph. The model performed autism detection using the Autism Brain Imaging Data Exchange (ABIDE) dataset and obtained an average accuracy of 77.2% and an AUC of 87.2%. These results exceed those of the baseline approach. Through extensive experiments, we demonstrate the competitiveness, robustness and effectiveness of our model in aiding ASD diagnosis.

Steady-state visual evoked potentials (SSVEP)-based brain-computer interfaces (BCIs) have the potential to be utilized in various fields due to their high accuracies and information transfer rates (ITR). High-frequency (HF) visual stimuli have shown promise in reducing visual fatigue and enhancing user comfort. However, these HF-SSVEP-BCIs often face limitations in the number of commands and typically require extensive individual training data to achieve high performance. In this study, we proposed a row-column dual-frequency encoding and decoding method using HF stimulation to develop a comfortable BCI system that supports multiple commands and reduces training costs. We arranged 20 targets in a matrix of five rows and four columns, with each target modulated by left-and-right field stimulation using two frequency-phase combinations. Targets in each row or column share a unique frequency-phase combination, allowing EEG data from the same row or column to be used collectively to train a row/column index decoding model for target identification. To evaluate the performance of our method, we constructed a 20-target asynchronous robotic arm control system with the adaptive window method. With only four training trials per target, the online system achieved an ITR of 105.14 ± 14.15 bits/min, a true positive rate of 98.18 ± 2.87%, a false positive rate of 7.39 ± 6.73%, and a classification accuracy of 91.88 ± 5.75%, with an average data length of 925.70 ± 45.44 ms. These results indicate that the proposed protocol can deliver accurate and rapid command outputs for a comfortable SSVEP-based BCI with minimal training data and fewer frequencies.

Detecting arm movement direction is significant for individuals with upper-limb motor disabilities to restore independent self-care abilities. It involves accurately decoding the fine movement patterns of the arm, which has become feasible using invasive brain-computer interfaces (BCIs). However, it is still a significant challenge for traditional electroencephalography (EEG) based BCIs to decode multi-directional arm movements effectively. This study designed an ultra-high-density (UHD) EEG system to decode multi-directional arm movements. The system contains 200 electrodes with an interval of about 4 mm. We analyzed the patterns of the UHD EEG signals induced by arm movements in different directions. To extract discriminative features from UHD EEG, we proposed a spatial filtering method combining principal component analysis (PCA) and discriminative spatial pattern (DSP). We collected EEG signals from five healthy subjects (two left-handed and three right-handed) to verify the system's feasibility. The movement-related cortical potentials (MRCPs) showed a certain degree of separability both in waveforms and spatial patterns for arm movements in different directions. This study achieved an average classification accuracy of 63.15 (8.71)% for both arms (eight-class task) with a peak accuracy of 77.24%. For the dominant arm (four-class task), we obtained an average accuracy of 75.31 (9.21)% with a peak accuracy of 85.00%. For the first time, this study simultaneously decodes multi-directional movements of both arms using UHD EEG. This study provides a promising approach for detecting information about arm movement directions, which is significant for the development of BCIs.

Currently, spatiotemporal convolutional neural networks (CNNs) for electroencephalogram (EEG) signals have emerged as promising tools for seizure prediction (SP), which explore the spatiotemporal biomarkers in an epileptic brain. Generally, these CNNs capture spatiotemporal features at single spectral resolution. However, epileptiform EEG signals contain irregular neural oscillations of different frequencies in different brain regions. Therefore, it may be underperforming and uninterpretable for the CNNs without capturing complex spectral properties sufficiently. This study proposed a novel interpretable multi-branch architecture for spatiotemporal CNNs, namely MultiSincNet. On the one hand, the MultiSincNet could directly show the frequency boundaries using the interpretable sinc-convolution layers. On the other hand, it could extract and integrate multiple spatiotemporal features across varying spectral resolutions using parallel branches. Moreover, we also constructed a post-hoc explanation technique for multi-branch CNNs, using the first- order Taylor expansion and chain rule based on the multivariate composite function, which demonstrates the crucial spatiotemporal features learned by the proposed multi-branch spatiotemporal CNN. When combined with the optimal MultiSincNet, ShallowConvNet, DeepConvNet, and EEGWaveNet had significantly improved the subject-specific performance on most metrics. Specifically, the optimal MultiSincNet significantly increased the average accuracy, sensitivity, specificity, binary F1-score, weighted F1-score, and AUC of EEGWaveNet by about 7%, 8%, 7%, 8%, 7%, and 7%, respectively. Besides, the visualization results showed that the optimal model mainly extracts the spectral energy difference from the high gamma band focalized to specific spatial areas as the dominant spatiotemporal EEG feature.

Transcranial alternating current stimulation (tACS) has been reported to treat refractory auditory hallucinations in schizophrenia. Despite diligent efforts, it is imperative to underscore that tACS does not uniformly demonstrate efficacy across all patients as with all treatments currently employed in clinical practice. The study aims to find biomarkers predicting individual responses to tACS, guiding treatment decisions, and preventing healthcare resource wastage. We divided 17 schizophrenic patients with refractory auditory hallucinations into responsive(RE) and non-responsive(NR) groups based on their auditory hallucination symptom reduction rates after one month of tACS treatment. The pre-treatment resting-state electroencephalogram(rsEEG) was recorded and then computed absolute power spectral density (PSD), Hjorth parameters (HPs, Hjorth activity (HA), Hjorth mobility (HM), and Hjorth complexity (HC) included) from different frequency bands to portray the brain oscillations. The results demonstrated that statistically significant differences localized within the high gamma frequency bands of the right brain hemisphere. Immediately, we input the significant dissociable features into popular machine learning algorithms, the Cascade Forward Neural Network achieved the best recognition accuracy of 93.87%. These findings preliminarily imply that high gamma oscillations in the right brain hemisphere may be the main influencing factor leading to different responses to tACS treatment, and incorporating rsEEG signatures could improve personalized decisions for integrating tACS in clinical treatment.

Brain-computer interface (BCI) technology for emotion recognition holds significant potential for future applications in the treatment of refractory emotional disorders. Stereo-electroencephalography (SEEG), being less invasive, can precisely record neural activities originating from the cortex and the deep structures of the brain. Thus, it has broad application prospects in constructing emotion recognition BCI. In this study, SEEG data from nine subjects were collected to construct an emotion dataset, and a Spatial Transformer-based Hybrid Network (STHN) was proposed for SEEG emotion recognition. The triple-classification accuracy of STHN reached 83.56%, outperforming the baseline methods such as EEGNet, TSception, and the deep convolution neural network. Moreover, STHN can assign weights to each SEEG channel and select those channels that contribute more significantly to emotion recognition. It was found that when using the top 30% weighted SEEG channels as model inputs, the accuracy did not decrease significantly. Most of the channels with higher weights were located in brain regions strongly associated with emotions, such as the frontal lobe, the temporal lobe, and the hippocampus. This indicates that STHN is not merely a "black-box" model but possesses a degree of explainability. To the best of our knowledge, this is the first study to develop an SEEG emotion recognition algorithm, which is expected to play a crucial role in the monitoring and treatment of patients with refractory emotional disorders in the future.

Emotions deeply influence human behavior and decision-making. Currently, the spatiotemporal joint neural processing pattern of human emotions remains largely unclear. This study employed EEG-fNIRS simultaneous recordings to capture the spatiotemporal neural processing characteristics of human emotions by presenting Chinese emotional video stimuli. (1) EEG microstates’ temporal dynamic: Compared to low emotional arousal, microstate C and D's activities and C⇌D's transition probability significantly increased. (2) fNIRS spatial patterns: Compared to low arousal, the dorsolateral prefrontal cortex (DLPFC), inferior frontal gyrus (IFG), and temporoparietal junction (TPJ)’s oxygenated hemoglobin (HbO) concentrations significantly increased, along with the significant increase of DLPFC-IFG&TPJ's functional connectivity during high arousal. Compared to high valence, the DLPFC was significantly activated during low valence. (3) EEG-fNIRS spatiotemporal joint features: Compared to low arousal, there was a significant positive correlation between the occurrence of microstate D (corresponded to dorsal attention network, DAN) and HbO concentrations of DLPFC (DAN's key node) during high arousal, which consistently revealed the DAN's involvement for human emotional arousal processing. These results could provide not only multimodal complementary features for promoting the development of affective braincomputer interface, but also potential objective spatiotemporal neural markers for emotional disorders.

Body motion is an important channel for human communication and plays a crucial role in automatic emotion recognition. This work proposes a multiscale spatio-temporal network, which captures the coarse-grained and fine-grained affective information conveyed by full-body motion and decodes the complex mapping between emotion and body movement. The proposed method consists of three main components. First, a scale selection algorithm based on the pseudo-energy model is presented, which guides our network to focus not only on long-term macroscopic body expressions, but also on short-term subtle posture changes. Second, we propose a hierarchical spatio-temporal network that can jointly process posture covariance matrices and 3D posture images with different time scales, and then hierarchically fuse them in a coarse-to-fine manner. Finally, a spatio-temporal iterative (ST-ITE) fusion algorithm is developed to jointly optimize the proposed network. The proposed approach is evaluated on five public datasets. The experimental results show that the introduction of the energy-based scale selection algorithm significantly enhances the learning capability of the network. The proposed ST-ITE fusion algorithm improves the generalization and convergence of our model. The average classification results of the proposed method exceed 86% on all datasets and outperform the state-of-the-art methods.

Transcutaneous vagus nerve stimulation (tVNS) shows a potential regulatory role for motor planning. Still, existing research mainly focuses on behavioral studies, and the neural modulation mechanism needs to be clarified. Therefore, we designed a multi-condition (active or sham, pre or under, difficult or easy, left-hand or right-hand) motor planning experiment to explore the effect of online tVNS (i.e., tVNS and tasks synchronized). Twenty-eight subjects were recruited and randomly assigned to active and sham groups. Both groups performed the same tasks in the experiment and separately collected task-state EEG and 5-min eye-open resting-state EEG. The results showed that the changes in event-related potential (ERP) and movement-related cortical potential (MRCP) amplitudes were more significant for the left-hand difficult task (LD) under active-tVNS. According to the power spectrum results, active-tVNS significantly modulated the activities of the contralateral motor cortex at beta and gamma bands in the resting state. The functional connectivity based on partial directed coherence (PDC) showed significant changes in the parietal lobe after active-tVNS. These findings suggest that tVNS is a promising way to improve motor planning ability.

Motor learning plays a crucial role in human life, and various neuromodulation methods have been utilized to strengthen or improve it. Transcutaneous auricular vagus nerve stimulation (taVNS) has gained increasing attention due to its non-invasive nature, affordability and ease of implementation. Although the potential of taVNS on regulating motor learning has been suggested, its actual regulatory effect has yet been fully explored. Electroencephalogram (EEG) analysis provides an in-depth understanding of cognitive processes involved in motor learning so as to offer methodological support for regulation of motor learning. To investigate the effect of taVNS on motor learning, this study recruited 22 healthy subjects to participate a single-blind, sham-controlled, and within-subject serial reaction time task (SRTT) experiment. Every subject involved in two sessions at least one week apart and received a 20-minute active/sham taVNS in each session. Behavioral indicators as well as EEG characteristics during the task state, were extracted and analyzed. The results revealed that compared to the sham group, the active group showed higher learning performance. Additionally, the EEG results indicated that after taVNS, the motor-related cortical potential amplitudes and alpha-gamma modulation index decreased significantly and functional connectivity based on partial directed coherence towards frontal lobe was enhanced. These findings suggest that taVNS can improve motor learning, mainly through enhancing cognitive and memory functions rather than simple movement learning. This study confirms the positive regulatory effect of taVNS on motor learning, which is particularly promising as it offers a potential avenue for enhancing motor skills and facilitating rehabilitation.

Colonoscopy is considered the best diagnostic tool for early detection and resection of polyps, which can effectively prevent consequential colorectal cancer. In clinical practice, segmenting and classifying polyps from colonoscopic images have a great significance since they provide precious information for diagnosis and treatment. In this study, we propose an efficient multi-task synergetic network (EMTS-Net) for concurrent polyp segmentation and classification, and we introduce a polyp classification benchmark for exploring the potential correlations of the above-mentioned two tasks. This framework is composed of an enhanced multi-scale network (EMS-Net) for coarse-grained polyp segmentation, an EMTS-Net (Class) for accurate polyp classification, and an EMTS-Net (Seg) for fine-grained polyp segmentation. Specifically, we first obtain coarse segmentation masks by using EMS-Net. Then, we concatenate these rough masks with colonoscopic images to assist EMTS-Net (Class) in locating and classifying polyps precisely. To further enhance the segmentation performance of polyps, we propose a random multi-scale (RMS) training strategy to eliminate the interference caused by redundant information. In addition, we design an offline dynamic class activation mapping (OFLD CAM) generated by the combined effect of EMTS-Net (Class) and RMS strategy, which optimizes bottlenecks between multi-task networks efficiently and elegantly and helps EMTS-Net (Seg) to perform more accurate polyp segmentation. We evaluate the proposed EMTS-Net on the polyp segmentation and classification benchmarks, and it achieves an average mDice of 0.864 in polyp segmentation and an average AUC of 0.913 with an average accuracy of 0.924 in polyp classification. Quantitative and qualitative evaluations on the polyp segmentation and classification benchmarks demonstrate that our EMTS-Net achieves the best performance and outperforms previous state-of-the-art methods in terms of both efficiency and generalization.

Mental workload (MWL) assessment is critical for accident prevention and operator safety. However, achieving cross-task generalization of MWL classification models is a significant challenge for real-world applications. Classifiers trained on labeled samples from one task often experience a notable performance drop when directly applied to samples from other tasks, limiting its use cases. To address this issue, we propose a semi-supervised cross-task domain adaptation (SCDA) method using power spectral density (PSD) features for MWL recognition across tasks (MATB-II and n-back). Our results demonstrated that the SCDA method achieved the best cross-task classification performance on our data and COG-BCI public dataset, with accuracies of 90.98% ± 9.36% and 96.61% ± 4.35%, respectively. Furthermore, in the cross-task classification of cross-subject scenarios, SCDA showed the highest average accuracy (75.39% ± 9.56% on our data, 90.98% ± 9.36% on the COG-BCI public dataset). The findings indicate that the semi-supervised transfer learning approach using PSD features is feasible and effective for cross-task MWL assessment.

Adding supernumerary robotic limbs (SRLs) to humans and controlling them directly through the brain are main goals for movement augmentation. However, it remains uncertain whether neural patterns different from the traditional inherent limbs motor imagery (MI) can be extracted, which is essential for high-dimensional control of external devices. In this work, we established a MI neo-framework consisting of novel supernumerary robotic sixth-finger MI (SRF-MI) and traditional right-hand MI (RH-MI) paradigms and validated the distinctness of EEG response patterns between two MI tasks for the first time. Twenty-four subjects were recruited for this experiment involving three mental tasks. Event-related spectral perturbation was adopted to supply details about event-related desynchronization (ERD). Activation region, intensity and response time (RT) of ERD were compared between SRF-MI and RH-MI tasks. Three classical classification algorithms were utilized to verify the separability between different mental tasks. And genetic algorithm aims to select optimal combination of channels for neo-framework. A bilateral sensorimotor and prefrontal modulation was found during the SRF-MI task, whereas in RH-MI only contralateral sensorimotor modulation was exhibited. The novel SRF-MI paradigm enhanced ERD intensity by a maximum of 117% in prefrontal area and 188% in the ipsilateral somatosensory-association cortex. And, a global decrease of RT was exhibited during SRF-MI tasks compared to RH-MI. Classification results indicate well separable performance among different mental tasks (88.1% maximum for 2-class and 88.2% maximum for 3-class). This work demonstrated the difference between the SRF-MI and RH-MI paradigms, widening the control bandwidth of the BCI system.

Semi-supervised learning has attracted wide attention from many researchers since its ability to utilize a few data with labels and relatively more data without labels to learn information. Some existing semi-supervised methods for medical image segmentation enforce the regularization of training by implicitly perturbing data or networks to perform the consistency. Most consistency regularization methods focus on data level or network structure level, and rarely of them focus on the task level. It may not directly lead to an improvement in task accuracy. To overcome the problem, this work proposes a semi-supervised dual-task consistent joint learning framework with task-level regularization for 3D medical image segmentation. Two branches are utilized to simultaneously predict the segmented and signed distance maps, and they can learn useful information from each other by constructing a consistency loss function between the two tasks. The segmentation branch learns rich information from both labeled and unlabeled data to strengthen the constraints on the geometric structure of the target. Experimental results on two benchmark datasets show that the proposed method can achieve better performance compared with other state-of-the-art works. It illustrates our method improves segmentation performance by utilizing unlabeled data and consistent regularization.

In the context of Industry 4.0, the medical industry is horizontally integrating the medical resources of the entire industry through the Internet of Things (IoT) and digital interconnection technologies. Speeding up the establishment of the public retrieval database of diagnosis-related historical data is a common call for the entire industry. Among them, the Magnetic Resonance Imaging (MRI) retrieval system, which is one of the key tools for secure and private the Internet of Medical Things (IoMT), is significant for patients to check their conditions and doctors to make clinical diagnoses securely and privately. Hence, this paper proposes a framework named MRCG that integrates Convolutional Neural Network (CNN) and Graph Neural Network (GNN) by incorporating the relationship between multiple gallery images in the graph structure. First, we adopt a Vgg16-based triplet network jointly trained for similarity learning and classification task. Next, a graph is constructed from the extracted features of triplet CNN where each node feature encodes a query-gallery image pair. The edge weight between nodes represents the similarity between two gallery images. Finally, a GNN with skip connections is adopted to learn on the constructed graph and predict the similarity score of each query-gallery image pair. Besides, Focal loss is also adopted while training GNN to tackle the class imbalance of the nodes. Experimental results on some benchmark datasets, including the CE-MRI dataset and a public MRI dataset from the Kaggle platform, show that the proposed MRCG can achieve 88.64% mAP and 86.59% mAP, respectively. Compared with some other state-of-the-art models, the MRCG can also outperform all the baseline models.

Brain-computer interfaces (BCIs) provide users with a direct communication pathway between the brain and the peripheral environment. BCI-controlled devices have the potential to assist disabled patients in regaining motor functions. However, it remains unclear what happens to the functional coupling between the brain and muscle after BCI-based long-term motor training. Therefore, we developed a neurofeedback training method for long-term motor training that combines visual scenes and electrical stimulation. During the experiment, we collected electroencephalography (EEG) and electromyography (EMG) data from 20 subjects to explore their neurophysiological responses and the EEG-EMG coupling relationship. Event-related desynchronization (ERD), root mean square (rms) analysis, transfer entropy (TE) patterns, and other techniques were used to evaluate the cortical muscle response. Compared with the initial states, the ERD and rms significantly improved after long-term motor training. However, there was no significant difference in BCI performance. Directional TE values revealed the cortical muscle mechanism. These results demonstrate that incorporating EEG and EMG patterns to evaluate and establish a BCI-based motor training method is feasible. Furthermore, this article could provide evidence for functional coupling mechanisms for cortical muscles and motor rehabilitation.

Automatic seizure detection algorithms are necessary for patients with refractory epilepsy. Many excellent algorithms have achieved good results in seizure detection. Still, most of them are based on discontinuous intracranial electroencephalogram (iEEG) and ignore the impact of different channels on detection. This study aimed to evaluate the proposed algorithm using continuous, long-term iEEG to show its applicability in clinical routine. In this study, we introduced the ability of the transformer network to calculate the attention between the channels of input signals into seizure detection. We proposed an end-to-end model that included convolution and transformer layers. The model did not need feature engineering or format transformation of the original multi-channel time series. Through evaluation on two datasets, we demonstrated experimentally that the transformer layer could improve the performance of the seizure detection algorithm. For the SWEC-ETHZ iEEG dataset, we achieved 97.5% event-based sensitivity, 0.06/h FDR, and 13.7 s latency. For the TJU-HH iEEG dataset, we achieved 98.1% event-based sensitivity, 0.22/h FDR, and 9.9 s latency. In addition, statistics showed that the model allocated more attention to the channels close to the seizure onset zone within 20 s after the seizure onset, which improved the explainability of the model. This paper provides a new method to improve the performance and explainability of automatic seizure detection.

Humans obtain characteristic information such as texture and weight of external objects, relying on the brain's integration and classification of tactile information; however, the decoding mechanism of multi-level tactile information is relatively elusive from the temporal sequence. In this paper, nonvariant frequency, along with the variant pulse width of electrotactile stimulus, was performed to generate multi-level pressure sensation. Event-related potentials (ERPs) were measured to investigate the mechanism of whole temporal tactile processing. Five ERP components, containing P100-N140-P200-N200-P300, were observed. By establishing the relationship between stimulation parameters and ERP component amplitudes, we found the following: (1) P200 is the most significant component for distinguishing multi-level tactile sensations; (2) P300 is correlated well with the subjective judgment of tactile sensation. The temporal sequence of brain topographies was implemented to clarify the spatiotemporal characteristics of the tactile process, which conformed to the serial processing model in neurophysiology and cortical network response area described by fMRI. Our results can help further clarify the mechanism of tactile sequential processing, which can be applied to improve the tactile BCI performance, sensory enhancement, and clinical diagnosis for doctors to evaluate the tactile process disorders by examining the temporal ERP components.

Although the inter-subject information has been demonstrated to be effective for a rapid calibration of the P300-based brain-computer interface (BCI), it has never been comprehensively tested to find if the incorporation of heterogeneous data could enhance the accuracy. This study aims to improve the subject-specific P300 classifier by adding other subject's data. A classifier calibration strategy, weighted ensemble learning generic information (WELGI), was developed, in which elementary classifiers were constructed by using both the intra- and inter-subject information and then integrated into a strong classifier with a weight assessment. 55 subjects were recruited to spell 20 characters offline using the conventional P300-based BCI, i.e. the P300-speller. Four different metrics, the P300 accuracy and precision, the round accuracy, and the character accuracy, were performed for a comprehensive investigation. The results revealed that the classifier constructed on the training dataset in combination with adding other subject's data was significantly superior to that without the inter-subject information. Therefore, the WELGI is an effective classifier calibration strategy which uses the inter-subject information to improve the accuracy of subject-specific P300 classifiers, and could also be applied to other BCI paradigms.

The use of hand gestures provides an attractive alternative to cumbersome interface devices for human-computer interaction (HCI). However, the number of hand gestures has not been fully explored for HCI application. It is necessary to achieve more gestures as the command of interface. This paper proposed a method to recognize nine different hand gestures. Using camera to get images of people wearing pink gloves, and then preprocess those images by color splitting, morphological processing and edge extraction. Fourier descriptor, edge histogram and boundary moment invariants are three methods of feature extraction. At last, the template matching was used to realize the hand gesture recognition. The average recognition rate of the nine different gestures employing three different methods is 0.859.