We present **Brain Harmony (BrainHarmonix)**, the first multimodal brain foundation model that unifies structural morphology and functional dynamics into compact 1D token representations. The model was pretrained on two of the largest neuroimaging datasets to date, encompassing 64,594 T1-weighted structural MRI 3D volumes (～ 14 million images) and 70,933 functional MRI (fMRI) time series. BrainHarmonix is grounded in two foundational neuroscience principles: *structure complements function* - structural and functional modalities offer distinct yet synergistic insights into brain organization; *function follows structure* - brain functional dynamics are shaped by cortical morphology. The modular pretraining process involves single-modality training with geometric pre-alignment followed by modality fusion through shared brain hub tokens. Notably, our dynamics encoder uniquely handles fMRI time series with heterogeneous repetition times (TRs), addressing a major limitation in existing models. BrainHarmonix is also the first to deeply compress high-dimensional neuroimaging signals into unified, continuous 1D tokens, forming a compact latent space of the human brain. BrainHarmonix achieves strong generalization across diverse downstream tasks, including neurodevelopmental and neurodegenerative disorder classification and cognition prediction - consistently outperforming previous approaches. Our models - pretrained on 8 H100 GPUs - aim to catalyze a new era of AI-driven neuroscience powered by large-scale multimodal neuroimaging.

Cross-dataset driver drowsiness recognition with EEG is important for the advancement of a calibration-free driver drowsiness recognition system. Nevertheless, this task is challenging due to the impact of distribution drift on recognition accuracy. In this paper, we propose a novel model named entropy optimization network (EON) for the task. The model takes a novel two-step strategy to separate the unlabeled data from the target domain. It firstly uses a novel modified entropy loss to encourage unlabeled samples well aligned with the source domain to form clear clusters. Next, it gradually separates samples from the target domain with a self-training framework by taking adequate advantage of underlying patterns inherent in it. The proposed method is tested on the domain adaptation task with two public datasets and achieves 2-class recognition accuracies of and , which beats other baseline methods. Our work illuminates a promising direction in achieving the ultimate objective of developing a driver drowsiness recognition system without calibration.

The deep random vector functional link (dRVFL) and ensemble dRVFL (edRVFL) succeed in various tasks and achieve state-of-the-art performance compared with other randomized neural networks (NNs). However, existing edRVFL structures need more diversity and error correction ability in an independent network. Our work fills the gap by combining stacked deep blocks and residual learning with the edRVFL. Subsequently, we propose a novel dRVFL combined with residual learning, ResdRVFL, whose deep layers calibrate the wrong estimations from shallow layers. Additionally, we propose incorporating a scaling parameter to control the scaling of residuals from shallow layers, thus mitigating the risk of overfitting. Finally, we present an ensemble deep stacking network, SResdRVFL, based on ResdRVFL. SResdRVFL aggregates multiple blocks into a cohesive network, leveraging the benefits of deep learning and ensemble learning. We evaluate the proposed model on 28 datasets and compare it with the state-of-the-art methods. The comparative study demonstrates that the SResdRVFL is the best-performing approach in terms of average ranking and errors based on 28 datasets.

Fatigue is one of the main factors that contribute to operator performance and maritime safety, making it important to develop fatigue recognition algorithms that can predict operator fatigue. In this paper. we propose a subject independent algorithm for fatigue recognition from heart rate using machine learning techniques for human factors evaluation. The final model with the Random Forest Classifier produced a mean classification accuracy of $67.2 \%$ for recognizing 2-levels of stress for unseen data. With a 1-minute data window for fatigue recognition updated every second, the proposed method could be applied for human factors evaluation including vessel traffic operators’ fatigue monitoring.

The Random Vector Functional Link Network (RVFL) is a single-layer feed-forward network characterized by randomised weights in its hidden layers. However, the randomness can introduce detrimental neurons, potentially impairing the network’s performance. In response, this paper introduces multiple strategies to mitigate the noise from these randomised weights in RVFL networks. We first present a neuron normalization method that enhances latent space diversity and the network’s resilience to input features. Additionally, we develop improved approaches incorporating various feature selection and elimination techniques. Furthermore, Bayesian Optimization is utilized to optimize hyperparameters within a defined space. The efficacy of these methods is demonstrated through results from UCI classification tasks, highlighting the statistically superior performance of our Noise Eliminated edRVFL (NE-edRVFL) with neuron normalization.

We introduce *Brain-JEPA*, a brain dynamics foundation model with the Joint-Embedding Predictive Architecture (JEPA). This pioneering model achieves state-of-the-art performance in demographic prediction, disease diagnosis/prognosis, and trait prediction through fine-tuning. Furthermore, it excels in off-the-shelf evaluations (e.g., linear probing) and demonstrates superior generalizability across different ethnic groups, surpassing the previous large model for brain activity significantly. Brain-JEPA incorporates two innovative techniques: **Brain Gradient Positioning** and **Spatiotemporal Masking**. Brain Gradient Positioning introduces a functional coordinate system for brain functional parcellation, enhancing the positional encoding of different Regions of Interest (ROIs). Spatiotemporal Masking, tailored to the unique characteristics of fMRI data, addresses the challenge of heterogeneous time-series patches. These methodologies enhance model performance and advance our understanding of the neural circuits underlying cognition. Overall, Brain-JEPA is paving the way to address pivotal questions of building brain functional coordinate system and masking brain activity at the AI-neuroscience interface, and setting a potentially new paradigm in brain activity analysis through downstream adaptation.

Developing an effective and efficient electroencephalography (EEG)-based drowsiness monitoring system is crucial for enhancing road safety and reducing the risk of accidents. For general usage, cross-subject evaluation is indispensable. Despite progress in unsupervised domain adaptation (UDA) and source-free domain adaptation (SFDA) methods, these often rely on the availability of labeled source data or white-box source models, posing potential privacy risks. This study explores a more challenging setting of UDA for EEG-based drowsiness detection, termed black-box domain adaptation (BBDA). In BBDA, adaptation in the target domain relies solely on a black-box source model, without access to the source data or parameters of the source model. Specifically, we propose a framework called Self-distillation and Pseudo-labelling for Ensemble Deep Random Vector Functional Link (edRVFL)-based Black-box Knowledge Adaptation (SPARK). SPARK employs entropy-based selection of high-confidence samples, which are then pseudo-labeled to train a student edRVFL network. Subsequently, ensemble self-distillation is performed to extract knowledge by training the edRVFL using refined labels introduced by ensemble learning. This process further improves the robustness of the student edRVFL network. The features of the edRVFL are beneficial for improving the computational efficiency of the framework, making it more suitable for tasks involving small datasets. The proposed SPARK framework is evaluated on two publicly available driver drowsiness datasets. Experimental results demonstrate its superior performance over strong baselines, while significantly reducing training time. These findings underscore the potential for practical integration of the proposed framework into drowsiness monitoring systems, thereby contributing substantially to the privacy preservation of source subjects.

Being in a state of stress can affect operators’ performance and might lead to a decrease in attention and cognition, which could affect vessel operational safety within the Vessel Traffic Management System. By directly measuring operator’s biosignals, it is possible to predict the level of his/her stress for corrective action to be taken, such as getting support from colleagues if needed. One of the convenient ways to monitor operators’ stress is using mobile heart rate devices. In this paper, we propose a cross-subject stress recognition algorithm utilizing Heart Rate Variability (HRV) features and the Adaboost classifier. The algorithm is subject independent and can be calibrated using previously collected stress data to recognize the 2-level stress states of an unseen subject with 82.4% accuracy.

This work investigated the use of an ensemble deep random vector functional link (edRVFL) network for electroencephalogram (EEG)-based driver fatigue recognition. Against the low feature learning capability of the edRVFL network from raw EEG signals, two strategies were exploited in this work. Specifically, the first one was to exploit the advantages of the feature extractor module in CNNs, i.e., use CNN features as the input of the edRVFL network. The second one was to improve the feature learning capability of the edRVFL network. An enhanced edRFVL network named FGloWD-edRVFL was proposed, in which four enhancements were implemented, including random forest-based Feature selection, Global output layer, Weighting and entropy-based Dynamic ensemble. The proposed FGloWD-edRVFL network was evaluated on the challenging cross-subject driver fatigue recognition tasks. The results indicated that the proposed model could boost the recognition performance, significantly outperforming all strong baselines. The step-wise analysis further demonstrated the effectiveness of the proposed enhancements in the edRVFL network.

Randomized neural networks (RNNs) have shown outstanding performance in many different fields. The superiority of having fewer training parameters and closed-form solutions makes them popular in small datasets analysis. However, automatically decoding raw electroencephalogram (EEG) data using RNNs is still challenging in EEG-based passive brain–computer interface (pBCI) classification tasks. Models with the high-dimension input of EEG may suffer from overfitting and the intrinsic characteristics of non-stationary, high-level noises and subject variability could limit the generation of distinctive features in the hidden layers. To address these problems in EEG-based pBCI tasks, this work proposes a spectral-ensemble deep random vector functional link (SedRVFL) network that focuses on feature learning in the frequency domain. Specifically, an unsupervised feature-refining (FR) block is proposed to improve the low feature learning capability in RNNs. Moreover, a dynamic direct link (DDL) is performed to further complement the frequency information. The proposed model has been evaluated on a self-collected dataset as well as a public driving dataset. The cross-subject classification results obtained demonstrated its effectiveness. This work offers a new solution for EEG decoding, i.e., using optimized RNNs for decoding complex raw EEG data and boosting the classification performance of EEG-based pBCI tasks.

Electroencephalogram (EEG) has become increasingly popular in driver fatigue monitoring systems. Several decomposition methods have been attempted to analyze the EEG signals that are complex, nonlinear and non-stationary and improve the EEG decoding performance in different applications. However, it remains challenging to extract more distinguishable features from different decomposed components for driver fatigue recognition. In this work, we propose a novel decomposition-based hybrid ensemble convolutional neural network (CNN) framework to enhance the capability of decoding EEG signals. Four decomposition methods are employed to disassemble the EEG signals into components of different complexity. Instead of handcraft features, the CNNs in this framework directly learn from the decomposed components. In addition, a component-specific batch normalization layer is employed to reduce subject variability. Moreover, we employ two ensemble modes to integrate the outputs of all CNNs, comprehensively exploiting the diverse information of the decomposed components. Against the challenging cross-subject driver fatigue recognition task, the models under the framework all showed superior performance to the strong baselines. Specifically, the performance of different decomposition methods and ensemble modes was further compared. The results indicated that discrete wavelet transform-based ensemble CNN achieved the highest average classification accuracy of 83.48% among the compared methods. The proposed framework can be extended to any CNN architecture and be applied to any EEG-related tasks, opening the possibility of extracting more beneficial features from complex EEG data.

This paper proposes a three-stage online deep learning model for time series based on the ensemble deep random vector functional link (edRVFL). The edRVFL stacks multiple randomized layers to enhance the single-layer RVFL's representation ability. Each hidden layer's representation is utilized for training an output layer, and the ensemble of all output layers forms the edRVFL's output. However, the original edRVFL is not designed for online learning, and the randomized nature of the features is harmful to extracting meaningful temporal features. In order to address the limitations and extend the edRVFL to an online learning mode, this paper proposes a dynamic edRVFL consisting of three online components, the online decomposition, the online training, and the online dynamic ensemble. First, an online decomposition is utilized as a feature engineering block for the edRVFL. Then, an online learning algorithm is designed to learn the edRVFL. Finally, an online dynamic ensemble method, which can measure the change in the distribution, is proposed for aggregating all layers' outputs. This paper evaluates and compares the proposed model with state-of-the-art methods on sixteen time series.

Since situation awareness (SA) plays an important role in many fields, the measure of SA is one of the most concerning problems. Using physiological signals to evaluate SA is becoming a popular research topic because of their advantages of non-intrusiveness and objectivity. However, previous studies mainly exploited the use of single physiological signals such as electroencephalogram (EEG) or eye tracking. The multi-modal SA recognition is still a research gap. Therefore, this work conducts a pilot study to investigate SA recognition by using two modalities: EEG and eye tracking data. Specifically, an optimized Stroop test that is more compatible with the definition of SA was used to induce different states of SA and collect physiological data. Furthermore, a random vector functional link-based stacking (RVFL-S) model was proposed to perform the multi-modal SA recognition. Experiment results showed that using the combination of EEG and eye tracking data can boost the performance of SA recognition. Moreover, the proposed RVFL-S model can effectively integrate the classification information from two modalities. It showed better performance than baseline methods, achieving 77.62% leave-one-subject-out (LOSO) average accuracy. This was around 5% improvement compared with the baseline classification models with input of only one modality. This pilot study demonstrated that the use of multi-modality is a potential strategy for SA recognition.

Deep learning for electroencephalogram-based classification is confronted with data scarcity, due to the time-consuming and expensive data collection procedure. Data augmentation has been shown as an effective way to improve data efficiency. In addition, contrastive learning has recently been shown to hold great promise in learning effective representations without human supervision, which has the potential to improve the electroencephalogram-based recognition performance with limited labeled data. However, heavy data augmentation is a key ingredient of contrastive learning. In view of the limited number of sample-based data augmentation in electroencephalogram processing, three methods, performance-measure-based time warp, frequency noise addition and frequency masking, are proposed based on the characteristics of electroencephalogram signal. These methods are parameter learning free, easy to implement, and can be applied to individual samples. In the experiment, the proposed data augmentation methods are evaluated on three electroencephalogram-based classification tasks, including situation awareness recognition, motor imagery classification and brain-computer interface steady-state visually evoked potentials speller system. Results demonstrated that the convolutional models trained with the proposed data augmentation methods yielded significantly improved performance over baselines. In overall, this work provides more potential methods to cope with the problem of limited data and boost the classification performance in electroencephalogram processing.

Extracting intra- and inter-subject parameters from Electroencephalogram (EEG) representing different Situation Awareness (SA) status is a critical challenge for objective SA recognition. Most of the existing work focuses on the subject-dependent classification that applies power spectrum density (PSD) features. In this paper, we propose a novel spectral-spatial (S-S) model for cross-subject fatigue-related SA recognition. The S-S model not only considers the biological topology across different brain regions to capture both local and global relations among different EEG channels, but also extracts spectral features for each EEG channel. Specifically, we firstly model the topological structure of EEG channels via an adjacency matrix which is built based on the Euclidean distance between EEG channels. Then, the graph convolution operation is employed to perform the neighbourhood aggregation for extracting spatial features. We test our model on a public dataset collected during driver’s task performance. The subject-independent performance of the model is explored. Results demonstrate (1) the superior performance of our model compared with the state-of-the-art models on SA recognition from EEG signals. Specifically, our S-S model achieves 70.6% accuracy which is higher than traditional machine learning methods by 2.7%-6.8% and deep learning methods by 10.3%-11.6%; (2) EEG signal at the occipital region can better reflect the change of SA.