As transportation systems grow in complexity and autonomous vehicle technologies advance, detecting the mental states of drivers and passengers is crucial for enhancing traffic safety and user experience. Recent studies have explored multi-biosignal fusion to monitor these states. Despite these efforts, existing methods fail to reflect the complexity and usefulness of various modalities, and do not consider the informativeness of biosignals based on signal quality or the contribution of each modality as biomarkers. To address this issue, we introduce dynamic multi-biosignal fusion (DMBF) to detect the mental states of drivers and passengers. DMBF employs a dynamic gate mechanism that estimates reliability based on data quality through confidence-aware learning, integrating this information into the learning process. Moreover, a spatial-temporal attention module is utilized to capture and combine key biosignal patterns across channels and time. Extensive evaluations were conducted on five datasets related to motion sickness, drowsiness, distraction, emotions, and sustained attention. Each dataset yielded F1 scores of 0.5569, 0.7187, 0.6647, 0.9378, and 0.8092, respectively, outperforming existing baseline models. The experimental results and ablation studies have demonstrated that DMBF is a more robust and versatile method for detecting the mental states of drivers and passengers. DMBF provides a foundation for future enhancements in traffic safety and passenger experience and has the potential for broader applications in multi-biosignal monitoring systems.

Drowsy driving is one of the greatest threats to road safety, which increases the importance of intelligent systems that can monitor driver drowsiness. Electroencephalogram (EEG)–based monitoring systems have gained attention because EEG is known to directly measure brain activities that reflect the mental state of the driver. However, calibration is necessary before using the system because EEG signals vary between and within subjects. Therefore, generalized EEG-based drowsiness estimation has become challenging. In this paper, we propose an EEG-based driver drowsiness classification framework without the need for calibration, which can be generalized to unseen subjects. We augment the features of unseen domains (i.e., subjects) with a Dirichlet mixup between prototypes of source domains to complement other domain knowledge. The parameter$\boldsymbol{\alpha}$vector of the Dirichlet distribution adjusts the intensity of the mixup, allowing for diverse enhancement. Furthermore, we utilize an auxiliary batch normalization module for augmented samples to avoid inaccurate estimation by the difference in distribution. The experiments were carried out using two EEG datasets, each measured using different drowsiness indicators, the Karolinska sleepiness scale, and reaction time. In leave-one-subject-out cross-validation, the proposed framework achieved outstanding performance in both datasets, anF1-score of 62.69% and 70.33% and an area under the receiver operating characteristic curve (AUROC) of 71.73% and 73.80%, respectively. The experimental results demonstrate the potential for practical applications of brain-computer interfaces without calibration.

Many people experience motion sickness. In order to analyze a driver’s motion sickness state and prevent accidents, a method of estimating the degree of motion sickness based on bio-signals is emerging. The brain-computer interface (BCI) systems using electroencephalogram (EEG) are used as the most direct method of estimating motion sickness conditions. However, EEG-based systems suffer from variability between subjects and over time, so a calibration process is required for every use. To address this problem, we mitigate the need for calibration through cross-subject transfer learning between the target data and the multi-subjects source data. All experiments were conducted in a domain adaptation setting. Meanwhile, we assume that there is an ordinal relationship between motion sickness scores. Thus, we performed an ordinal classification task so that the feature vectors were mapped by reflecting the ordinal characteristics according to the motion sickness state. In this paper, we propose a motion sickness classification BCI framework in combination with ordinal classification, resting-state prototype-based ordinal distance learning, and a subject-specific embedding module. Taking into account constraints of ordinal rank, the feature extractor is trained with prototype-based ordinal distance learning to measure the relative distance between the resting-state and motion sickness state. We further utilize an embedding module that encodes subject-specific information combined with task discriminative features to be effective for domain adaptation tasks. The proposed framework achieved the highest performance (accuracy 60.21 %) through comparative experiments with other models.

Drowsy driving causes severe road traffic accidents and significantly threatens road driving. Recently, electroencephalogram (EEG)-based drowsiness state classification has gained attention in the field of brain-computer interface (BCI). Because of the inter-and intra-subject variability of EEG signals, EEG-based drowsiness state classification is still challenging in developing an estimator applicable to unseen subjects. Generally, calibration sessions are required to tune the model with subject-specific data. In this paper, we propose an EEG-based driver drowsiness state (i.e., alert and drowsy) classification framework that improves the generalization performance to unseen subjects. Style features of multi-domain instances are mixed to generate unseen domains, and the distance of labels within classes is minimized to learn robust representations. Experiments were conducted on EEG data acquired from a drowsy driving experiment in a simulated-driving environment. Our proposed framework achieved an accuracy of 77.26%, an F1-score of 0.6266, and a recall of 0.6813 across eleven subjects in leave-one-subject-out cross-validation. The experimental results showed an improvement in the generalization performance for novel target subjects in driver drowsiness state classification and demonstrated the potential for calibration-free BCI.

Brain-computer interface (BCI) is used for communication between humans and devices by recognizing humans’ status and intention. Communication between humans and a drone using electroencephalogram (EEG) signals is one of the most challenging issues in the BCI domain. In particular, the control of drone swarms (the direction and formation) has more advantages compared to the control of a drone. The visual imagery (VI) paradigm is that subjects visually imagine specific objects or scenes. Reduction of the variability among subjects’ EEG signals is essential for practical BCI-based systems. In this study, we proposed the subepoch-wise feature encoder (SEFE) to improve the performances in the subject-independent tasks by using the VI dataset. This study is the first attempt to demonstrate the possibility of generalization among subjects in the VI-based BCI. We used the leave-one-subject-out cross-validation for evaluating the performances. We obtained higher performances when including our proposed module than excluding our proposed module. The DeepConvNet with SEFE showed the highest performance of 0.72 among six different decoding models. Hence, we demonstrated the feasibility of decoding the VI dataset in the subject-independent task with robust performances by using our proposed module.