Sleep is important to human health. To improve sleep quality, recorded EEG signals have been utilized for automated sleep staging either in real time during sleep or after sleep. However, because previous approaches classify events that have already occurred rather than forecasting the future, their effectiveness is limited for personalized sleep management. This study proposes the sleep stage forecaster with sleep EEG tokenizer (SSF-SET) framework, which predicts the future sleep-stage sequence using only earlier EEG of the current sleep. SET combines a multi-branch transformer for epoch-level representations, an LSTM-based sequence encoder-decoder, and quantization to convert continuous EEG features into sleep-informative tokens. This quantization makes an information bottleneck that reveals the latent transition structure and suppresses artifacts, enabling reliable next-stage prediction. The decoder-only transformer SSF is first pretrained for next-token prediction with causal attention, then fine-tuned via reinforcement learning that uses sequence-level macro-F1 and token-consistency rewards; throughout it does not access future EEG at inference. With subject-wise cross-validation on the SleepEDF20 and SleepEDF78 datasets, SSF-SET consistently outperformed direct forecasting and forecasting with predicted sleep stages. On SleepEDF20, accuracy was 0.596 and macro-F1 was 0.516. In addition, we achieved an accuracy of 0.611 and a macro-F1 of 0.537 on SleepEDF78. These results show that quantized sleep EEG tokens are effective for autoregressive prediction and demonstrate that future sleep stages can be predicted without future EEG. We believe that SSF-SET is an important component for developing closed-loop and personalized sleep interventions that can act before disruptive transitions occur, and we expect it to improve sleep quality.

Brain-computer interface (BCI) technology enables communication between humans and devices by reflecting users' status and intentions. Electroencephalography (EEG) signals are utilized to capture brain electrical activity with no surgical operation. When conducting motor imagery (MI), one of the endogenous BCI paradigms, the users imagine the movement of muscles used when performing a certain movement without actual physical movement. However, not all subjects show outstanding classification performance in decoding MI-based EEG signals. We propose the novel method that utilizes the weights of the pre-trained model to generate the personalized weights, effectively combining the general MI features with the personalized features. We used the 5-fold cross-validation for evaluating the performances, and conducted the experiments in 3 different pre-trained models (Top-3, Top-5, and Top-7). We compared the performances of our proposed method using the baseline and the full fine-tuning. In comparison to our proposed method with the baseline, our proposed method achieved the improvement of the average accuracies in all pre-trained models, and those values were 0.123, 0.138, and 0.143, respectively. When comparing our proposed method with the full fine-tuning, the average accuracies of our proposed method were the highest in all pre-trained models, and the differences in the average accuracies were 0.012, 0.019, and 0.009, respectively. Hence, we demonstrated the possibility of improving the precision and effectiveness of the EEG-based systems by reflecting the individual differences in EEG signals among the subjects with low classification accuracy.

Brain-computer interface (BCI) enables the communication between humans and devices by reflecting humans' intentions and status. Endogenous BCI is the imagined-based BCI and it has the advantage that the fatigue level of the body, especially the eyes, is relatively low and no additional equipment for offering stimulation is required. When conducting imagined speech, one of the endogenous BCI paradigms, the users imagine the pronunciation as if actually speaking. In contrast, overt speech is that the users directly pronounce the words. We proposed the transfer learning-based method from overt speech- to imagined speech-based electroencephalogram (EEG) signals (TOINet). The proposed method utilizes an encoder to extract the feature vector of imagined speech from EEG signals, which is subsequently reconstructed into overt speech signals using the decoder. Through this process, the model can identify the significant and common features present in EEG signals for both overt and imagined speech, facilitating the classification of EEG signals associated with imagined speech. Eight subjects participated in the experiment. The average accuracy of the TOINet was 0.4841 for classifying four words and the EEG features of overt speech improved the performance by 0.0742. Hence, we demonstrated that EEG features of overt speech could improve the decoding performance of imagined speech.

Detection of the pilots’ mental states is particularly critical because their abnormal mental states (AbSs) could cause catastrophic accidents. In this study, we presented the feasibility of classifying the various specific AbSs (namely, low fatigue, high fatigue, low workload, high workload, low distraction, and high distraction) by applying the deep learning method. To the best of our knowledge, this study is the first attempt to classify multiple AbSs of pilots. We proposed the hybrid deep neural networks with five convolutional blocks and two long short-term memory layers for decoding multiple AbSs. We designed the model to extract the informative features from electroencephalography signals. A total of ten pilots conducted the experiment in a simulated flight environment. Compared with five conventional models, our proposed model achieved the highest grand-average accuracy of 68.04$(\pm$5.26)% which is at least 6.55% higher than other conventional models for classifying seven mental states across all subjects. Our proposed model could distinguish and classify low and high levels for each status category and give appropriate feedback to the subjects. In addition, we found nine indicators that showed the statistically significant differences between two mental states (p$<$0.05). Hence, we believe that it will contribute significantly to autonomous driving or autopilot advances based on artificial intelligence technology in the future.

Brain-computer interface (BCI) is a technology that controls computers by reflecting users’ intentions. Especially the electroencephalogram (EEG)-based BCI systems have been developed because of their potential utility. In BCI studies, controlling the drone swarm is one of the important issues since it improves work efficiency and safety. Also, current research has investigated how the drone swarms are controlled by imagining their formations using visual imagery (VI)-based EEG signals. The raw EEG signals and the spectrogram are widely used as input representations for decoding EEG signals. However, the decoding performance of the VI-based EEG signals is low to control the drone swarm due to noise in the raw EEG signals and information loss problems that may arise in the spectrogram. In this paper, we develop the EEGgram generator that extracts spectrogram-like features from the raw EEG signals minimizing information loss problems. Also, we propose the EEGgramNet, which could extract the significant information from VI-based EEG signals using both the spectrogram and the EEGgram as inputs. The proposed method outperforms an accuracy of 0.643, which is 8.4 % higher than that of the best conventional method. Hence, we demonstrate the possibility of constructing a VI-based BCI system to control the drone swarm by imagining its formations.

Brain-computer interface (BCI) is used for communication between humans and devices by recognizing humans’ status and intention. Communication between humans and a drone using electroencephalogram (EEG) signals is one of the most challenging issues in the BCI domain. In particular, the control of drone swarms (the direction and formation) has more advantages compared to the control of a drone. The visual imagery (VI) paradigm is that subjects visually imagine specific objects or scenes. Reduction of the variability among subjects’ EEG signals is essential for practical BCI-based systems. In this study, we proposed the subepoch-wise feature encoder (SEFE) to improve the performances in the subject-independent tasks by using the VI dataset. This study is the first attempt to demonstrate the possibility of generalization among subjects in the VI-based BCI. We used the leave-one-subject-out cross-validation for evaluating the performances. We obtained higher performances when including our proposed module than excluding our proposed module. The DeepConvNet with SEFE showed the highest performance of 0.72 among six different decoding models. Hence, we demonstrated the feasibility of decoding the VI dataset in the subject-independent task with robust performances by using our proposed module.