Recent advancements in imitation learning have shown promising results in robotic manipulation, driven by the availability of high-quality training data. To improve data collection efficiency, some approaches focus on developing specialized teleoperation devices for robot control, while others directly use human hand demonstrations to obtain training data. However, the former requires both a robotic system and a skilled operator, limiting scalability, while the latter faces challenges in aligning the visual gap between human hand demonstrations and the deployed robot observations. To address this, we propose a human hand data collection system combined with our hand-to-gripper generative model, which translates human hand demonstrations into robot gripper demonstrations, effectively bridging the observation gap. Specifically, a GoPro fisheye camera is mounted on the human wrist to capture human hand demonstrations. We then train a generative model on a self-collected dataset of paired human hand and UMI gripper demonstrations, which have been processed using a tailored data pre-processing strategy to ensure alignment in both timestamps and observations. Therefore, given only human hand demonstrations, we are able to automatically extract the corresponding SE(3) actions and integrate them with high-quality generated robot demonstrations through our generation pipeline for training robotic policy model. In experiments, the robust manipulation performance demonstrates not only the quality of the generated robot demonstrations but also the efficiency and practicality of our data collection method. More demonstrations can be found at: https://rwor.github.io/.

The deployment of large-scale data analytics between on-premise and cloud sites, i.e., hybrid clouds, requires careful partitioning of both data and computation to avoid massive networking costs. We present Moirai, a cost-optimization framework that analyzes job accesses and data dependencies and optimizes the placement of both in hybrid clouds. Moirai informs the job scheduler of data location and access predictions, so it can determine where jobs should be executed to minimize data transfer costs. Our optimizer achieves scalability and cost efficiency by exploiting recurring jobs to identify data dependencies and job access characteristics and reduces the search space by excluding data not accessed recently.

Cooperative AI and AI alignment research are increasingly important fields of study as machine learning models are becoming more prevalent in society. Applications such as self-driving cars, realistic AI in games, and human-AI teams, all require further advancement in cooperative and alignment research before more widespread applications can be achieved. However, research in these fields has typically lagged behind other machine learning applications due to the difficulty of creating models that are robust to and can adapt to novel human partners. We attempt to address this through the creation of a framework that uses Archetypal Analysis, a unique clustering algorithm that finds extremal ‘archetype’ points in a dataset and expresses each other point as a convex combination of these archetypes. This framework creates understandable archetypes of players which a reinforcement learning agent can use to adapt accordingly to unseen partners. We show that this framework not only results in performance comparable to other cooperative benchmark models but also achieves higher levels of perceived cooperativeness without the need for human involvement during the training process. As such, we demonstrate that the use of clustering techniques to better model different types of human behaviour and strategies can be an effective approach in improving the ability of AI models to adapt to and improve cooperation with novel partners.

Driving under drowsy conditions significantly escalates the risk of vehicular accidents. Recent endeavors to prevent driving accidents have focused on using electroencephalography (EEG) to identify drowsy mental states. However, a more versatile EEG-based assistant system is better equipped to achieve smooth human-machine interaction in driving scenarios. This system should be capable of understanding a driver’s intention while demonstrating resilience to artifacts induced by unexpected sudden movements. Building on this goal, this paper pioneers a novel research direction in the field of brain-computer interface for assisted driving, studying the neural patterns associated with driving intentions and presenting a novel method for driving intention prediction. In particular, our preliminary analysis of the EEG signal using independent component analysis suggests a close relation between the intention of driving maneuvers and the neural activities in central-frontal and parietal areas. Power spectral density analysis at a group level also reveals a notable distinction among various driving intentions in the frequency domain. To exploit these brain dynamics, we propose a novel Masked EEG Modeling (MEM) framework for predicting human driving intentions, including the intention for left turning, right turning, and straight proceeding. Extensive experiments, encompassing comprehensive quantitative and qualitative assessments on a publicly available driving dataset, demonstrate the proposed MEM method is proficient in predicting driving intentions across various vigilance states. Specifically, our model attains an accuracy of 85.19% when predicting driving intentions for drowsy subjects, which shows its promising potential for mitigating traffic accidents related to drowsy driving. Ablation also demonstrates that our method significantly enhances the flexibility in handling missing channels. Notably, our method maintains over 75% accuracy when more than half of the channels are missing or corrupted, underscoring its adaptability in real-life driving scenarios.

Prompting has become a practical method for utilizing pre-trained language models (LMs). This approach offers several advantages. It allows an LM to adapt to new tasks with minimal training and parameter updates, thus achieving efficiency in both storage and computation. Additionally, prompting modifies only the LM's inputs and harnesses the generative capabilities of language models to address various downstream tasks in a unified manner. This significantly reduces the need for human labor in designing task-specific models. These advantages become even more evident as the number of tasks served by the LM scales up. Motivated by the strengths of prompting, we are the first to explore the potential of prompting speech LMs in the domain of speech processing. Recently, there has been a growing interest in converting speech into discrete units for language modeling. Our pioneer research demonstrates that these quantized speech units are highly versatile within our unified prompting framework. Not only can they serve as class labels, but they also contain rich phonetic information that can be re-synthesized back into speech signals for speech generation tasks. Specifically, we reformulate speech processing tasks into speech-to-unit generation tasks. As a result, we can seamlessly integrate tasks such as speech classification, sequence generation, and speech generation within a single, unified prompting framework. The experiment results show that the prompting method can achieve competitive performance compared to the strong fine-tuning method based on self-supervised learning models with a similar number of trainable parameters. The prompting method also shows promising results in the few-shot setting. Moreover, with the advanced speech LMs coming into the stage, the proposed prompting framework attains great potential.

The explainability of reinforcement learning (RL) models has received vast amount of interest as its applications have widened. Most existing explainable RL models focus on improving the explainability of an agent's observations instead of the relationships between agent states and actions. This study presents a fuzzy centered explainable network (FCEN) for RL tasks to interpret the relationships between agent states and actions. The proposed FCEN leverages the interpretability of fuzzy neural networks to establish if–then rules and a generative model to visualize learned knowledge. Precisely, the FCEN includes if–then rules that formulate state-action mappings with human-understandable logic, such as the form “IF Input is A THEN Output is B.” In addition, these rules connect with a generative model that concretizes the states into human-understandable patterns (figures). Our experimental results obtained on 4 Atari games show that the proposed FCEN can achieve a high level of performance in RL tasks and enormously boost the explainability of RL agents both globally and locally. In other words, the FCEN maintains a high-level explanation for the agent decision logic and the possibility of low-level analysis for each given observation sample. The explainability boost does not undermine reward learning performance, humans can even enhance the agent's performance with the provided explainability.

The translation of brain dynamics into natural language is pivotal for brain-computer interfaces (BCIs), a field that has seen substantial growth in recent years. With the swift advancement of large language models, such as ChatGPT, the need to bridge the gap between the brain and languages becomes increasingly pressing. Current methods, however, require eye-tracking fixations or event markers to segment brain dynamics into word-level features, which can restrict the practical application of these systems. These event markers may not be readily available or could be challenging to acquire during real-time inference, and the sequence of eye fixations may not align with the order of spoken words. To tackle these issues, we introduce a novel framework, DeWave, that integrates discrete encoding sequences into open-vocabulary EEG-to-text translation tasks. DeWave uses a quantized variational encoder to derive discrete codex encoding and align it with pre-trained language models. This discrete codex representation brings forth two advantages: 1) it alleviates the order mismatch between eye fixations and spoken words by introducing text-EEG contrastive alignment training, and 2) it minimizes the interference caused by individual differences in EEG waves through an invariant discrete codex. Our model surpasses the previous baseline (40.1 and 31.7) by 3.06% and 6.34\%, respectively, achieving 41.35 BLEU-1 and 33.71 Rouge-F on the ZuCo Dataset. Furthermore, this work is the first to facilitate the translation of entire EEG signal periods without the need for word-level order markers (e.g., eye fixations), scoring 20.5 BLEU-1 and 29.5 Rouge-1 on the ZuCo Dataset, respectively.

Abstract In the financial market, the stock price prediction is a challenging task which is influenced by many factors. These factors include economic change, politics and global events that are usually recorded in text format, such as the daily news. Therefore, we assume that real-world text information can be used to forecast stock market activity. However, only a few works considered both text and numerical information to predict or analyse stock trends. These works used preprocessed text features as the model inputs; therefore, latent information in text may be lost because the relationships between the text and stock price are not considered. In this paper, we propose a fusion network, i.e. a spatial-temporal attention-based convolutional network (STACN) that can leverage the advantages of an attention mechanism, a convolutional neural network and long short-term memory to extract text and numerical information for stock price prediction. Benefiting from the utilisation of an attention mechanism, reliable text features that are highly relevant to stock value can be extracted, which improves the overall model performance. The experimental results on real-world stock data demonstrate that our STACN model and training scheme can handle both text and numerical data and achieve high accuracy on stock regression tasks. The STACN is compared with CNNs and LSTMs with different settings, e.g. a CNN with only stock data, a CNN with only news titles and LSTMs with only stock data. CNNs considering only stock data and news titles have mean squared errors of 28.3935 and 0.1814, respectively. The accuracy of LSTMs is 0.0763. The STACN can achieve an accuracy of 0.0304, outperforming CNNs and LSTMs in stock regression tasks.

Brain-computer interface (BCI) technologies are popular methods of communication between the human brain and external devices. One of the most popular approaches to BCI is motor imagery (MI). In BCI applications, the electroencephalography (EEG) is a very popular measurement for brain dynamics because of its noninvasive nature. Although there is a high interest in the BCI topic, the performance of existing systems is still far from ideal, due to the difficulty of performing pattern recognition tasks in EEG signals. This difficulty lies in the selection of the correct EEG channels, the signal-to-noise ratio of these signals, and how to discern the redundant information among them. BCI systems are composed of a wide range of components that perform signal preprocessing, feature extraction, and decision making. In this article, we define a new BCI framework, called enhanced fusion framework, where we propose three different ideas to improve the existing MI-based BCI frameworks. First, we include an additional preprocessing step of the signal: a differentiation of the EEG signal that makes it time invariant. Second, we add an additional frequency band as a feature for the system: the sensorimotor rhythm band, and we show its effect on the performance of the system. Finally, we make a profound study of how to make the final decision in the system. We propose the usage of both up to six types of different classifiers and a wide range of aggregation functions (including classical aggregations, Choquet and Sugeno integrals, and their extensions and overlap functions) to fuse the information given by the considered classifiers. We have tested this new system on a dataset of 20 volunteers performing MI-based brain-computer interface experiments. On this dataset, the new system achieved 88.80% accuracy. We also propose an optimized version of our system that is able to obtain up to 90.76%. Furthermore, we find that the pair Choquet/Sugeno integrals and overlap functions are the ones providing the best results.

In this article, we develop moderate deviation functions to measure similarity and dissimilarity among a set of given interval-valued data to construct interval-valued aggregation functions, and we apply these functions in two motor-imagery brain–computer interface (MI-BCI) systems to classify electroencephalography signals. To do so, we introduce the notion of interval-valued moderate deviation function and, in particular, we study those interval-valued moderate deviation functions, which preserve the width of the input intervals. In order to apply them in an MI-BCI system, we first use fuzzy implication operators to measure the uncertainty linked to the output of each classifier in the ensemble of the system, and then we perform the decision making phase using the new interval-valued aggregation functions. We have tested the goodness of our proposal in two MI-BCI frameworks, obtaining better results than those obtained using other numerical aggregation and interval-valued ordered weighted averaging operators, and obtaining competitive results versus some nonaggregation-based frameworks.

Distracted driving refers to multisensory integration and attention shifts between attentional driving and different interferences from different modalities, including visual and auditory stimuli. Here, we compared the behavioral performance with interacting multisensory distractors during attentional driving. Then, the independent component analysis (ICA) and event-related spectral perturbation (ERSP) were applied to investigate the neural oscillation changes. The behavioral results showed that the response times (RTs) increased when distractors appeared in response to attentional driving. Moreover, the RTs were longer when the distractor interference was presented in the auditory modality compared with the visual modality. Eye movement intervals showed shorter tracking saccades under distractor interference. These results may indicate that attentional driving performance was impaired under the exposure to multisensory distractor interference. The ERSPs under visual and auditory distraction exposure showed decreased beta power in the frontal area, increased theta and delta power in the central area, and decreased alpha power in the parietal area. During this process, distracted driving under cross-modal sensory interference required more neural oscillation involvement. Moreover, the visual modality showed increased gamma power in the frontal, central, parietal and occipital areas, while the auditory modality showed decreased gamma power in the frontal area, indicating that auditory interference could intervene in top-down attentional processing.

Neurons usually converse through electrochemical signals and pooled neuronal firings feasibly be recorded on the scalp through the medium of electroencephalogram (EEG). EEG waveforms are recorded, analysed and categorized across directives concerning a Brain-Computer Interface (BCI). Deteriorated signal to noise ratio and non-stationarities stand as a paramount obstacle in steady decoding of EEG. Appearance of non-stationarities across EEG patterns notably upset the feature waveforms thus worsening the functioning of detection block and as a whole the Brain Computer Interface. Stationary Subspace schemes bring to light subspaces within which data distribution persists stably over time. Current work focuses on the development of a novel spatial transform based feature extraction scheme to address nonstationarity in EEG signals recorded against a drowsiness detection problem (a machine learning regression scenario). The presented approach: F-DIV-IT-JAD-WS derived features distinctly surpassed DivOVR-FuzzyCSP-WS based standard features across RMSE and CC performance criteria pair. We construe that the propounded feature derivation approach based on F-DIV-IT-JAD-WS will usher a significant attention in researchers who are developing algorithms for signal processing, specifically, for BCI regression scenarios.

In physical human-robot collaboration (pHRC), singularity avoidance strategies are often critical to obtain stable interaction dynamics. It is hypothesised a predictable singularity avoidance strategy is preferred in pHRC as humans tend to maximise predictability when using complex systems. By using an electroencephalogram (EEG), it is possible to assess the predictability of a task through a feature found in event-related potentials (ERP) and called prediction-error negativity (PEN). In this paper, two research questions are addressed. Can a complex pHRC singularity avoidance strategy generate a detectable PEN? Are PEN and human preferences related when comparing different control settings in a singularity avoidance strategy? Fourteen participants compared two different sets of parameters (modes) in a singularity avoidance strategy based on the exponentially damped least-squared (EDLS) method. ERP results are presented in terms of power spectral density (PSD). ERP results were then compared with human preferences to see whether they are related. Results show that the mode that causes PEN is also the one that participants did not like, suggesting that a lack of predictability might have an impact on human preference.

Vehicle accidents are the primary cause of fatalities worldwide. Most often, experiencing fatigue on the road leads to operator errors and behavioral lapses. Thus, there is a need to predict the cognitive state of drivers, particularly their fatigue level. Electroencephalography (EEG) has been demonstrated to be effective for monitoring changes in the human brain state and behavior. Thirty-seven subjects participated in this driving experiment and performed a perform lane-keeping task in a visual-reality environment. Three domains, namely, frequency, temporal, and 2-D spatial information, of the EEG channel location were comprehensively considered. A 4-D convolutional neural-network (4-D CNN) algorithm was then proposed to associate all information from the EEG signals and the changes in the human state and behavioral performance. A 4-D CNN achieves superior forecasting performance over 2-D CNN, 3-D CNN, and shallow networks. The results showed a 3.82% improvement in the root mean-square error, a 3.45% improvement in the error rate, and a 11.98% improvement in the correlation coefficient with 4-D CNN compared with 3-D CNN. The 4-D CNN algorithm extracts the significant theta and alpha activations in the frontal and posterior cingulate cortices under distinct fatigue levels. This work contributes to enhancing our understanding of deep learning methods in the analysis of EEG signals. We even envision that deep learning might serve as a bridge between translation neuroscience and further real-world applications.

EEG signals can be processed and classified into commands for brain-computer interface (BCI). Stable deciphering of EEG is one of the leading challenges in BCI design owing to low signal to noise ratio and non-stationarities. Presence of non-stationarities in the EEG signals significantly perturb the feature distribution thus deteriorating the performance of Brain Computer Interface. Stationary Subspace methods discover subspaces in which data distribution remains steady over time. In this paper, we develop novel spatial filtering based feature extraction methods for dealing with nonstationarity in EEG signals from a drowsiness detection problem (a machine learning regression problem). The proposed method: DivOVR-FuzzyCSP-WS based features clearly outperformed fuzzy CSP based baseline features in terms of both RMSE and CC performance metrics. It is hoped that the proposed feature extraction method based on DivOVR-FuzzyCSP-WS will bring in a lot of interest in researchers working in developing algorithms for signal processing, in general, for BCI regression problems.

Cancer treatment is extremely aggressive and, in addition to causing considerable discomfort, can lead to death. Therefore, identifying aspects related to treatment assertiveness may be efficient for reducing the mortality rate of cancer patients. This paper seeks to identify the prognosis of cancer treatment survival through hybrid techniques based on the autonomous fuzzification process and artificial neural networks. The public dataset on cancer mortality is the source for conducting treatment assertiveness rating tests. The hybrid model had its results compared to other models present in the pattern classification literature with superior accuracy and identification of people likely to survive treatment (90.46%), and the fuzzy rules obtained with the execution of the model corroborate the high assertiveness of the model, even surpassing state of the art for the theme.

Dimensionality reduction is an important operation in information visualization, feature extraction, clustering, regression, and classification, especially for processing noisy high dimensional data. However, most existing approaches preserve either the global or the local structure of the data, but not both. Approaches that preserve only the global data structure, such as principal component analysis (PCA), are usually sensitive to outliers. Approaches that preserve only the local data structure, such as locality preserving projections, are usually unsupervised (and hence cannot use label information) and uses a fixed similarity graph. We propose a novel linear dimensionality reduction approach, supervised discriminative sparse PCA with adaptive neighbors (SDSPCAAN), to integrate neighborhood-free supervised discriminative sparse PCA and projected clustering with adaptive neighbors. As a result, both global and local data structures, as well as the label information, are used for better dimensionality reduction. Classification experiments on nine high-dimensional datasets validated the effectiveness and robustness of our proposed SDSPCAAN.

Fatigue is one problem with driving as it can lead to difficulties with sustaining attention, behavioral lapses, and a tendency to ignore vital information or operations. In this research, we explore multimodal physiological phenomena in response to driving fatigue through simultaneous functional near-infrared spectroscopy (fNIRS) and electroencephalography (EEG) recordings with the aim of investigating the relationships between hemodynamic and electrical features and driving performance. Sixteen subjects participated in an event-related lane-deviation driving task while measuring their brain dynamics through fNIRS and EEGs. Three performance groups, classified as Optimal, Suboptimal, and Poor, were defined for comparison. From our analysis, we find that tonic variations occur before a deviation, and phasic variations occur afterward. The tonic results show an increased concentration of oxygenated hemoglobin (HbO2) and power changes in the EEG theta, alpha, and beta bands. Both dynamics are significantly correlated with deteriorated driving performance. The phasic EEG results demonstrate event-related desynchronization associated with the onset of steering vehicle in all power bands. The concentration of phasic HbO2 decreased as performance worsened. Further, the negative correlations between tonic EEG delta and alpha power and HbO2 oscillations suggest that activations in HbO2 are related to mental fatigue. In summary, combined hemodynamic and electrodynamic activities can provide complete knowledge of the brain’s responses as evidence of state changes during fatigue driving.

Physical Human-Robot Collaboration (pHRC) is about the interaction between one or more human operator(s) and one or more robot(s) in direct contact and voluntarily exchanging forces to accomplish a common task. In any pHRC, the intuitiveness of the interaction has always been a priority, so that the operator can comfortably and safely interact with the robot. So far, the intuitiveness has always been described in a qualitative way. In this paper, we suggest an objective way to evaluate intuitiveness, known as prediction error negativity (PEN) using electroencephalogram (EEG). PEN is defined as a negative deflection in event related potential (ERP) due to cognitive conflict, as a consequence of a mismatch between perception and reality. Experimental results showed that the forces exchanged between robot and human during pHRC modulate the amplitude of PEN, representing different levels of cognitive conflict. We also found that PEN amplitude significantly decreases (p <; 0.05) when a mechanical resistance is being applied smoothly and more time in advance before an invisible obstacle, when compared to a scenario in which the resistance is applied abruptly before the obstacle. These results indicate that an earlier and smoother resistance reduces the conflict level. Consequently, this suggests that smoother changes in resistance make the interaction more intuitive.

Electroencephalogram (EEG) signals are one of the most widely used noninvasive signals in brain-computer interfaces. Large dimensional EEG recordings suffer from poor signal-tonoise ratio. These signals are very much prone to artifacts and noise, so sufficient preprocessing is done on raw EEG signals before using them for classification or regression. Properly selected spatial filters enhance the signal quality and subsequently improve the rate and accuracy of classifiers, but their applicability to solve regression problems is quite an unexplored objective. This paper extends common spatial patterns (CSP) to EEG state space using fuzzy time delay and thereby proposes a novel approach for spatial filtering. The approach also employs a novel fuzzy information theoretic framework for filter selection. Experimental performance on EEG-based reaction time (RT) prediction from a lane-keeping task data from 12 subjects demonstrated that the proposed spatial filters can significantly increase the EEG signal quality. A comparison based on root-mean-squared error (RMSE), mean absolute percentage error (MAPE), and correlation to true responses is made for all the subjects. In comparison to the baseline fuzzy CSP regression one versus rest, the proposed Fuzzy Time-delay Common Spatio-Spectral filters reduced the RMSE on an average by 9.94%, increased the correlation to true RT on an average by 7.38%, and reduced the MAPE by 7.09%.

This paper proposes applying recurrent residual network (RRN) for analyzing electroencephalogram (EEG) data captured during a simulated sustained attention driving task. We first address the suitableness of utilizing residual structure as well as adopting recurrent structure for EEG signal processing. Then based on these descriptions a recurrent residual network is tailored and depicted in detail. Thirdly we use an EEG dataset obtained from a sustained-attention experiment for our model justification. By applying the RRN model to the experimental data and via the competitive result achieved, we demonstrate the elegance of the proposed model. At last, we discuss the characteristics of the learned filters and their interpretations from EEG frequency band perspectives.

The class imbalance problem in machine learning occurs when certain classes are underrepresented relative to the others, leading to a learning bias toward the majority classes. To cope with the skewed class distribution, many learning methods featuring minority oversampling have been proposed, which are proved to be effective. To reduce information loss during feature space projection, this study proposes a novel oversampling algorithm, named minority oversampling in kernel adaptive subspaces (MOKAS), which exploits the invariant feature extraction capability of a kernel version of the adaptive subspace self-organizing maps. The synthetic instances are generated from well-trained subspaces and then their pre-images are reconstructed in the input space. Additionally, these instances characterize nonlinear structures present in the minority class data distribution and help the learning algorithms to counterbalance the skewed class distribution in a desirable manner. Experimental results on both real and synthetic data show that the proposed MOKAS is capable of modeling complex data distribution and outperforms a set of state-of-the-art oversampling algorithms.

© 2017 IEEE. Brain-Computer interface (BCI) which aims at enabling users to perform tasks through their brain waves has been a feasible and worth developing solution for growing demand of healthcare. Current proposed BCI systems are often with lower applicability and do not provide much help for reducing burdens of users because of the time-consuming preparation required by adopted wet sensors and the shortage of provided interactive functions. Here, by integrating a state visually evoked potential (SSVEP)-based BCI system and a robotic eating assistive system, we propose a non-invasive wireless steady state visually evoked potential (SSVEP)-based BCI eating assistive system that enables users with physical disabilities to have meals independently. The analysis compared different methods of classification and indicated the best method. The applicability of the integrated eating assistive system was tested by an Amyotrophic Lateral Sclerosis (ALS) patient, and a questionnaire reply and some suggestion are provided. Fifteen healthy subjects engaged the experiment, and an average accuracy of 91.35%, and information transfer rate (ITR) of 20.69 bit per min are achieved. For online performance evaluation, the ALS patient gave basic affirmation and provided suggestions for further improvement. In summary, we proposed a usable SSVEP-based BCI system enabling users to have meals independently. With additional adjustment of movement design of the robotic arm and classification algorithm, the system may offer users with physical disabilities a new way to take care of themselves.

Brain-state drifts could significantly impact on the performance of machine-learning algorithms in brain computer interface (BCI). However, less is understood with regard to how brain transition states influence a model and how it can be represented for a system. Herein we are interested in the hidden information of brain state-drift occurring in both simulated and real-world human-system interaction. This research introduced the Riemann metric to categorize EEG data, and visualized the clustering result so that the distribution of the data can be observable. Moreover, to defeat subjective uncertainty of electroencephalography (EEG) signals, fuzzy theory was employed. In this study, we built a fuzzy rule-based brain-state-drift detector to observe the brain state and imported data from different subjects to testify the performance. The result of the detection is acceptable and shown in this paper. In the future, we expect that brain-state drifting can be connected with human behaviors via the proposed fuzzy rule-based classification. We also will develop a new structure for a fuzzy rule-based brain-state-drift detector to improve the detection accuracy.

Drowsiness driving is one major factor of traffic accident. Monitoring the changes of brain signals provides an effective and direct way for drowsiness detection. One 3D convolutional neural network (3D CNN)-based forecasting system has been proposed to monitor electroencephalography (EEG) signals and predict fatigue level during driving. The limited weight sharing and channel-wise convolution were both applied to extract the significant phenomenon in various frequency bands of brain signals and the spatial information of EEG channel location, respectively. The proposed 3D CNN with limited weight sharing and channel-wise convolution has been demonstrated to predict reaction time (RT) of driving with low root mean square error (RMSE) through the brain dynamics. This proposed approach outperforms with the state-of-the-art algorithms, such as traditional CNN, Neural Network (NN), and support vector regression (SVR). Compared with traditional CNN and Artificial Neural Network, the RMSE of 3D CNN-based RT prediction has been improved 9.5% (RMSE from 0.6322 to 0.5720) and 8% (RMSE from 0.6217 to 0.5720), respectively. We envision that this study might open a new branch between deep learning application in neuro-cognitive analysis and real world application.

Research has indicated that fatigue is a critical factor in cognitive lapses because it negatively affects an individual's internal state, which is then manifested physiologically. This study explores neurophysiological changes, measured by electroencephalogram (EEG), due to fatigue. This study further demonstrates the feasibility of an online closed-loop EEG-based fatigue detection and mitigation system that detects physiological change and can thereby prevent fatigue-related cognitive lapses. More importantly, this work compares the efficacy of fatigue detection and mitigation between the EEG-based and a nonEEG-based random method. Twelve healthy subjects participated in a sustained-attention driving experiment. Each participant's EEG signal was monitored continuously and a warning was delivered in real-time to participants once the EEG signature of fatigue was detected. Study results indicate suppression of the alpha- and theta-power of an occipital component and improved behavioral performance following a warning signal; these findings are in line with those in previous studies. However, study results also showed reduced warning efficacy (i.e. increased response times (RTs) to lane deviations) accompanied by increased alpha-power due to the fluctuation of warnings over time. Furthermore, a comparison of EEG-based and nonEEG-based random approaches clearly demonstrated the necessity of adaptive fatigue-mitigation systems, based on a subject's cognitive level, to deliver warnings. Analytical results clearly demonstrate and validate the efficacy of this online closed-loop EEG-based fatigue detection and mitigation mechanism to identify cognitive lapses that may lead to catastrophic incidents in countless operational environments.

Brain-computer interface (BCI) has shown explosive growth for multiple applications in the recently years. Removing artifacts and selecting useful brain sources are essential in BCI research. Independent Component Analysis (ICA) has been proven as an effective technique to remove artifacts and many brain related researches are based on ICA. However, the useful independent components with brain sources are usually selected manually according to the scalp-plots. This is great inconvenience and a barrier for real-time BCI applications of EEG. In this investigation, a two-layer automatic identification model is proposed to select useful brain sources. It is based on neural network including support vector machine with radial basis function (SVMRBF) and self-organizing map (SOM). In the first layer, SVM discriminates useful independent components from the artifact effectively. In the second layer, these selected useful components are automatically classified to different spatial brain sources according to SOM. This study suggests this model to one general application for EEG study. It can reduce the effect of subjective judgment and improve the performance of EEG analysis.

Distraction during driving has been recognized as a significant cause of traffic accidents. The aim of this study is to investigate Electroencephalography (EEG) -based brain dynamics in response to driving distraction. To study human cognition under specific driving tasks in a simulated driving experiment, this study utilized two simulated events including unexpected car deviations and mathematics questions. The raw data were first separated into independent brain sources by Independent Component Analysis. Then, the EEG power spectra were used to evaluate the time-frequency brain dynamics. Results showed that increases of theta band and beta band power were observed in the frontal cortex. Further analysis demonstrated that reaction time and multiple cortical EEG power had high correlation. Thus, this study suggested that the features extracted by EEG signal processing, which were the theta power increases in frontal area, could be used as the distracted indexes for early detection of driver inattention in real driving.

Drivers' distraction is widely recognized as a leading cause of car accidents. To investigate the distracting effect of dual-tasks involving driving and answering mathematical equations in the stimulus onset asynchrony (SOA) conditions, we design five different cases: two cases involving single-tasks and three cases involving dual-tasks. We have found that there is no statistically significant change in the behavioral data among the three dual-tasks. This raises an important question - is there any detectable effect of the dual tasks on the brain waves? To answer this, we use the Self-Organizing Map (SOM) to recognize the changes, if any, in the Electroencephalography (EEG) dynamics associated with such dual-tasks. Our SOM analysis based on independent components corresponding to EEG signals extracted from Frontal and Motor areas revealed that single- and dual-tasks have distinguishable signatures in the EEG signals. Specifically, each of the two single-task conditions is clustered in a distinct spatial area of the map. Two of the dual-tasks also exhibit distinct spatial clusters, while the third case although shows differences from the other two, the neurons corresponding to this case are sub-clustered reflecting the fact that different subjects may give different priorities to the tasks when confronted with two tasks simultaneously. SOM-based exploratory analysis reveals the existence of distinct EEG signatures among the distracting and non-distracting tasks, although there is no any noticeable difference in the behavioral data among these cases.