Decoding human movement intentions from electroencephalography (EEG) signals is critical for brain-computer interface (BCI) applications in motor neurorehabilitation, active assistance, and functional augmentation. However, current BCI models face two challenges for motor-impaired patients: 1) prolonged EEG data collection from patients is difficult; 2) differences in brain functional structures and motor behaviors between healthy individuals and patients limit the generalizability of models trained on healthy individuals' EEG data. To address these challenges, this study proposes a transfer learning-based model, TL-ME, to bridge the gap between healthy individuals' and patients' EEG data and improve movement decoding accuracy for patients. TL-ME integrates an attention-based feature extractor, adversarial domain discriminator, multi-source selection, and movement classifier to transfer knowledge from healthy individuals' EEG data (source domain) to patients' EEG data (target domain). Temporal and spectral visualizations are used to inspect brain activation patterns for shared motor tasks between healthy individuals and patients. Experimental results show a 10.8% improvement in upper-limb movement decoding's accuracy using TL-ME, with each module contributing to performance gains. Visualization analyses also demonstrate similar brain activation patterns across domains, validating the transferability of healthy individuals' EEG data to patient-specific models. This work introduces a novel cross-population transfer learning approach that leverages healthy individuals' EEG data to enhance neural decoding for motor-impaired patients, bridging the gap between experimental studies and real-world applications in BCI-based neurorehabilitation.

With increasing emphasis on vehicular automation and traffic efficiency, the management and coordination of platoon-based systems have become important. This research introduces a unique control framework based on a behavioral stability strategy, designed to enhance the cohesion of vehicle platoons and improve their ability to resist disturbances. Our approach integrates a vehicle scheduling system with a real-time platoon control mechanism to enhance the behavioral stability, robustness, and safety of the platoon. Given the heterogeneous nature of vehicles, we propose an optimal platoon formation model. This model strategically determines the number of platoons, arranges the sequence of vehicles within each platoon, and selects optimal cruising speeds to maximize platoon cohesion. To further enhance system robustness, a centralized robust model predictive controller is deployed for each platoon, ensuring stability against stochastic perturbations in vehicle dynamics and guaranteeing platoon safety. Finally, we conduct a simulation study involving multiple platoons with 20 heterogeneous vehicles to validate the effectiveness of the multi-layer optimization model.

Brain-teleoperation robot control ensures that human beings interact with telepresence mobile systems through the brain neural signals. In this study, a hierarchical robust predictive control framework consisting of a two-loop control scheme is developed to simultaneously enhance the safety, navigation, and robustness performance of electroencephalography (EEG)-based robotic systems and minimize the loss of control by the end-user. The outer loop is a model-based predictive controller to guarantee the optimal velocity evolution under various constraints. The inner loop is the integral sliding mode controller constructed by a novel integral sliding manifold and enables the velocity tracking properties under uncertainty compensation. Human-in-the-loop driving experiments are performed under different disturbances, and the results show that the proposed system offers advantages of safety, enhanced navigation performance, and stronger robustness over those conventional direct control of EEG-based robots. Therefore, brain-robot teleoperation is improved in terms of robust motion control and velocity modulation, providing insights into similar brain-controlled dynamic systems.

Brain-controlled intelligent vehicles (BCIVs) refer to intelligent vehicles, where brain-computer interfaces (BCIs) are applied to help a person operate (or teleoperate) a vehicle by decoding human intention from brain signals. Existing studies on BCIVs are focused on the single-task operation scenario. Considering that the multitask operation is common in practice, in this article, we design a multitask-oriented BCIV system for the first time by integrating a novel neural decoding method of driver-secondary-task intention with an adaptive brain–machine collaborative controller. We build an experimental platform of the proposed multitask-oriented BCIV system and test the performance of both the primary and secondary tasks by human-and-hardware-in-the-loop experiments. Experimental results show that the proposed multitask-oriented BCIV system performs well. This work has essential values in moving the exploration of brain-controlled systems toward a new step of the multitask operation and opens a new avenue for cognitive neuroscience to be applied to intelligent systems and human–machine integration.

In this article, we develop a robust sliding-mode nonlinear predictive controller for brain-controlled robots with enhanced performance, safety, and robustness. First, the kinematics and dynamics of a mobile robot are built. After that, the proposed controller is developed by cascading a predictive controller and a smooth sliding-mode controller. The predictive controller integrates the human intention tracking with safety guarantee objectives into an optimization problem to minimize the invasion to human intention while maintaining robot safety. The smooth sliding-mode controller is designed to achieve robust desired velocity tracking. The results of human-in-the-loop simulation and robotic experiments both show the efficacy and robust performance of the proposed controller. This work provides an enabling design to enhance the future research and development of brain-controlled robots.

Current driving behavior studies have limits in obtaining driver’s state information, and recent studies involving driver state focus on driver distraction or inattention. But it is more common that drivers operate in an intermediate subconscious state, where the drivers are neither cognitively fully focused nor distracted. There was little research to address this topic. In this study, the driver cognition alertness state information, which indicates driver subconscious alertness, is detected from eye and iris activities by non-contact computer vision methods. And a novel analysis is conducted and reveals the strong correlation between driver performance and driver cognition alertness state from experiment results. In detail, the driver cognition alertness state is quantified by the proposed metric– iris movement index, which is calculated from iris-eye relative displacements. The developed computer vision method produces high precision results in detecting faces, eyes, and irises in the experiment based on multiple deep learning networks applied in cascade, and enables displacement tracking of eye and iris targets. A filtering method is proposed and removes artifacts due to eye blinks in displacement measurements. The driving performance is estimated by a proposed performance evaluation method in a series of traffic scenarios designed with normal and emergent conditions.

Brain-controlled vehicles (BCVs) have vital practical values for the disabled and healthy people. To improve the performance of existing BCVs and lower the workload generated by BCVs to drivers, in this paper, we propose a novel control framework of BCVs, which consists of a brain-computer interface (BCI) with a probabilistic output model, an adaptive fuzzy logic-based interface model, and a model predictive control (MPC) shared controller. The BCI with a probabilistic output model can output all commands in a probabilistic form rather than a specific single command once. The adaptive fuzzy logic-based interface can convert the probabilities into the vehicle’s input signals (including the vehicle acceleration and the increment of steering wheel angle) according to the vehicle state and road information. The MPC shared controller can ensure the control authority of brain-control drivers and reduce drivers’ workload on the premise of maintaining safety. We establish an experimental platform to validate the proposed method by using the intersection selection and obstacle avoidance scenarios with eight subjects. The experimental results show the effectiveness of the proposed method in improving driving performance and decreasing drivers’ workload. This work can contribute to the research and development of BCVs and provide some new insights into the study of intelligent vehicles and human-vehicle integration.

Using brain signals rather than limbs to drive a vehicle may not only help persons with disabilities to acquire driving ability, but also provide a new alternative interface for healthy people to control a vehicle. However, the longitudinal driving performance of brain-controlled vehicles (BCVs) at a relatively high speed is not good enough. In this paper, to improve the performance of the longitudinal brain-control driving, we propose a new predictive control method based on the models of human behaviors and vehicle dynamics. The proposed method is designed to maintain rear-end safety of BCVs and driver ride comfort while ensuring the maximum control authority of brain-control drivers. Driver-and-hardware-in-the-loop experiments are conducted with different subjects under three kinds of scenarios to validate the proposed method. The results show that the proposed method is effective in maintaining rear-end safety and driver ride comfort while preserving driver intention.

Using brain signals rather than limbs to drive a vehicle can help persons with disabilities to extend their movement range and, thus, to improve their self-independence. However, the driving performance of brain-controlled vehicles (BCVs) is poor. In this paper, to improve the performance of BCVs, we propose a new shared control method based on the model predictive control (MPC) strategy. Particularly, to maintain the maximum control authority of brain-control drivers while ensuring the safety of BCVs, the MPC controller is designed by introducing a penalty on the deviation from drivers output in the cost function and setting safety constraints. Driver-and-hardware-in-the-loop experiments are conducted under two road-keeping scenarios and one obstacle-avoidance scenario with different subjects to validate the proposed method. The results demonstrate the effectiveness of the proposed method in avoiding roadway departures and obstacles while maintaining the control authority of users.

In this paper, we propose a new approach of detecting emergency braking intention for brain-controlled vehicles by interpreting electroencephalography (EEG) signals of drivers. Regularization linear discriminant analysis with spatial-frequency features is applied to build the detection model. These spatial-frequency features are selected from the powers of frequency points across sixteen channels by using the sequential forward floating search. Experimental results from twelve subjects show that on average, the proposed method can detect emergency braking intentions 420 ms after the onset of emergency situations with the system accuracy of over 94%, showing the feasibility of developing a practical system of detecting driver emergency braking intention with the power spectra of EEG signals for brain-controlled vehicles.

The control performance and safety of current brain-controlled mobile robots are limited. To address this problem, in this paper, we design an assistive controller based on the model predictive control method. The proposed controller fuses tracking user intention and guaranteeing safety of brain-controlled mobile robots into an optimization problem. In this way, the proposed controller can make users control a brain-controlled mobile robot as much as possible given the mobile robot is safe. The experimental results show that the proposed controller can improve the control performance of the brain-controlled simulated mobile robot and guarantee its safety.

In this paper, we propose a novel interface between drivers and in-vehicle devices by using steady-state visual evoked potential (SSVEP) of brain signals. The SSVEP is detected by a canonical correlation analysis (CCA) classifier and applied to turn on and turn off in-vehicle devices. The proposed interface is built and tested online in a driving simulator by requiring drivers to use the interface to interact with the in-vehicle device while performing the primary driving tasks including lane keeping and obstacle avoidance. The pilot experimental results show the feasibility of the proposed interface.

This paper proposes a novel method to recognize an emergency situation by translating EEG signals of a disabled driver while he or she uses a brain-machine interface without using his or her limbs to drive a vehicle. EEG signals were first filtered by independent component analysis along with information entropy. And then the sums of powers of theta wave in the power spectrum of EEG signals from 13 channels were used as features of the classifier built by linear discriminant analysis. The pilot experimental results from two participants in a driving simulator indicated that the model recognized emergency situations (e.g., pedestrian sudden occurrence) 400 ms earlier than the response of drivers with a hit rate of 76.4%, suggesting that the proposed method is feasible. The proposed method can be used as a complementary method to the existing ones based on detecting external objects with sensors.

Driver intention detection is an important component in human-centric driver assistance systems. This article proposes a novel method for detecting driver normal and emergency left- or right-lane-changing intentions by using driver models based on the queuing network cognitive architecture. Driver lane-changing and lane-keeping models are developed and used to simulate driver behavior data associated with 5 kinds of intentions (i.e., normal and emergency left- or right-lane-changing and lane-keeping intentions). The differences between 5 sets of simulated behavior data and the collected actual behavior data are computed, and the intention associated with the smallest difference is determined as the detection outcome. The experimental results from 14 drivers in a driving simulator show that the method can detect normal and emergency lane-changing intentions within 0.325 s and 0.268 s of the steering maneuver onset, respectively, with high accuracy (98.27% for normal lane changes and 90.98% for emergency lane changes) and low false alarm rate (0.294%).

This paper describes the development of a novel driver lateral control model by integrating the driver's neuromuscular dynamics into the queuing network (QN)-based driver lateral control model. Experimental results from 16 participants in a driving simulator show that, compared to the QN-based model, the proposed model performs better, and its performance is closer to that of drivers when a vehicle runs at a relatively high speed. The proposed model not only has the advantages of the models based on a cognitive architecture but also captures the dynamic interaction between the vehicular steering system and the driver's neuromuscular system. Thus, it can better represent driver lateral control and has greater value in supporting the development of driver assistance systems.

In this paper, we propose a novel driver-vehicle interface for individuals with severe neuromuscular disabilities to use intelligent vehicles by using P300 and steady-state visual evoked potential (SSVEP) brain-computer interfaces (BCIs) to select a destination and test its performance in the laboratory and real driving conditions. The proposed interface consists of two components: the selection component based on a P300 BCI and the confirmation component based on an SSVEP BCI. Furthermore, the accuracy and selection time models of the interface are built to help analyze the performance of the entire system. Experimental results from 16 participants collected in the laboratory and real driving scenarios show that the average accuracy of the system in the real driving conditions is about 99% with an average selection time of about 26 s. More importantly, the proposed system improves the accuracy of destination selection compared with a single P300 BCI-based selection system, particularly for those participants with relatively low level of accuracy in using the P300 BCI. This study not only provides individuals with severe motor disabilities with an interface to use intelligent vehicles and thus improve their mobility, but also facilitates the research on driver-vehicle interface, multimodal interaction, and intelligent vehicles. Furthermore, it opens an avenue on how cognitive neuroscience may be applied to intelligent vehicles.

In this paper, we propose a novel driver-vehicle interface by using the event-related desynchronizations (ERD) and event-related synchronization (ERS) potentials induced by motor imagery in conjunction with alpha rhythm to interact with a vehicle. The alpha rhythm, which is recognized by using a linear discriminant analysis (LDA) classifier, is employed to control the starting and stopping of the vehicle. The ERD/ERS brain-computer interface (BCI) is applied to control the turning left and right of the vehicle. A simulated vehicle based on the proposed driver-vehicle interface is developed and tested online by applying a driving task, including vehicle starting and stopping, lane keeping and curve negotiation, and avoiding obstacle. The experimental results suggest that the proposed interface is feasible.

Computational dual-task models of driving with a secondary task can help compute, simulate, and predict driving behavior in dual task situations. These models can thus help improve the process of developing in-vehicle devices by reducing or eliminating the need for conducting driver experiments in the early stage of the development. Further, these models can help improve traffic flow simulation. This article develops a dual-task model of driving with a visual distraction task using the Queuing Network model of driver lateral control and a logistic regression model. The comparison between the model simulation data and the human data from drivers in a driving simulator shows that this computational model can perform driving with a secondary visual task well and its performance is consistent with the driver data.

In this paper, we propose a new steady-state visually evoked potential (SSVEP) brain-computer interface (BCI) with visual stimuli presented on a windshield via a head-up display, and we apply this BCI in conjunction with an alpha rhythm to control a simulated vehicle with a 14-DOF vehicle dynamics model. A linear discriminant analysis classifier is applied to detect the alpha rhythm, which is used to control the starting and stopping of the vehicle. The classification models of the SSVEP BCI with three commands (i.e., turning left, turning right, and going forward) are built by using a support vector machine with frequency domain features. A real-time brain-controlled simulated vehicle is developed and tested by using four participants to perform a driving task online, including vehicle starting and stopping, lane keeping, avoiding obstacles, and curve negotiation. Experimental results show the feasibility of using the human “mind” alone to control a vehicle, at least for some users.

Inferring driver intentions plays an important role in developing human-centric intelligent driver assistance systems. In this paper, we propose a method of inferring the lane-changing intention of drivers by using a driver model based on the queuing network (QN) cognitive architecture. Driver behavior data associated with a range of possible driver intentions are simulated by using the QN-based driver model previously validated. The intentions of drivers are deduced by comparing these sets of simulated behavior data with the collected behavior data of drivers. The experimental results in a driving simulator show that the method can infer typical and rapid lane-changing intention of drivers well.

EEG-based brain-controlled mobile robots can serve as powerful aids for severely disabled people in their daily life, especially to help them move voluntarily. In this paper, we provide a comprehensive review of the complete systems, key techniques, and evaluation issues of brain-controlled mobile robots along with some insights into related future research and development issues. We first review and classify various complete systems of brain-controlled mobile robots into two categories from the perspective of their operational modes. We then describe key techniques that are used in these brain-controlled mobile robots including the brain-computer interface techniques and shared control techniques. This description is followed by an analysis of the evaluation issues of brain-controlled mobile robots including participants, tasks and environments, and evaluation metrics. We conclude this paper with a discussion of the current challenges and future research directions.

In this paper, we propose a P300 brain-computer interface (BCI) with visual stimuli presented on a head-up display and we apply this BCI for selecting destinations of a simulated vehicle in a virtual scene. To improve the usability of the selection system, we analyze the effects of the number of electroencephalogram (EEG) rounds on system performance. Experimental results from eight participants show that the BCI-based model of destination selection can be built with EEG data from eight channels, and participants can use this BCI to select a desired destination with an accuracy value of 93.6% ± 1.6% (mean value with standard error) in about 12 s of selection time. This paper lays a foundation for developing vehicles that use a BCI to select a desired destination from a list of predefined destinations and then use an autonomous navigation system to reach the desired destination.

In this paper, we propose a computational model of driver lateral control based on the queuing network cognitive architecture and the driver preview model about driver lateral control activities. This computational model was applied to model the dual tasks of driving with a cognitive distraction task. The comparison between human driver data and model simulation data shows that this computational model can perform vehicle lateral control well, and its performance is consistent with that of drivers under single- and dual-task driving conditions. Furthermore, we examine the effectiveness of some parameters of the model in representing different styles of driving and discuss the value of this computational model in facilitating the evaluation of vehicle dynamics and driver assistant systems and providing new insights into research on unmanned vehicle control techniques.

This article reports two experiments and the corresponding modeling research on the effects of symmetry and the number of compositional elements on Chinese users' aesthetic ratings of interfaces composed with abstract black-and-white geometric images and realistic-looking web pages. Symmetry and the number of compositional elements are the two independent variables, each with three levels. The dependent variable is subjective ratings of aesthetic appeal. The results from both experiments show that symmetry of compositional elements significantly affects aesthetic ratings of Chinese users, whereas the number alone does not have a significant effect on aesthetic ratings. However, subjects preferred realistic web page images with few elements when they lack symmetry. We also describe our development and evaluation of a computational model of aesthetic ratings based on symmetry and the number of compositional elements to predict aesthetic ratings, which can be used to evaluate and support aesthetic design of interfaces.

Empirical data modeling can be used to model human performance and explore the relationships between diverse sets of variables. A major challenge of empirical data modeling is how to generalize or extrapolate the findings with a limited amount of observed data to a broader context. In this paper, we introduce an approach from machine learning, known as support vector regression (SVR), which can help address this challenge. To demonstrate the method and the value of modeling human performance with SVR, we apply SVR to a real-world human factors problem of night vision system design for passenger vehicles by modeling the probability of pedestrian detection as a function of image metrics. The results indicate that the SVR-based model of pedestrian detection shows good performance. Some suggestions on modeling human performance by using SVR are discussed.

The primary purpose of night-vision systems in civilian vehicles is to help drivers detect pedestrians. Pedestrian detection distance with night-vision systems has been modeled based on image metrics. However, the probability of pedestrian detection, in particular considering the factor of distance, has not been modeled based on image metrics. In this paper, we first describe a model of the probability of pedestrian detection, which compares several combinations of image-based clutter, contrast, and pedestrian size metrics using a simple mathematical equation. Next, we describe a model of the probability of pedestrian detection as a function of distance and image-based metrics by combining the model of pedestrian-detection probability and a model that represents the relationship between the distance to a pedestrian and an image-based pedestrian size metric. In the final model, image-based metrics are used to predict pedestrian-detection performance and can also be used to evaluate and support the development of night-vision systems in vehicles.