Brain-Computer Interfaces (BCIs) are one of the few remaining communication options for individuals in a Completely Locked-In State (CLIS), where all voluntary motor functions are lost. However, decoding electroencephalographic (EEG) signals in CLIS is particularly challenging due to low signal-to-noise ratios, high intra- and inter-session variability, and cognitive fluctuations. In this study, we systematically evaluate classical and deep learning-based (DL) classification methods on a longitudinal P300-based BCI dataset acquired from a CLIS patient over ten months, comprising seven different stimulation paradigms.A systematic approach is followed to assess model generalization across BCI sessions and paradigms. Overall, more than 40 approaches are compared, including spatial filters for feature extraction with standard classifiers, as well as DL methods based on CNNs and Attention-based architectures. All methods are evaluated with raw input data and three different normalization strategies. Additionally, SMOTE data augmentation is applied to upsample the minority class. The results show high generalization performance across sessions and paradigms, with some approaches achieving nearly 100% performance. Normalization strategies significantly influence performance, while SMOTE often leads to performance degradation. These findings offer valuable insights for designing more robust BCI systems tailored to CLIS users, showing that collecting data across sessions and multiple BCI paradigms can improve BCI performance, while reducing or eliminating the need for per session calibration. Despite the very promising results, they are based on offline analysis. Thus, the best-performing approaches now require online validation for deployment in real-world CLIS scenarios.

Error-related potentials (ErrPs) extracted from electroencephalographic signals hold potential for application in Brain-Machine Interfaces, in contexts such as robot teleoperation or shared control in assistive platforms. Due to difficulties in signal classification, in part caused by its non-stationary and noisy nature, their use has not been fully realized yet.This work proposes a new approach to ErrP classification based on a two-stream deep learning architecture with three training stages. Its first stage is a self-supervised autoencoder architecture with a multi-head attention layer providing relevant latent features. The second stage comprises a supervised contrastive learning approach considering two backbone networks, where one inherits weights from the first stage and the other is updated by considering the feature embeddings distribution. The final stage comprises supervised classification, where the two backbones are fused and used to classify the input EEG signal. At the end of the three stages, a data-driven two-stream ErrP model is obtained.Twenty-five variants of the proposed approach using the Deep Convolutional Network, Shallow Convolutional Network and EEGNet backbones were tested in an ablation study and benchmarked against a large number of classical classification methods, using data from the BNCI dataset intended to assess cross subject generalization capabilities. The proposed approach obtained the best results overall, highlighting the approach’s capabilities in capturing relevant representations of the EEG signal.

Visual user interface paradigms are one of the key modules for brain-actuated wheelchairs, which are a class of promising assistive devices that can increase the autonomy and mobility of people suffering from severe motor impairments. This work proposes a Dynamic Environment-based Visual Interface System (DEVIS) for intuitive navigation target selection for brain-actuated wheelchairs. It is composed of a novel Dynamic Visual Interface (DVI), an RGB image-based perception module (indoor scene classification, object detection and classification, and object tracking), and a P300-based BCI. The DVI displays potential navigation goals in three forms of visual cues: an RGB camera image streaming with object bounding boxes overlaid on objects detected and tracked, three global points of interest (indoor places), and two static commands. Hence, the DVI allows a user to select, through the P300-based BCI, navigation targets/commands that are flashing independently and randomly to create an oddball paradigm. Experimental evaluations were carried out in a dynamic setting with 5 participants, who were asked to select targets displayed in the DVI. The dynamic setting was obtained using RGB image sequences of the ISR-RGB-D Dataset, which represents a mission performed by a mobile robotic platform. The target selection was performed through a P300-based BCI, using non-self-paced and self-paced modalities. The obtained results show that the proposed DEVIS can run in real-time and that users are able to correctly select targets with high BCI accuracy rates.

The accurate detection of motor imagery (MI) from electroencephalography (EEG) is a fundamental, as well as challenging, task to provide reliable control of robotic devices to support people suffering from neuro-motor impairments, e.g., in brain-computer interface (BCI) applications. Recently, deep learning approaches have been able to extract subject-independent features from EEG, to cope with its poor SNR and high intra-subject and cross-subject variability. In this paper, we first present a review of the most recent studies using deep learning for MI classification, with particular attention to their cross-subject performance. Second, we propose DynamicNet, a Python-based tool for quick and flexible implementations of deep learning models based on convolutional neural networks. We showcase the potentiality of DynamicNet by implementing EEGNet, a well-established architecture for effective EEG classification. Finally, we compare its performance with the filter bank common spatial pattern (FBCSP) in a 4-class MI task (data from a public dataset). To infer cross-subject classification performance, we applied three different cross-validation schemes. From our results, we show that EEGNet implemented with DynamicNet outperforms FBCSP by about 25 %, with a statistically significant difference when cross-subject validation schemes are applied. We conclude that deep learning approaches might be particularly helpful to provide higher cross-subject classification performance in multiclass MI classification scenarios. In the future, it is expected to improve DynamicNet to implement new architectures to further investigate cross-subject classification of MI tasks in real-world scenarios.

Brain-controlled wheelchairs (BCWs) are a promising solution for people with severe motor disabilities, who cannot use conventional interfaces. However, the low reliability of electroencephalographic signal decoding and the high user's workload imposed by continuous control of a wheelchair requires effective approaches. In this article, we propose a self-paced P300-based brain-computer interface (BCI) combined with dynamic time-window commands and a collaborative controller. The self-paced approach allows users to switch between control and noncontrol states without requiring any additional task or mental strategy, while the dynamic time-window commands allow balancing the reliability and speed of the BCI. The collaborative controller, combining user's intentions and navigation information, offers the possibility to navigate in complex environments and to improve the overall system reliability. The feasibility of the proposed approach and the impact of each system component (self-paced, dynamic time window, and collaborative controller) are systematically validated in a set of experiments conducted with seven able-bodied participants and six physically disabled participants steering a robotic wheelchair in real-office-like environments. These two groups controlled the BCW with a final driving accuracy greater than 99%. Quantitative and subjective results, assessed through questionnaires, attest to the effectiveness of the proposed approach. Altogether, these findings contribute to improving the usability of BCWs and, hence, the potential for their use by target users in home settings.

This paper presents a new collaborative approach for robot motion planning of an assistive robotic platform that takes into account the intentions of the user provided through Electrooculographic (EOG) signals, as well as obstacles surrounding the robotic platform. In order to increase human confidence in the operation of robotic platforms with some degree of navigational autonomy, the intent of the user must be included in the decision process. In our system, the humanrobot interface works through ocular movements (saccades and blinks), which are acquired as EOG signals and classified using a Convolutional Neural Network. In our proposed approach, a model-free Reinforcement Learning (RL) layer is used to provide commands to a virtual robotic platform. The RL layer is constantly being updated with the inputs from the user's intent, environment perception and previous machine-based decisions. In order to prevent collisions, machine-based perception using the proposed RL motion planning approach will assist the user by selecting suitable actions while learning from prior driving behaviors. The approach was validated by a set of tests that consisted of driving a robotic platform in an in-house 3D virtual model of our Research Center (ISR-UC). The experimental results show a better performance of the proposed approach with RL when compared to the version without the RL-based motion planning component. Results show that the approach is a promising step in the concept put forward for collaborative Human-Robotic Interaction (HRI), and opens a path for future research.

Over the past few years, virtual and mixed reality systems have evolved significantly yielding high immersive experiences. Most of the metaphors used for interaction with the virtual environment do not provide the same meaningful feedback, to which the users are used to in the real world. This paper proposes a cyber-glove to improve the immersive sensation and the degree of embodiment in virtual and mixed reality interaction tasks. In particular, we are proposing a cyber-glove system that tracks wrist movements, hand orientation and finger movements. It provides a decoupled position of the wrist and hand, which can contribute to a better embodiment in interaction and manipulation tasks. Additionally, the detection of the curvature of the fingers aims to improve the proprioceptive perception of the grasping/releasing gestures more consistent to visual feedback. The cyber-glove system is being developed for VR applications related to real estate promotion, where users have to go through divisions of the house and interact with objects and furniture. This work aims to assess if glove-based systems can contribute to a higher sense of immersion, embodiment and usability when compared to standard VR hand controller devices (typically button-based). Twenty-two participants tested the cyber-glove system against the HTC Vive controller in a 3D manipulation task, specifically the opening of a virtual door. Metric results showed that 83% of the users performed faster door pushes, and described shorter paths with their hands wearing the cyber-glove. Subjective results showed that all participants rated the cyber-glove based interactions as equally or more natural, and 90% of users experienced an equal or a significant increase in the sense of embodiment.

This paper proposes a P300-based BCI speller called LSC-4Q that takes advantage of steady state visual evoked potentials (SSVEP) that appear naturally in the brain as a side effect of the inter-stimulus interval of P300 visual paradigms. The LSC-4Q speller has a circular layout divided into quadrants, and symbols flash individually with a given pseudo-random strategy. Controlling the sequence of the events such that consecutive flashes alternate between sides or quadrants of the speller, we research the possibility of detecting the SSVEP phase associated with the side or quadrant for which the user is focusing on the target symbol, without explicitly incorporating a SSVEP flickering stimulator. The SSVEP phase is extracted using a statistical spatio-spectral Fisher criterion beamformer (SSFCB) implemented in the frequency domain. Results show that SSFCB efficiently extracts phase tags from the SSVEPs embedded on the visual evoked potentials of the oddball paradigm. Preliminary results with 4 participants suggest that it is possible to detect with high accuracy the side of the screen to which the user is looking, and with less precision the detection of the quadrant. Main issues are related to phase variability across sessions. Online results show that some participants can benefit from the combined P300-Lateral approach, improving the overall classification when P300 misclassifications occur.

Remote monitoring of health parameters is a promising approach to improve the health condition and quality of life of particular groups of the population, which can also alleviate the current expenditure and demands of healthcare systems. The elderly, usually affected by chronic comorbidities, are a specific group of the population that can strongly benefit from telehealth technologies, allowing them to reach a more independent life, by living longer in their own homes. Usability of telehealth technologies and their acceptance by end-users are essential requirements for the success of telehealth implementation. Older people are resistant to new technologies or have difficulty in using them due to vision, hearing, sensory and cognition impairments. In this paper, we describe the implementation of an IoT-based telehealth solution designed specifically to address the elderly needs. The end-user interacts with a TV-set to record biometric parameters, and to receive warning and recommendations related to health and environmental sensor recordings. The familiarization of older people with the TV is expected to provide a more user-friendly interaction ensuring the effectiveness integration of the end-user in the overall telehealth solution.

Automatic recognition of error-related potentials (ErrPs) requires a long calibration time in order to have enough error-samples to train the classifier. In this paper we analyze whether it is possible to reduce the ErrP-calibration time in a P300-based brain-computer interface (BCI), by calibrating the BCI with a high rate of errors (wrong detections of user intent). We analyze if a high error-rate condition still produces a discriminable ErrP and if its classification model generalizes well in sessions of different error-rates. Results show that the classification model built from a high error-rate calibration can be used successfully on sessions with lower error-rates.

Experiments with a semi-autonomous wheelchair controlled by means of a Brain-Computer Interface (BCI) are presented. The navigation system, having at its core a collaborative controller, performs smooth and safe manoeuvres following sparse steering commands provided by the user. The user intents are decoded from electroencephalographic signals evoked by a visual P300-based paradigm. Experiments have been performed by several able-bodied users and motor disabled participants, showing the effectiveness of the approach.

This paper presents a new shared-control approach for assistive mobile robots, using Brain Computer Interface (BCI) as the Human-Machine Interface (HMI). A P300-based paradigm that allows the selection of brain-actuated commands to steer a Robotic Wheelchair (RW), is proposed. At least one specific motor skill, such as the control of arms, legs, head or voice, is required to operate a conventional HMI. Due to this reason, they are not suited for people suffering from severe motor disorders. BCI may open a new communication channel to these users, since it does not require any muscular activity. The number of decoded symbols per minute (SPM) in a BCI is still very low, which means that users can only provide sparse, and discrete commands. The RW must rely on the navigation system to validate user commands effectively. A two-layer shared-control approach is proposed. The first, a virtual-constraint layer, is responsible for enabling/disabling the user commands, based on certain context restrictions. The second layer is an user-intent matching responsible for determining the suitable steering command, that better fits the user command, taking the user competence on steering the wheelchair into account. Experimental results using Robchair, the RW platform developed at ISR-UC [1], [2] are presented, showing the effectiveness of the proposed methodologies.

Abstract-Brain Computer Interface (BCI) systems based on electroencephalography (EEG) open a new communication channel for people with severe motor disabilities, without recurring to the conventional motor output pathways. The very low signal-to-noise ratio and low spatial resolution still limits severely BCIs communication bandwidth. This paper presents the ongoing work toward the development of a BCI system for wheelchair steering. A full system based on a visual P300 oddball paradigm is proposed. The signal processing algorithms are computationally efficient and require a short phase training. Temporal features and EEG channels are selected through a Fisher criteria. For enhancement of signal-to-noise ratio and data dimensionality reduction, a spatial filter named Common Spatial Patterns is applied. This method is widely used for classification of motor imagery events, however it is not very often used for classification of event related potentials such as P300. In this paper we show that Common Spatial Patterns is an effective approach to improve P300 classification rates. In our approach, the input features for classification are the projections of the filtered data instead of the variance of the projections as typically used in motor imagery. Offline classification results, obtained with a Bayesian classifier, are presented showing the effectiveness of the overall methodology.