Motor Imagery-based Brain-Computer Interfaces (MI-BCI) allow users to control a computer for various applications using their brain activity alone, which is usually recorded by an electroencephalogram (EEG). Although BCI applications are numerous, their use outside laboratories is still scarce due to their poor accuracy. Some users cannot use BCIs, a phenomenon sometimes called "BCI illiteracy", which impacts around 10% to 30% of BCI users, who cannot produce discriminable EEG patterns. By performing neurophysiological analyses, and notably by identifying neurophysiological predictors of BCI performance, we may understand this phenomenon and its causes better. In turn, this may also help us to better understand and thus possibly improve, BCI user training. Therefore, this paper presents statistical models dedicated to the prediction of MI-BCI user performance, based on neurophysiological users' features extracted from a two minute EEG recording of a "relax with eyes open" condition. We consider data from 56 subjects that were recorded in a `relax with eyes open' condition before performing a MI-BCI experiment. We used machine learning regression algorithm with leave-one-subject-out cross-validation to build our model of prediction. We also computed different correlations between those features (neurophysiological predictors) and users' MI-BCI performances. Our results suggest such models could predict user performances significantly better than chance (p ≤ 0.01) but with a relatively high mean absolute error of 12.43%. We also found significant correlations between a few of our features and the performance, including the previously explored μ-band predictor, as well as a new one proposed here: the μ-peak location variability. These results are thus encouraging to better understand and predict BCI illiteracy. However, they also require further improvements in order to obtain more reliable predictions.

When persons interact with the environment and experience or witness an error (e.g. an unexpected event), a specific brain pattern, known as error-related potential (ErrP) can be observed in the electroencephalographic signals (EEG). Virtual Reality (VR) technology enables users to interact with computer-generated simulated environments and to provide multi-modal sensory feedback. Using VR systems can, however, be error-prone. In this paper, we investigate the presence of ErrPs when Virtual Reality users face 3 types of visualization errors: (Te) tracking errors when manipulating virtual objects, (Fe) feedback errors, and (Be) background anomalies. We conducted an experiment in which 15 participants were exposed to the 3 types of errors while performing a center-out pick and place task in virtual reality. The results showed that tracking errors generate error-related potentials, the other types of errors did not generate such discernible patterns. In addition, we show that it is possible to detect the ErrPs generated by tracking losses in single trial, with an accuracy of 85%. This constitutes a first step towards the automatic detection of error-related potentials in VR applications, paving the way to the design of adaptive and self-corrective VR/AR applications by exploiting information directly from the user’s brain.

Brain-Computer Interfaces (BCIs) enable users to interact with computers without any dedicated movement, bringing new hands-free interaction paradigms. In this paper we study the combination of BCI and Augmented Reality (AR). We first tested the feasibility of using BCI in AR settings based on Optical See-Through Head-Mounted Displays (OST-HMDs). Experimental results showed that a BCI and an OST-HMD equipment (EEG headset and Hololens in our case) are well compatible and that small movements of the head can be tolerated when using the BCI. Second, we introduced a design space for command display strategies based on BCI in AR, when exploiting a famous brain pattern called Steady-State Visually Evoked Potential (SSVEP). Our design space relies on five dimensions concerning the visual layout of the BCI menu; namely: orientation, frame-of-reference, anchorage, size and explicitness. We implemented various BCI-based display strategies and tested them within the context of mobile robot control in AR. Our findings were finally integrated within an operational prototype based on a real mobile robot that is controlled in AR using a BCI and a HoloLens headset. Taken together our results (4 user studies) and our methodology could pave the way to future interaction schemes in Augmented Reality exploiting 3D User Interfaces based on brain activity and BCIs.

While the Sense of Agency (SoA) has so far been predominantly characterised in VR as a component of the Sense of Embodiment, other communities (e.g., in psychology or neurosciences) have investigated the SoA from a different perspective proposing complementary theories. Yet, despite the acknowledged potential benefits of catching up with these theories a gap remains. This paper first aims to contribute to fill this gap by introducing a theory according to which the SoA can be divided into two components, the feeling and the judgment of agency, and relies on three principles, namely the principles of priority, exclusivity and consistency. We argue that this theory could provide insights on the factors influencing the SoA in VR systems. Second, we propose novel approaches to manipulate the SoA in controlled VR experiments (based on these three principles) as well as to measure the SoA, and more specifically its two components based on neurophysiological markers, using ElectroEncephaloGraphy (EEG). We claim that these approaches would enable us to deepen our understanding of the SoA in VR contexts. Finally, we validate these approaches in an experiment. Our results (N=24) suggest that our approach was successful in manipulating the SoA as the modulation of each of the three principles induced significant decreases of the SoA (measured using questionnaires). In addition, we recorded participants' EEG signals during the VR experiment, and neurophysiological markers of the SoA, potentially reflecting the feeling and judgment of agency specifically, were revealed. Our results also suggest that users' profile, more precisely their Locus of Control (LoC), influences their level of immersion and SoA.

Although Mental Imagery based Brain-Computer Interfaces (MI-BCIs) seem to be very promising for many applications, they are still rarely used outside laboratories. This is partly due to suboptimal training protocols, which provide little help to users learning how to control the system. Indeed, they do not take into account recommendations from instructional design. However, it has been shown that MI-BCI performances are significantly correlated to certain aspects of the users' cognitive profile, such as their Spatial Abilities (SA). Thus, it remains to be elucidated whether training the SA of BCI users would also improve their BCI control performance. Therefore, we proposed and validated an SA training that aimed at being included in an MI-BCI training protocol. Our pre-studies indeed confirmed that such a training does increase people's SA abilities. We then conducted a pilot study with 3 participants, one with a standard MI-BCI training protocol, one with the proposed SA training integrated into a standard MI-BCI training, and another control integrating another training, here verbal comprehension tasks, into a standard MI-BCI training. While such a small population cannot lead to any strong result, our first results show that SA training can indeed be integrated into MI-BCI training and is thus worth being further investigated for BCI user training.

Despite their potential for many applications, Brain -- Computer Interfaces (BCI) are still rarely used due to their low reliability and long training. These limitations are partly due to inappropriate training protocols, which includes the feedback provided to the user. While feedback should theoretically be explanatory, motivating and meaningful, current BCI feedback is usually boring, corrective only and difficult to understand. In this study, different features of the electroencephalogram signals were explored to be used as a richer, explanatory BCI feedback. First, based on offline mental imagery BCI data, muscular relaxation was notably found to be negatively correlated to BCI performance. Second, this study reports on an online BCI evaluation using muscular relaxation as additional feedback. While this additional feedback did not lead to significant change in BCI performance, this study showed that multiple feedbacks can be used without deteriorating performance and provided interesting insights for explanatory BCI feedback design.