The detection of an individual's attentional state via a Brain-Computer Interface (BCI) holds significant promise, offering a multitude of possibilities, including enhancing the usability of applications and enabling timely alerts in hazardous situations. However, the variability of EEG data between individuals and the dynamic nature of recordings pose practical challenges for achieving reliable results with BCIs. Thus, conventional methods often require the collection of each person's training data prior to usage, which is then used to train an individual model for detection. Such training data collection, makes it difficult to achieve practical use. To overcome this challenge, we propose a self-improving online learning system that personalizes a person-independent model for detecting attentional state in real-time during runtime. This eliminates the need for collecting individual training data prior to usage and instead generates the necessary labels for adaptation using automatically detected error-related potentials. The system was developed based on pre-trained models of two classifiers and used to evaluate different strategies of adaptation and label generation. A statistically significant accuracy improvement of 0.088 was achieved across all available subjects, based on simulations with pre-recorded data. These results suggest that person-dependent models for attentional state detection could in the future be substituted by self-improving classifiers that do not require a dedicated training data collection.

Attentional shifts can occur voluntarily (endogenous control) or reflexively (exogenous control). Previous studies have shown that the neural mechanisms underlying these shifts produce different activity patterns in the brain. Changes in visual-spatial attention are usually accompanied by eye movements and a fixation on the new center of attention. In this study, we analyze the fixation-related potentials in electroencephalographic recordings of 10 participants during computer screen-based viewing tasks. During task performance, we presented salient visual distractors to evoke reflexive attention shifts. Surrounding each fixation, 0.7-second data windows were extracted and labeled as “endogenous” or “exogenous”. Averaged over all participants, the balanced classification accuracy using a person-dependent Linear Discriminant Analysis reached 59.84%. In a leave-one-participant-out approach, the average classification accuracy reached 58.48%. Differentiating attention shifts, based on fixation-related potentials, could be used to deepen the understanding of human viewing behavior or as a Brain-Computer Interface for attention-aware user interface adaptations.

Steady-state visually evoked potentials (SSVEP) are a widely used paradigm for the detection of attended objects. However, their aid in the recognition of other attentional states has not yet been studied in detail. In this study (n=21), we assessed the benefits of including SSVEP stimuli as probes in a screen-based task to classify internal and external attention based on 16-channel EEG data offline. Previous studies have shown that the distinction between these two attentional states based on brain activity is possible. We compared several SSVEP-stimulus settings with a baseline where no SSVEP stimulus was present. Different flickering frequencies and stimulus placements were evaluated for the possibilities of different experimental setups. We found that the influence of the stimulus on the classification accuracy is highly dependent on the settings. The Linear Discriminant Analysis (LDA) performance increased significantly when an SSVEP-evoking stimulus with a low flickering frequency was present in the center of fixation. As well as when a Canonical Correlation Analysis (CCA)-coefficient was added as the SSVEP-specific feature to a generic band-power feature set. A simple training-free, person-independent threshold approach for internal and external attention detection resulted in accuracies significantly higher than chance based on SSVEP-features that were calculated only on three occipital electrodes. These results show that such stimuli can aid the recognition of internal and external attention. Thus, they can be used in experiments or applications for a more robust detection rate. Specifically, they could improve SSVEP-based BCI paradigms by adding another level of attention-awareness.

Augmented reality (AR) glasses enable the embedding of visual content in a real-world surroundings. In this PhD project, I will implement user interfaces which adapt to the cognitive state of the user, for example by avoiding distractions or re-directing the user’s attention towards missed information. For this purpose, sensory data from the user is captured (Brain activity via EEG of fNIRS, eye tracking, physiological measurements) and modeled with machine learning techniques. The focus of the cognitive state estimation is centered around attention related aspects. The main task is to build models for an estimation of a person’s attentional state from the combination and classification of multimodal data streams and context information, as well as their evaluation. Furthermore, the goal is to develop prototypical user interfaces for AR glasses and to test their usability in different scenarios.

We investigate the integration of eye-tracking and a Brain-Computer Interface into an Augmented Reality system to control a smart home environment. Through a head-mounted display, we present context-dependent control elements which the user selects by directing attention towards them. We show that the combination of both modalities leads to the most robust detection of selections and an interface which is accepted by its users.