We propose auditory tagging, a novel method to enhance decoding performance in auditory brain-computer interface (BCI) paradigms. Drawing inspiration from steady-state visually evoked potentials (SSVEPs), auditory taggers involve embedding a steady frequency onto an auditory stimulus with the goal of eliciting a detectable neuronal response. In this work, we introduce three such approaches and evaluate them on the auditory intention decoding (AID) paradigm. In AID, subjects are primed with a question and potential target and non-target answer options are provided for this question. The BCI then decodes whether a given sample is a target or non-target. Despite the conceptual promise of the auditory taggers, our experiment results did not reveal statistically significant improvements in decoding accuracy using the proposed tagging approaches. We discuss potential explanations for this observation and highlight possible avenues of improvement for future research.

The Conversational Brain-Artificial Intelligence Interface (BAI) is a novel brain-computer interface (BCI) that uses artificial intelligence (AI) to help individuals with severe language impairments communicate. It translates users’ broad intentions into coherent, context-specific responses through an advanced AI conversational agent. A critical aspect of intention translation in BAI is the decoding of code-modulated visual evoked potentials (c-VEP) signals. This study evaluates five different artificial neural network (ANN) architectures for decoding c-VEP-based EEG signals in the BAI system, highlighting the efficacy of lightweight, shallow ANN models and pre-training strategies using data from other participants to enhance classification performance. These results provide valuable insights for the application of ANN models in decoding c-VEP-based EEG signals and may benefit other c-VEP-based BCI systems.

Explaining how neuronal activity gives rise to cognition arguably remains the most significant challenge in cognitive neuroscience. We introduce neuro-cognitive multilevel causal modeling (NC-MCM), a framework that bridges the explanatory gap between neuronal activity and cognition by construing cognitive states as (behaviorally and dynamically) causally consistent abstractions of neuronal states. Multilevel causal modeling allows us to interchangeably reason about the neuronal- and cognitive causes of behavior while maintaining a physicalist (in contrast to a strong dualist) position. We introduce an algorithm for learning cognitive-level causal models from neuronal activation patterns and demonstrate its ability to learn cognitive states of the nematode C. elegans from calcium imaging data. We show that the cognitive-level model of the NC-MCM framework provides a concise representation of the neuronal manifold of C. elegans and its relation to behavior as a graph, which, in contrast to other neuronal manifold learning algorithms, supports causal reasoning. We conclude the article by arguing that the ability of the NC-MCM framework to learn causally interpretable abstractions of neuronal dynamics and their relation to behavior in a purely data-driven fashion is essential for understanding biological systems whose complexity prohibits the development of hand-crafted computational models.

Algorithmic recourse recommendations inform stakeholders of how to act to revert unfavorable decisions. However, existing methods may recommend actions that lead to acceptance (i.e., revert the model's decision) but do not lead to improvement (i.e., may not revert the underlying real-world state). To recommend such actions is to recommend fooling the predictor. We introduce a novel method, Improvement-Focused Causal Recourse (ICR), which involves a conceptual shift: Firstly, we require ICR recommendations to guide toward improvement. Secondly, we do not tailor the recommendations to be accepted by a specific predictor. Instead, we leverage causal knowledge to design decision systems that predict accurately pre- and post-recourse, such that improvement guarantees translate into acceptance guarantees. Curiously, optimal pre-recourse classifiers are robust to ICR actions and thus suitable post-recourse. In semi-synthetic experiments, we demonstrate that given correct causal knowledge ICR, in contrast to existing approaches, guides toward both acceptance and improvement.

The Shapley Additive Global Importance (SAGE) value is a theoretically appealing interpretability method that fairly attributes global importance to a model’s surplus performance contributions over an exponential number of feature sets. This is computationally expensive, particularly because estimating the surplus contributions requires sampling from conditional distributions. Thus, SAGE approximation algorithms only take a fraction of the feature sets into account. We propose d-SAGE, a method that accelerates SAGE approximation. d-SAGE is motivated by the observation that conditional independencies (CIs) between a feature and the model target imply zero surplus contributions, such that their computation can be skipped. To identify CIs, we leverage causal structure learning (CSL) to infer a graph that encodes (conditional) independencies in the data as d-separations. This is computationally more efficient because the expense of the one-time graph inference and the d-separation queries is negligible compared to the expense of surplus contribution evaluations. Empirically we demonstrate that d-SAGE enables the efficient and accurate estimation of SAGE values.

Causal Feature Learning (CFL) infers macro-level causes (e.g., an aggregation of pixels in a traffic light image) from micro-level data (e.g., pixels of the image) by clustering the predicted probabilities of effect states (e.g., state of the traffic light). The current method for CFL uses a two-step procedure. First, a classifier for the effect states is trained, and afterwards, the predicted effect state probabilities are clustered. With CaFe DBSCAN, we present a novel density-based clustering method that conducts CFL directly by estimating conditional probabilities during clustering. To this end, we introduce the notion of clustering regions with similar conditional probabilities of the effect states given their micro-level data points. Our single-step approach has the following benefits: (1) CaFe DBSCAN introduces a comprehensive approach to Causal Feature Learning. Unlike existing methods, CaFe DBSCAN uses a probabilistic framework and does not require separate classification and clustering steps implemented by different algorithms relying on various assumptions, parameter settings, and optimization goals. (2) We do not need to train and tune a classifier first, hence the algorithm is more runtime-efficient than the current approach. (3) Due to the properties of density-based clustering algorithms, CaFe DBSCAN is robust against noise and outliers, which leads to purer clusters. (4) Our algorithm automatically infers a reasonable number of clusters, i.e., macro-level causes. We demonstrate the benefits of CaFe DBSCAN on synthetic and real-world data.

Riemannian frameworks are the basis for some of the best-performing decoding methods in EEG-based Brain-Computer Interfacing. In this work, we consider whether a nonlinear extension of the Riemannian framework, obtained by replacing the channel-wise covariance of the EEG signal with the nonlinear distance covariance, improves decoding performance. We study the theoretical properties of the distance covariance metric in this framework, in particular invariance to affine transformations, and compare the proposed method with established Riemannian methods on three different EEG data sets. We do not find evidence that the distance covariance extension improves decoding performance in comparison to the linear Riemannian framework.

Research in human–robot interaction for assistive robotics usually presents many technical challenges for experimenters, forcing researchers to split their time between solving technical problems and conducting experiments. In addition, previous work in virtual reality setups tends to focus on a single assistive robotics application. In order to alleviate these problems, we present ArmSym, a virtual reality laboratory with a fully simulated and developer-friendly robot arm. The system is intended as a testbed to run many sorts of experiments on human control of a robotic arm in a realistic environment, ranging from an upper limb prosthesis to a wheelchair-mounted robotic manipulator. To highlight the possibilities of this system, we perform a study comparing different sorts of prosthetic control types. Looking at nonimpaired subjects, we study different psychological metrics that evaluate the interaction of the user with the robot under different control conditions. Subjects report a perception of embodiment in the absence of realistic cutaneous touch, supporting previous studies in the topic. We also find interesting correlations between control and perceived ease of use. Overall our results confirm that ArmSym can be used to gather data from immersive experiences prosthetics, opening the door to closer collaboration between device engineers and experience designers in the future.

Interpretable Machine Learning (IML) methods are used to gain insight into the relevance of a feature of interest for the performance of a model. Commonly used IML methods differ in whether they consider features of interest in isolation, e.g., Permutation Feature Importance (PFI), or in relation to all remaining feature variables, e.g., Conditional Feature Importance (CFI). As such, the perturbation mechanisms inherent to PFI and CFI represent extreme reference points. We introduce Relative Feature Importance (RFI), a generalization of PFI and CFI that allows for a more nuanced feature importance computation beyond the PFI versus CFI dichotomy. With RFI, the importance of a feature relative to any other subset of features can be assessed, including variables that were not available at training time. We derive general interpretation rules for RFI based on a detailed theoretical analysis of the implications of relative feature relevance, and demonstrate the method's usefulness on simulated examples.

We consider the task of causal structure learning over measurement dependence inducing latent (MeDIL) causal models. We show that this task can be framed in terms of the graph theoretic problem of finding edge clique covers,resulting in an algorithm for returning minimal MeDIL causal models (minMCMs). This algorithm is non-parametric, requiring no assumptions about linearity or Gaussianity. Furthermore, despite rather weak assumptions aboutthe class of MeDIL causal models, we show that minimality in minMCMs implies some rather specific and interesting properties. By establishing MeDIL causal models as a semantics for edge clique covers, we also provide a starting point for future work further connecting causal structure learning to developments in graph theory and network science.

Neurophysiological laboratory studies are often constraint to immediate geographical surroundings and access to equipment may be temporally restricted. Limitations of ecological validity, scalability, and generalizability of findings pose a significant challenge for the development of brain-computer interfaces (BCIs), which ultimately need to function in any context, on consumer-grade hardware. We introduce MYND: An open-source framework that couples consumer-grade recording hardware with an easy-to-use application for the unsupervised evaluation of BCI control strategies. Subjects are guided through experiment selection, hardware fitting, recording, and data upload in order to self-administer multi-day studies that include neurophysiological recordings and questionnaires at home. As a use case, thirty subjects evaluated two BCI control strategies "Positive memories" and "Music imagery" by using a four-channel electroencephalogram (EEG) with MYND. Neural activity in both control strategies could be decoded with an average offline accuracy of 68.5% and 64.0% across all days.

Stroke-induced disturbances of large-scale cortical networks are known to be associated with the extent of motor deficits. We argue that identifying brain networks representative of motor behavior in the resting brain would provide significant insights for current neurorehabilitation approaches. Particularly, we aim to investigate the global configuration of brain rhythms and their relation to motor skill, instead of learning performance as broadly studied. We empirically approach this problem by conducting a three-dimensional physical space visuomotor learning experiment during electroencephalographic (EEG) data recordings with thirty-seven healthy participants. We demonstrate that across-subjects variations in average movement smoothness as the quantified measure of subjects' motor skills can be predicted from the global configuration of resting-state EEG alpha-rhythms (8-14 Hz) recorded prior to the experiment. Importantly, this neural signature of motor skill was found to be orthogonal to (independent of) task-as well as to learning-related changes in alpha-rhythms, which we interpret as an organizing principle of the brain. We argue that disturbances of such configurations in the brain may contribute to motor deficits in stroke, and that reconfiguring stroke patients' brain rhythms by neurofeedback may enhance post-stroke neurorehabilitation.

Various neuroimaging studies aim to understand the complex nature of human motor behavior. There exists a variety of experimental approaches to study neurophysiological correlates of performance during different motor tasks. As distinct from studies based on visuomotor learning, we investigate changes in electroencephalographic (EEG) activity during an actual physical motor adaptation learning experiment. Based on statistical analysis of EEG signals collected during a force-field adaptation task performed with the dominant hand, we observe a modulation of pre-movement upper alpha (10-12 Hz) and lower beta (13-16 Hz) powers over the contralateral region. This modulation is observed to be stronger in lower beta range and, through a regression analysis, is shown to be related with motor adaptation performance on a subject-specific level.

We propose to fuse two currently separate research lines on novel therapies for stroke rehabilitation: brain-computer interface (BCI) training and transcranial electrical stimulation (TES). Specifically, we show that BCI technology can be used to learn personalized decoding models that relate the global configuration of brain rhythms in individual subjects (as measured by EEG) to their motor performance during 3D reaching movements. We demonstrate that our models capture substantial across-subject heterogeneity, and argue that this heterogeneity is a likely cause of limited effect sizes observed in TES for enhancing motor performance. We conclude by discussing how our personalized models can be used to derive optimal TES parameters, e.g., stimulation site and frequency, for individual patients.

A brain-computer interface (BCI) is used to enable communication between humans and machines by decoding elicited brain activity patterns. However, these patterns have been found to vary across subjects or even for the same subject across sessions. Such problems render the performance of a BCI highly specific to subjects, requiring expensive and time-consuming individual calibration sessions to adapt BCI systems to new subjects. This work tackles the aforementioned problem in a Bayesian multi-task learning (MTL) framework to transfer common knowledge across subjects and sessions for the adaptation of a BCI to new subjects. In particular, a recent framework, that is able to exploit the structure of multi-channel electroencephalography (EEG), is extended by a Bayesian hierarchical logistic regression decoder for probabilistic binary classification. The derived model is able to explicitly learn spatial and spectral features, therefore making it further applicable for identification, analysis and evaluation of paradigm characteristics without relying on expert knowledge. An offline experiment with the new decoder shows a significant improvement in performance on calibration-free decoding compared to previous MTL approaches for rule adaptation and uninformed models while also outperforming them as soon as subject-specific data becomes available. We further demonstrate the ability of the model to identify relevant topographies along with signal band-power features that agree with neurophysiological properties of a common sensorimotor rhythm paradigm.

Brain-computer interfaces (BCIs) are often based on the control of sensor motor processes, yet sensor motor processes are impaired in patients suffering from amyotrophic lateral sclerosis (ALS). We devised a new paradigm that targets higher level cognitive processes to transmit information from the user to the BCI. We instructed five ALS patients and eleven healthy subjects to either activate self-referential memories or to focus on processes without mnemonic content, while recording a high density electroencephalogram (EEG). Both tasks are likely to modulate activity in the default mode network (DMN) without involving sensor motor pathways. We find that the two tasks can be distinguished from band power modulations in the theta (3 -- 7 Hz) and alpha-range (8 -- 13 Hz) in fronto-parietal areas, consistent with modulation of neural activity in primary nodes of the DMN. Training a support vector machine (SVM) to discriminate the two tasks on theta- and alpha-power in the precuneus, as estimated by a beam forming procedure, resulted in above chance-level decoding accuracy after only one experimental session. Therefore, the presented work could serve as a basis for a novel tool which allows for simple, reliable communication with patients in late stages of ALS.

Despite intensive efforts, no significant benefit of rehabilitation robotics in post-stroke motor-recovery has yet been demonstrated in large-scale clinical trials. The present work is based on the premise that future advances in rehabilitation robotics require an enhanced understanding of the neural processes involved in motor learning after stroke. We present a system that combines a Barret WAM™seven degree-of-freedom robot arm with neurophysiological recordings for the purpose of studying post-stroke motor learning. We used this system to conduct a pilot study on motor learning during reaching movements with two stroke patients. Preliminary results indicate that pre-trial brain activity in ipsilesional sensorimotor areas may be a neural correlate of the current state of motor learning. These results are discussed in terms of their relevance for future rehabilitation strategies that combine rehabilitation robotics with real-time analyses of neuro-physiological recordings.

Brain-Computer Interfaces (BCIs) in combination with robot-assisted physical therapy may become a valuable tool for neurorehabilitation of patients with severe hemiparetic syndromes due to cerebrovascular brain damage (stroke) and other neurological conditions. A key aspect of this approach is reestablishing the disrupted sensorimotor feedback loop, i.e., determining the intended movement using a BCI and helping a human with impaired motor function to move the arm using a robot. It has not been studied yet, however, how artificially closing the sensorimotor feedback loop affects the BCI decoding performance. In this article, we investigate this issue in six healthy subjects, and present evidence that haptic feedback facilitates the decoding of arm movement intention. The results provide evidence of the feasibility of future rehabilitative efforts combining robot-assisted physical therapy with BCIs. Moreover, the results suggest that shared-control strategies in Brain-Machine Interfaces (BMIs) may benefit from haptic feedback.

EEG connectivity measures could provide a new type of feature space for inferring a subject's intention in Brain-Computer Interfaces (BCIs). However, very little is known on EEG connectivity patterns for BCIs. In this study, EEG connectivity during motor imagery (MI) of the left and right is investigated in a broad frequency range across the whole scalp by combining Beamforming with Transfer Entropy and taking into account possible volume conduction effects. Observed connectivity patterns indicate that modulation intentionally induced by MI is strongest in the gamma-band, i.e., above 35 Hz. Furthermore, modulation between MI and rest is found to be more pronounced than between MI of different hands. This is in contrast to results on MI obtained with bandpower features, and might provide an explanation for the so far only moderate success of connectivity features in BCIs. It is concluded that future studies on connectivity based BCIs should focus on high frequency bands and consider experimental paradigms that maximally vary cognitive demands between conditions.

The performance of EEG-based Brain-Computer-Interfaces (BCIs) critically depends on the extraction of features from the EEG carrying information relevant for the classification of different mental states. For BCIs employing imaginary movements of different limbs, the method of Common Spatial Patterns (CSP) has been shown to achieve excellent classification results. The CSP-algorithm however suffers from a lack of robustness, requiring training data without artifacts for good performance. To overcome this lack of robustness, we propose an adaptive spatial filter that replaces the training data in the CSP approach by a-priori information. More specifically, we design an adaptive spatial filter that maximizes the ratio of the variance of the electric field originating in a predefined region of interest (ROI) and the overall variance of the measured EEG. Since it is known that the component of the EEG used for discriminating imaginary movements originates in the motor cortex, we design two adaptive spatial filters with the ROIs centered in the hand areas of the left and right motor cortex. We then use these to classify EEG data recorded during imaginary movements of the right and left hand of three subjects, and show that the adaptive spatial filters outperform the CSP-algorithm, enabling classification rates of up to 94.7 % without artifact rejection.

Given a linear and instantaneous mixture model, we prove that for blind source separation (BSS) algorithms based on mutual information, only sources with non-Gaussian distribution are consistently reconstructed independent of initial conditions. This allows the identification of non-Gaussian sources and consequently the identification of signal and noise subspaces through BSS. The results are illustrated with a simple example, and the implications for a variety of signal processing applications, such as denoising and model identification, are discussed.