The current standard for training brain-computer interface (BCI) machine learning models is user-specific. There is a high interest in developing generic models that are trained on data from other users to minimize BCI calibration time; however, this is limited by noisy, non-stationary brain signals and high inter-user variability. We investigate the trade-off between training data quality and quantity on P300 BCI performance in individuals with amyotrophic lateral sclerosis (ALS) with representative traditional machine learning (stepwise linear discriminant analysis, SWLDA) and deep learning (EEGNet) models. Results show that data quality and domain alignment are more critical than dataset size: user-specific models trained on significantly less data outperformed generic models; generic models trained on ALS data outperformed models trained on non-ALS data; block-averaging of features was mostly detrimental to EEGNet but beneficial to SWLDA; and accounting for inter-stimulus interval differences between ALS and non-ALS data had minimal effect. Our findings highlight the importance of individualized model tuning for reliable P300 BCIs.

Brain-computer interfaces (BCIs) can provide a means of communication for individuals with severe neuromuscular diseases, the target end-users. While personalized BCI machine learning models are the current standard, models trained on data from other users could reduce BCI calibration time. We use a novel dataset with BCI users with and without amyotrophic lateral sclerosis (ALS) and a popular BCI deep learning model, EEGNet, to assess the impact of population domain data on transfer learning of a P300 speller task in the ALS cohort. Results show that training on source data from the non-ALS cohort was detrimental to transfer learning. In contrast, generic EEGNet models trained on source data from the ALS cohort performed comparably as user-specific models. Our findings highlight the need for more data from target end-users populations in publicly available BCI datasets.

Mixture-of-Experts (MoE) models have become popular in machine learning, boosting performance by partitioning tasks across multiple experts. However, the need for several experts often results in high computational costs, limiting their application on resource-constrained devices with stringent real-time requirements, such as cochlear implants (CIs). We introduce the Omni-Expert (OE) – a simple and efficient solution that leverages feature transformations to achieve the 'divide-and-conquer' functionality of a full MoE ensemble in a single expert model. We demonstrate the effectiveness of the OE using phoneme-specific time-frequency masking for speech dereverberation in a CI. Empirical results show that the OE delivers statistically significant improvements in objective intelligibility measures of CI vocoded speech at different levels of reverberation across various speech datasets at a much reduced computational cost relative to a counterpart MoE.

Brain-computer interfaces (BCIs), such as the P300 speller, can provide a means of communication for individuals with severe neuromuscular limitations. BCIs interpret electroencephalography (EEG) signals in order to translate embedded information about a user's intent into executable commands to control external devices. However, EEG signals are inherently noisy and nonstationary, posing a challenge to extended BCI use. Conventionally, a BCI classifier is trained via supervised learning in an offline calibration session; once trained, the classifier is deployed for online use and is not updated. As the statistics of a user's EEG data change over time, the performance of a static classifier may decline with extended use. It is therefore desirable to automatically adapt the classifier to current data statistics without requiring offline recalibration. In an existing semi-supervised learning approach, the classifier is trained on labeled EEG data and is then updated using incoming unlabeled EEG data and classifier-predicted labels. To reduce the risk of learning from incorrect predictions, a threshold is imposed to exclude unlabeled data with low-confidence label predictions from the expanded training set when retraining the adaptive classifier. In this work, we propose the use of a language model for spelling error correction and disambiguation to provide information about label correctness during semi-supervised learning. Results from simulations with multi-session P300 speller user EEG data demonstrate that our language-guided semi-supervised approach significantly improves spelling accuracy relative to conventional BCI calibration and threshold-based semi-supervised learning.

Speech intelligibility in cochlear implant (CI) users degrades considerably in listening environments with reverberation and noise. Previous research in automatic speech recognition (ASR) has shown that phoneme-based speech enhancement algorithms improve ASR system performance in reverberant environments as compared to a global model. However, phoneme-specific speech processing has not yet been implemented in CIs. In this paper, we propose a causal deep learning framework for classifying phonemes using features extracted at the time-frequency resolution of a CI processor. We trained and tested long short-term memory networks to classify phonemes and manner of articulation in anechoic and reverberant conditions. The results showed that CI-inspired features provide slightly higher levels of performance than traditional ASR features. To the best of our knowledge, this study is the first to provide a classification framework with the potential to categorize phonetic units in real-time in a CI.

Cochlear implant (CI) users experience substantial difficulties in understanding reverberant speech. A previous study proposed a strategy that leverages automatic speech recognition (ASR) to recognize reverberant speech and speech synthesis to translate the recognized text into anechoic speech. However, the strategy was trained and tested on the same reverberant environment, so it is unknown whether the strategy is robust to unseen environments. Thus, the current study investigated the performance of the previously proposed algorithm in multiple unseen environments. First, an ASR system was trained on anechoic and reverberant speech using different room types. Next, a speech synthesizer was trained to generate speech from the text predicted by the ASR system. Experiments were conducted in normal hearing listeners using vocoded speech, and the results showed that the strategy improved speech intelligibility in previously unseen conditions. These results suggest that the ASR-synthesis strategy can potentially benefit CI users in everyday reverberant environments.

Latent Dirichlet allocation (LDA) is a statistical model that is often used to discover topics or themes in a large collection of documents. In the LDA model, topics are modeled as discrete distributions over a finite vocabulary of words. The LDA is also a popular choice to model other datasets spanning a discrete domain, such as population genetics and social networks. However, in order to model data spanning a continuous domain with the LDA, discrete approximations of the data need to be made. These discrete approximations to continuous data can lead to loss of information and may not represent the true structure of the underlying data. We present an augmented version of the LDA topic model, where topics are represented using Gaussian mixture models (GMMs), which are multi-modal distributions spanning a continuous domain. This augmentation of the LDA topic model with Gaussian mixture topics is denoted by the GMM-LDA model. We use Gibbs sampling to infer model parameters. We demonstrate the utility of the GMM-LDA model by applying it to the problem of clustering sleep states in electroencephalography (EEG) data. Results are presented demonstrating superior clustering performance with our GMM-LDA algorithm compared to the standard LDA and other clustering algorithms.

Individuals with cochlear implants (CIs) experience more difficulty understanding speech in reverberant environments than normal hearing listeners. As a result, recent research has targeted mitigating the effects of late reverberant signal reflections in CIs by using a machine learning approach to detect and delete affected segments in the CI stimulus pattern. Previous work has trained electrode-specific classification models to mitigate late reverberant signal reflections based on features extracted from only the acoustic activity within the electrode of interest. Since adjacent CI electrodes tend to be activated concurrently during speech, we hypothesized that incorporating additional information from the other electrode channels, termed cross-channel information, as features could improve classification performance. Cross-channel information extracted in real-world conditions will likely contain errors that will impact classification performance. To simulate extracting cross-channel information in realistic conditions, we developed a graphical model based on the Ising model to systematically introduce errors to specific types of cross-channel information. The Ising-like model allows us to add errors while maintaining the important geometric information contained in cross-channel information, which is due to the spectro-temporal structure of speech. Results suggest the potential utility of leveraging cross-channel information to improve the performance of the reverberation mitigation algorithm from the baseline channel-based features, even when the cross-channel information contains errors.

Stimulus-driven brain-computer interfaces (BCIs), such as the P300 speller, rely on using a sequence of sensory stimuli to elicit specific neural responses as control signals, while a user attends to relevant target stimuli that occur within the sequence. In current BCIs, the stimulus presentation schedule is typically generated in a pseudo-random fashion. Given the non-stationarity of brain electrical signals, a better strategy could be to adapt the stimulus presentation schedule in real-time by selecting the optimal stimuli that will maximize the signal-to-noise ratios of the elicited neural responses and provide the most information about the user's intent based on the uncertainties of the data being measured. However, the high-dimensional stimulus space limits the development of algorithms with tractable solutions for optimized stimulus selection to allow for real-time decision-making within the stringent time requirements of BCI processing. We derive a simple analytical solution of an information-based objective function for BCI stimulus selection by transforming the high-dimensional stimulus space into a one-dimensional space that parameterizes the objective function - the prior probability mass of the stimulus under consideration, irrespective of its contents. We demonstrate the utility of our adaptive stimulus selection algorithm in improving BCI performance with results from simulation and real-time human experiments.

Immersive virtual reality (VR) systems offer flexible control of an interactive environment, along with precise position and orientation tracking of realistic movements. Immersive VR can also be used in conjunction with neurophysiological monitoring techniques, such as electroencephalography (EEG), to record neural activity as users perform complex tasks. As such, the fusion of VR, kinematic tracking, and EEG offers a powerful testbed for naturalistic neuroscience research. In this study, we combine these elements to investigate the cognitive and neural mechanisms that underlie motor skill learning during a multi-day simulated marksmanship training regimen conducted with 20 participants. On each of 3 days, participants performed 8 blocks of 60 trials in which a simulated clay pigeon was launched from behind a trap house. Participants attempted to shoot the moving target with a firearm game controller, receiving immediate positional feedback and running scores after each shot. Over the course of the 3 days that individuals practiced this protocol, shot accuracy and precision improved significantly while reaction times got significantly faster. Furthermore, results demonstrate that more negative EEG amplitudes produced over the visual cortices correlate with better shooting performance measured by accuracy, reaction times, and response times, indicating that early visual system plasticity underlies behavioral learning in this task. These findings point towards a naturalistic neuroscience approach that can be used to identify neural markers of marksmanship performance.

The P300-based brain-computer interface (BCI) speller relies on eliciting and detecting specific brain responses to target stimulus events, termed event-related potentials (ERPs). In a visual speller, ERPs are elicited when the user's desired character, i.e. the “target,” is flashed on a computer screen. The P300 speller is currently limited by its relatively slow typing speed due to the need for repetitive data measurements that are necessary to achieve reasonable signal-to-noise ratios. In addition, refractory effects limit the ability to elicit ERPs with every target stimulus event presentation. In this paper, we present a new method to design the stimulus presentation paradigm for the P300 speller by exploiting an information-theoretic approach to maximize the information content that is presented to the user while also mitigating refractory effects. We present results with real-time BCI use which demonstrate significant performance improvements with our performance-based paradigm compared to the conventional stimulus presentation paradigm.

Brain-computer interfaces (BCIs) can provide an alternative means of communication for individuals with severe neuromuscular limitations. The P300-based BCI speller relies on eliciting and detecting transient event-related potentials (ERPs) in electroencephalography (EEG) data, in response to a user attending to rarely occurring target stimuli amongst a series of non-target stimuli. However, in most P300 speller implementations, the stimuli to be presented are randomly selected from a limited set of options and stimulus selection and presentation are not optimized based on previous user data. In this work, we propose a data-driven method for stimulus selection based on the expected discrimination gain metric. The data-driven approach selects stimuli based on previously observed stimulus responses, with the aim of choosing a set of stimuli that will provide the most information about the user's intended target character. Our approach incorporates knowledge of physiological and system constraints imposed due to real-time BCI implementation. Simulations were performed to compare our stimulus selection approach to the row-column paradigm, the conventional stimulus selection method for P300 spellers. Results from the simulations demonstrated that our adaptive stimulus selection approach has the potential to significantly improve performance from the conventional method: up to 34% improvement in accuracy and 43% reduction in the mean number of stimulus presentations required to spell a character in a 72-character grid. In addition, our greedy approach to stimulus selection provides the flexibility to accommodate design constraints.