Brain-Computer Interfaces (BCIs) using functional near-infrared spectroscopy (fNIRS) hold promise for future interactive user interfaces due to their ease of deployment and declining cost. However, they typically require a separate calibration process for each user and task, which can be burdensome. Machine learning helps, but faces a data scarcity problem. Due to inherent inter-user variations in physiological data, it has been typical to create a new annotated training dataset for every new task and user. To reduce dependence on such extensive data collection and labeling, we present an adaptive learning framework, NIRS-X, to harness more easily accessible unlabeled fNIRS data. NIRS-X includes two key components: NIRSiam and NIRSformer. We use the NIRSiam algorithm to extract generalized brain activity representations from unlabeled fNIRS data obtained from previous users and tasks, and then transfer that knowledge to new users and tasks. In conjunction, we design a neural network, NIRSformer, tailored for capturing both local and global, spatial and temporal relationships in multi-channel fNIRS brain input signals. By using unlabeled data from both a previously released fNIRS2MW visual n-back dataset and a newly collected fNIRS2MW audio n-back dataset, NIRS-X demonstrates its strong adaptation capability to new users and tasks. Results show comparable or superior performance to supervised methods, making NIRS-X promising for real-world fNIRS-based BCIs.

Brain-computer interfaces (BCI) are an emerging technology with many potential applications. Functional near-infrared spectroscopy (fNIRS) can provide a convenient and unobtrusive real time input for BCI. fNIRS is especially promising as a signal that could be used to automatically classify a user’s current cognitive workload. However, the data needed to train such a classifier is currently not widely available, difficult to collect, and difficult to interpret due to noise and cross-subject variation. A further challenge is the need for significant user-specific calibration. To address these issues, we introduce a new dataset gathered from 15 subjects and a new multi-stage supervised machine learning pipeline. Our approach learns from both observed data and augmented data derived from multiple subjects in its early stages, and then fine-tunes predictions to an individual subject in its last stage. We show promising gains in accuracy in a standard “n-back” cognitive workload classification task compared to baselines that use only subject-specific data or only group-level data, even when our approach is given much less subject-specific data. Even though these experiments analyzed the data retrospectively, we carefully removed anything from our process that could not have been done in real time, because our process is targeted at future real-time operation. This paper contributes a new dataset, a new multi-stage training pipeline, results showing significant improvement compared to alternative pipelines, and discussion of the implications for user interface design. Our complete dataset and software are publicly available at https://tufts-hci-lab.github.io/code_and_datasets/. We hope these results make fNIRS-based interactive brain input easier for a wide range of future researchers and designers to explore.