We study an online and streaming unsupervised classification system. Our setting consists of a collection of classifiers (with unknown confusion matrices) each of which can classify one sample per unit time, and which are accessed by a stream of unlabeled samples. Each sample is dispatched to one or more classifiers, and depending on the labels collected from these classifiers, may be sent to other classifiers to collect additional labels. The labels are continually aggregated. Once the aggregated label has high enough accuracy (a pre-specified threshold for accuracy) or the sample is sent to all the classifiers, the now labeled sample is ejected from the system. For any given pre-specified threshold for accuracy, the objective is to sustain the maximum possible rate of arrival of new samples, such that the number of samples in memory does not grow unbounded. In this paper, we characterize the Pareto-optimal region of accuracy and arrival rate, and develop an algorithm that can operate at any point within this region. Our algorithm uses queueing-based routing and scheduling approaches combined with novel online tensor decomposition method to learn the hidden parameters, to Pareto-optimality guarantees. We finally verify our theoretical results through simulations on two ensembles formed using AlexNet, VGG, and ResNet deep image classifiers.

This study alters various exemplar presentation parameters to determine their effects on human online category learning for a future system that combines humans and computer vision (CV). Online category learning is necessary in this system because we envision that humans will need to provide input to assist CV modules in determining category labels without reducing throughput and without necessarily having expert knowledge of each category. In our study, subjects participated in a Rapid Serial Visual Presentation paradigm in which they were asked to determine the target category from highlighted exemplar images interspersed among distractor images. In Experiment 1, the highlighting method was varied among four options and a negative (no-label) and positive (explicit, text-based) control. In Experiment 2, label accuracy was altered by incorrectly labeling some distractor and exemplar images. In both experiments, there were three levels of difficulty that varied the similarity between distractor and exemplar images. The results show that most highlighting methods resulted in equivalent accuracy to the positive control, but certain modalities were more effective at varying difficulty levels. In addition, the subject accuracy was sensitive to distractors highlighted as targets, but not to nonhighlighted exemplars. Our results indicate that human online category learning can be optimized for human-system interaction.

One big challenge that hinders the transition of brain-computer interfaces (BCIs) from laboratory settings to real-life applications is the availability of high-performance and robust learning algorithms that can effectively handle individual differences, i.e., algorithms that can be applied to a new subject with zero or very little subject-specific calibration data. Transfer learning and domain adaptation have been extensively used for this purpose. However, most previous works focused on classification problems. This paper considers an important regression problem in BCI, namely, online driver drowsiness estimation from EEG signals. By integrating fuzzy sets with domain adaptation, we propose a novel online weighted adaptation regularization for regression (OwARR) algorithm to reduce the amount of subject-specific calibration data, and also a source domain selection (SDS) approach to save about half of the computational cost of OwARR. Using a simulated driving dataset with 15 subjects, we show that OwARR and OwARR-SDS can achieve significantly smaller estimation errors than several other approaches. We also provide comprehensive analyses on the robustness of OwARR and OwARR-SDS.

The papers in this special section focus on brain computer interfaces. A brain computer interface (BCI) enables direct communication between the brain and a computer. It can be used to research, repair, or enhance human cognitive or sensorymotor functions. BCIs have attracted rapidly increasing research interest in the last decade, thanks to recent advances in neurosciences, wearable/mobile biosensors, and analytics. However, there are many challenges in the transition from laboratory settings to real-life applications, including the reliability and convenience of the sensing hardware, the availability of high performance and robust algorithms for signal analysis and interpretation, and fundamental advances in automated reasoning that enable the reasoning and generalization across individuals. Computational intelligence techniques, particularly fuzzy sets and systems, have demonstrated outstanding performance in handling uncertainties in many real-world applications.

As the technology for acquiring and storing images becomes more prevalent, we are faced with a growing need to sort and label these images. At this time, computer vision algorithms cannot parse abstract concepts from images like a human. As a result, there may be performance gains possible from the integration of human analysts with computer vision agents. We present an image triage system which facilitates the collaboration of heterogeneous agents through a novel unsupervised meta-learning technique. The system iteratively allocates images for binary classification among heterogeneous agents according to the Generalized Assignment Problem (GAP) and combines the classification results using the Spectral Meta-Learner (SML). In simulation, we demonstrate that the proposed system achieves significant speed-up over a naive parallel assignment strategy without sacrificing accuracy.

Ensemble learning is a powerful approach to construct a strong learner from multiple base learners. The most popular way to aggregate an ensemble of classifiers is majority voting, which assigns a sample to the class that most base classifiers vote for. However, improved performance can be obtained by assigning weights to the base classifiers according to their accuracy. This paper proposes an agreement rate initialized maximum likelihood estimator (ARIMLE) to optimally fuse the base classifiers. ARIMLE first uses a simplified agreement rate method to estimate the classification accuracy of each base classifier from the unlabeled samples, then employs the accuracies to initialize a maximum likelihood estimator (MLE), and finally uses the expectation-maximization algorithm to refine the MLE. Extensive experiments on visually evoked potential classification in a brain-computer interface application show that ARIMLE outperforms majority voting, and also achieves better or comparable performance with several other state-of-the-art classifier combination approaches.

There are many important regression problems in real-world brain-computer interface (BCI) applications, e.g., driver drowsiness estimation from EEG signals. This paper considers offline analysis: given a pool of unlabeled EEG epochs recorded during driving, how do we optimally select a small number of them to label so that an accurate regression model can be built from them to label the rest? Active learning is a promising solution to this problem, but interestingly, to our best knowledge, it has not been used for regression problems in BCI so far. This paper proposes a novel enhanced batch-mode active learning (EBMAL) approach for regression, which improves upon a baseline active learning algorithm by increasing the reliability, representativeness and diversity of the selected samples to achieve better regression performance. We validate its effectiveness using driver drowsiness estimation from EEG signals. However, EBMAL is a general approach that can also be applied to many other offline regression problems beyond BCI.

To facilitate the transition of brain-computer interface (BCI) systems from laboratory settings to real-world application, it is very important to minimize or even completely eliminate the subject-specific calibration requirement. There has been active research on calibrationless BCI systems for classification applications, e.g., P300 speller. To our knowledge, there is no literature on calibrationless BCI systems for regression applications, e.g., estimating the continuous drowsiness level of a driver from EEG signals. This paper proposes a novel spectral meta-learner for regression (SMLR) approach, which optimally combines base regression models built from labeled data from auxiliary subjects to label offline EEG data from a new subject. Experiments on driver drowsiness estimation from EEG signals demonstrate that SMLR significantly outperforms three state-of-the-art regression model fusion approaches. Although we introduce SMLR as a regression model fusion in the BCI domain, we believe its applicability is far beyond that.

Rapid serial visual presentation based brain-computer interface (BCI) system relies on single-trial classification of event-related potentials. Because of large individual differences, some labeled subject-specific data are needed to calibrate the classifier for each new subject. This paper proposes an online weighted adaptation regularization (OwAR) algorithm to reduce the online calibration effort, and hence to increase the utility of the BCI system. We show that given the same number of labeled subject-specific training samples, OwAR can significantly improve the online calibration performance. In other words, given a desired classification accuracy, OwAR can significantly reduce the number of labeled subject-specific training samples. Furthermore, we also show that the computational cost of OwAR can be reduced by more than 50% by source domain selection, without a statistically significant sacrifice of classification performance.

Prior research has shown the utility of labeling images by rapidly displaying them to humans via a Rapid Serial Visual Presentation (RSVP) paradigm, classifying the resulting neural data, and integrating the results with computer vision. However, there is currently very little research on providing feedback to the human interacting with one of these systems. To explore this question, an RSVP task was developed to examine the effectiveness of feedback to induce changes in target category in near-real time. Three different factors involved in image presentation were explored: image presentation duration, target/distract or similarity, and feedback modality. Significant, nonlinear changes in performance were related to these independent variables. These results demonstrate the complexity inherent to human category learning and will guide future use of image presentation parameters to optimize human performance within a human-assisted computing system that is focused on image analysis.

Electroencephalography (EEG) has been widely used in a variety of contexts, including medical monitoring of subjects as well as performance monitoring in healthy individuals. Recent technological advances have now enabled researchers to quickly record and collect EEG on a wide scale. Although EEG is fairly easy to record, it is highly susceptible to noise sources called artifacts which can occur at amplitudes several times greater than the EEG signal of interest. Because of this, users must manually annotate the EEG signal to identify artifact regions in the data prior to any downstream processing. This can be time-consuming and impractical for large data collections. In this paper we present a method which uses Active Learning (AL) to improve the reliability of existing EEG artifact classifiers with minimal amounts of user interaction. Our results show that classification accuracy equivalent to classifiers trained on full data annotation can be obtained while labeling less than 25% of the data. This suggests significant time savings can be obtained when manually annotating artifacts in large EEG data collections.

Single-trial classification of Event-Related Potentials (ERPs) is needed in many real-world brain-computer interface (BCI) applications. However, because of individual differences, the classifier needs to be calibrated by using some labeled subject specific training samples, which may be inconvenient to obtain. In this paper we propose a weighted adaptation regularization (wAR) approach for offline BCI calibration, which uses data from other subjects to reduce the amount of labeled data required in offline single-trial classification of ERPs. Our proposed model explicitly handles class-imbalance problems which are common in many real-world BCI applications. War can improve the classification performance, given the same number of labeled subject-specific training samples, or, equivalently, it can reduce the number of labeled subject-specific training samples, given a desired classification accuracy. To reduce the computational cost of wAR, we also propose a source domain selection (SDS) approach. Our experiments show that wARSDS can achieve comparable performance with wAR but is much less computationally intensive. We expect wARSDS to find broad applications in offline BCI calibration.

Controls frameworks for human-autonomy integration (HAI) often treat human sources of information as highly reliable [1]. However, data from human sensing are quite variable, both because of human nature and limitations on sensor technologies. This paper focuses on estimating the degree of uncertainty in human sensed data, i.e., developing confidence metrics, and implementing those metrics in a HAI for target recognition. Here we demonstrate that applying such confidence estimates to sensed human data can mitigate effects of variability in human sensing and improve HAI performance.

Single-trial Event-Related Potential (ERP) classification is a key requirement for several types of Brain-Computer Interaction (BCI) technologies. However, strong individual differences make it challenging to develop a generic single-trial ERP classifier that performs well for all subjects. Usually some subject-specific training samples need to be collected in an initial calibration session to customize the classifier. However, if implemented into an actual BCI system, then this calibration process would decrease the utility of the system, potentially decreasing its usability. In this paper we propose a Transfer Learning approach for reducing the amount of subject-specific data in online single-trial ERP classifier calibration, and an Active Transfer Learning approach for offline calibration. By applying these approaches to data from a Visually-Evoked Potential EEG experiment, we demonstrate that they improve the classification performance, given the same number of labeled subject-specific training samples. In other words, these approaches can also attain a desired level of classification accuracy with less labeling effort when compared to a randomly selected training set.

Neurotechnologies based on electroencephalography (EEG) and other physiological measures to improve task performance in complex environments will require tools and analysis methods that can account for increased environmental noise and task complexity compared to traditional neuroscience laboratory experiments. We propose a bag-of-words (BoW) model to address the difficulties associated with realistic applications in complex environments. In this paper, our proof-of-concept results show that a BoW classifier can discriminate two task-relevant states (high versus low task-load) while an individual performs a simulated security patrol mission with complex, concurrent tasking. Classifier performance is largely consistent across six simulation missions for a given participant, but performance decreases when trying to predict between two individuals. Overall, these initial results suggest that this BoW approach holds promise for detecting task-relevant states in real-world settings.

As the proliferation of technology dramatically infiltrates all aspects of modern life, in many ways the world is becoming so dynamic and complex that technological capabilities are overwhelming human capabilities to optimally interact with and leverage those technologies. Fortunately, these technological advancements have also driven an explosion of neuroscience research over the past several decades, presenting engineers with a remarkable opportunity to design and develop flexible and adaptive brain-based neurotechnologies that integrate with and capitalize on human capabilities and limitations to improve human-system interactions. Major forerunners of this conception are brain-computer interfaces (BCIs), which to this point have been largely focused on improving the quality of life for particular clinical populations and include, for example, applications for advanced communications with paralyzed or “locked in” patients as well as the direct control of prostheses and wheelchairs. Near-term applications are envisioned that are primarily task oriented and are targeted to avoid the most difficult obstacles to development. In the farther term, a holistic approach to BCIs will enable a broad range of task-oriented and opportunistic applications by leveraging pervasive technologies and advanced analytical approaches to sense and merge critical brain, behavioral, task, and environmental information. Communications and other applications that are envisioned to be broadly impacted by BCIs are highlighted; however, these represent just a small sample of the potential of these technologies.

A closed-loop system that offers real-time assessment and manipulation of a user's affective and cognitive states is very useful in developing adaptive environments which respond in a rational and strategic fashion to real-time changes in user affect, cognition, and motivation. The goal is to progress the user from suboptimal cognitive and affective states toward an optimal state that enhances user performance. In order to achieve this, there is need for assessment of both 1) the optimal affective/cognitive state and 2) the observed user state. This paper presents approaches for assessing these two states. Arousal, an important dimension of affect, is focused upon because of its close relation to a user's cognitive performance, as indicated by the Yerkes-Dodson Law. Herein, we make use of a Virtual Reality Stroop Task (VRST) from the Virtual Reality Cognitive Performance Assessment Test (VRCPAT) to identify the optimal arousal level that can serve as the affective/cognitive state goal. Three stimuli presentations (with distinct arousal levels) in the VRST are selected. We demonstrate that when reaction time is used as the performance measure, one of the three stimuli presentations can elicit the optimal level of arousal for most subjects. Further, results suggest that high classification rates can be achieved when a support vector machine is used to classify the psychophysiological responses (skin conductance level, respiration, ECG, and EEG) in these three stimuli presentations into three arousal levels. This research reflects progress toward the implementation of a closed-loop affective computing system.