A brain-computer interface (BCI) enables direct communication between the brain and an external device. Electroencephalogram (EEG) is the preferred input signal in non-invasive BCIs, due to its convenience and low cost. EEG-based BCIs have been successfully used in many applications, such as neurological rehabilitation, text input, games, and so on. However, EEG signals inherently carry rich personal information, necessitating privacy protection. This paper demonstrates that multiple types of private information (user identity, gender, and BCI-experience) can be easily inferred from EEG data, imposing a serious privacy threat to BCIs, To address this issue, we design perturbations to convert the original EEG data into privacy-protected EEG data, which conceal the private information while maintaining the primary BCI task performance. Experimental results demonstrated that the privacy-protected EEG data can significantly reduce the classification accuracy of user identity, gender and BCI-experience, but almost do not affect at all the classification accuracy of the primary BCI task, enabling user privacy protection in EEG-based BCIs.

Emotion recognition is an important part of affective computing. Utilizing information from multiple modalities would facilitate more accurate emotion recognition. The performance of data-driven machine learning models usually relies on a large amount of labeled training data. However, labeling emotional data is expensive, because each sample usually requires multiple evaluators to annotate. To alleviate the annotation cost, this paper proposes a cross-modal diversity measure that considers the correlation between different modalities and integrates it with the representativeness for sample selection in unsupervised active learning (AL) for regression. To our knowledge, this challenging multi-modal unsupervised AL scenario has not been explored before: previous research only considered either unsupervised uni-modal AL or supervised multi-modal AL. Experiments on RECOLA and IEMOCAP datasets demonstrated the effectiveness of our proposed AL approach.

Emotion recognition, which aims to identify an individual’s emotional state from the acquired physiological or body signals, is very important in affective computing. Emotions have two common representations: categorical, e.g., happy, sad, etc., and dimensional (continuous), e.g., valence, arousal and dominance. Training a good emotion classification or regression model usually requires a large number of labeled data. However, the labeling process is very difficult. As emotions are subtle and uncertain, it usually requires multiple assessors to label each emotional instance to obtain the groundtruth categorical label or dimensional values. In this paper, we propose a multi-task active learning (MTAL) framework to query the most useful samples for labeling, which enables the efficient training of an emotion classification model and multiple emotion regression models simultaneously. This is novel and challenging, as all previous research considered only emotion classification or regression alone, but not simultaneously. Experimental results on the IEMOCAP dataset demonstrated that MTAL outperformed random selection and several state-of-the-art single task active learning approaches, i.e., with the same number of labeled samples, MTAL can obtain better emotion classification and regression models simultaneously.

Effective and real-time eyeblink detection is of wide-range applications, such as deception detection, drive fatigue detection, face anti-spoofing. Despite previous efforts, most of existing focus on addressing the eyeblink detection problem under constrained indoor conditions with relative consistent subject and environment setup. Nevertheless, towards practical applications, eyeblink detection in the wild is highly preferred, and of greater challenges. In this paper, we shed the light to this research topic. A labelled eyeblink in the wild dataset (i.e., HUST-LEBW) of 673 eyeblink video samples (i.e., 381 positives, and 292 negatives) is first established. These samples are captured from the unconstrained movies, with the dramatic variation on face attribute, head pose, illumination condition, imaging configuration, etc. Then, we formulate eyeblink detection task as a binary spatial-temporal pattern recognition problem. After locating and tracking human eyes using SeetaFace engine and KCF (Kernelized Correlation Filters) tracker respectively, a modified LSTM model able to capture the multi-scale temporal information is proposed to verify eyeblink. A feature extraction approach that reveals the appearance and motion characteristics simultaneously is also proposed. The experiments on HUST-LEBW reveal the superiority and efficiency of our approach. The comparisons with the existing state-of-the-art methods validate the advantages of our manner for eyeblink detection in the wild.