The brain-computer interface (BCI) based on steady-state visual evoked potential (SSVEP) has received considerable attention for its high communication speed. While large datasets provide an important opportunity to enhance decoding accuracies, the key challenge lies in the exploration of existing data to extract valuable information based on the distinctive characteristics of brain responses. In this study, we introduce ConsenNet, a framework designed to enhance SSVEP classification performance by leveraging information from the diverse perspectives of existing subjects. First, this study exploits the diversity of existing subjects to generate new samples, which retain both task-related components and variability. This effectively enhances the network generalization capability on new subjects. Second, the structured knowledge that encapsulates the interrelationships between categories has been constructed and then transferred from the teacher network to the student network, guiding the student network to extract invariant features across subjects. Finally, our model incorporates a small amount of new subject data for model calibration in the final stage. Offline experiments conducted on three public datasets demonstrate the superiority of ConsenNet over 19 methods compared in this study, while online experiments validate its feasibility for real-world applications.

Recently, brain-computer interface (BCI) technology has made impressive progress and has been developed for many applications. Thereinto, the BCI system based on rapid serial visual presentation (RSVP) is a promising information detection technology. However, the use of RSVP is closely related to the user's performance, which can be influenced by their vigilance levels. Therefore it is crucial to detect vigilance levels in RSVP-based BCI. In this paper, we conducted a long-term RSVP target detection experiment to collect electroencephalography (EEG) and electrooculogram (EOG) data at different vigilance levels. In addition, to estimate vigilance levels in RSVP-based BCI, we propose a multimodal method named VigilanceNet using EEG and EOG. Firstly, we define the multiplicative relationships in conventional EOG features that can better describe the relationships between EOG features, and design an outer product embedding module to extract the multiplicative relationships. Secondly, we propose to decouple the learning of intra- and inter-modality to improve multimodal learning. Specifically, for intra-modality, we introduce an intra-modality representation learning (intra-RL) method to obtain effective representations of each modality by letting each modality independently predict vigilance levels during the multimodal training process. For inter-modality, we employ the cross-modal Transformer based on cross-attention to capture the complementary information between EEG and EOG, which only pays attention to the inter-modality relations. Extensive experiments and ablation studies are conducted on the RSVP and SEED-VIG public datasets. The results demonstrate the effectiveness of the method in terms of regression error and correlation.

Brain-computer interface (BCI) systems provide a direct connection between the human brain and external devices. Visual evoked BCI systems including Event-related Potential (ERP) and Steady-state Visual Evoked Potential (SSVEP) have attracted extensive attention because of their strong brain responses and wide applications. Previous studies have made some breakthroughs in within-subject decoding algorithms for specific tasks. However, there are two challenges in current decoding algorithms in BCI systems. Firstly, current decoding algorithms cannot accurately classify EEG signals without the data of the new subject, but the calibration procedure is time-consuming. Secondly, algorithms are tailored to extract features for one specific task, which limits their applications across tasks. In this study, we proposed a Temporal-Frequency Fusion Transformer (TFF-Former) for zero-training decoding across two BCI tasks. EEG data were organized into temporal-spatial and frequency-spatial forms, which can be considered as two views. In the TFF-Former framework, two symmetrical Transformer streams were designed to extract view-specific features. The cross-view module based on the cross-attention mechanism was proposed to guide each stream to strengthen common representations of features across EEG views. Additionally, an attention-based fusion module was built to fuse the representations from the two views effectively. The mean mask mechanism was applied to adaptively decrease redundant EEG tokens aggregation for the integration of common representations. We validated our method on the self-collected RSVP dataset and benchmark SSVEP dataset. Experimental results demonstrated that our TFF-Former model achieved competitive performance compared with models in each of the above paradigms. It can further promote the application of visual evoked EEG-based BCI system.

Brain-Computer Interface (BCI) is a direct communication medium between brain and outside world. This study focuses on a Steady-State Visually Evoked Potential (SSVEP)based BCI due to its large number of instruction set. However, it is still challenging to decode multi-class SSVEPs. To improve target identification accuracy, we propose a Siamese Correlation Analysis model (SiamCA), which involves two feature extractors with tied parameters and a top decision network. We consider two datasets for benchmarking the performance of the proposed model and compare it with FBCCA, TRCA, ensemble-TRCA and a deep learning method named ConvCA. The proposed method realize a significantly higher average classification accuracy than the compared method at different data length (0.2–1.0s) on two datasets. This suggests that the proposed SiamCA model is a promising methodology for target identification of SSVEPs and could further improve the performance of SSVEP-based BCI system.

Rapid Serial Visual Presentation (RSVP) is an important paradigm in Brain-Computer Interface (BCI). It can be used in speller, image retrieval, anomaly detection, etc. RSVP paradigm uses a small number of target pictures in a high speed presented picture sequence to induce specific event-related potential (ERP) components. However, the application of RSVP based BCI is challenged by the accuracy of ERP detection. Thus, the goal of this study is to introduce other related modalities to the traditional EEG-based BCI to make robust predictions and improve the detection performance. First, we introduce the eye movement modality into the RSVP-based BCI and collect a multimodality RSVP-based dataset simultaneously during the image retrieval task. Second, we design a simple but efficient CNN-based network with two modality fusion modules to fully utilize the multi-modality data in two stages. In the feature extraction stage, we propose a Cross-modality-Guided Feature Calibration (cm-GFC) module to enable the EEG modality feature to modify the eye movement modality feature, and the aim is to make eye movement modality features and EEG modality features are more complementary. In the feature fusion stage, we propose a Dynamic Gated Fusion (DGF) module, which applies modality-specific gates to retain the complementary information of the two modalities and reduce redundant information from the two modalities. To evaluate our method, we conduct extensive experiments on the dataset with EEG and eye movement data are from 20 subjects. The proposed method achieves a high balanced accuracy of 87.83 ± 2.31% of classification, which outperforms a series of single modality and multi-modality approaches.

Motor imagery (MI) based Brain-computer interface (BCI) is a promising BCI paradigm that can help neuromuscular injury patients to recover or replace their motor abilities. However, electroencephalography (EEG) based MI-BCI suffers from its long calibration time and low classification accuracy, which restrict its application. Thus, it is important to reduce the calibration time of MI-BCI and enhance its prediction accuracy. In this study, we propose a filter bank Wasserstein adversarial domain adaptation framework (FBWADA) that uses a short amount of training data from a new target subject, and all collected data from an existing subject. A Convolutional Neural Networks (CNN) based feature extractor is designed to extract feature from EEG data. Filter bank strategy is employed to extract feature from multiple sub bands and integrate predictions from all sub bands. Wasserstein Generative Adversarial Networks (WGAN) based domain adaptation network aligns the marginal and conditional distribution of target and source. We evaluate our method on Data set 2a of BCI competition IV. Experiment results show that our method achieves the best performance among compared methods under different amount of training data. Performance of our method trained with certain blocks of data is similar to or better than the best comparing method trained with one more block. This indicates that our method could reduce the need for training data for at least one block.

Retinal layers segmentation in optical coherence tomography (OCT) images is a critical step in the diagnosis of numerous ocular diseases. Automatic layers segmentation requires separating each individual layer instance with accurate boundary detection, but remains a challenging task since it suffers from speckle noise, intensity inhomogeneity, and the low contrast around boundary. In this work, we proposed a boundary aware U-Net (BAU-Net) for retinal layers segmentation by detecting accurate boundary. Based on encoder-decoder architecture, we design a dual tasks framework with low-level outputs for boundary detection and high-level outputs for layers segmentation. Specifically, we first use the multi-scale input strategy to enrich the spatial information in the deep features of encoder. For low-level features from encoder, we design an edge aware (EA) module in skip connection to extract the pure edge features. Then, a U-structure feature enhanced (UFE) module is designed in all skip connections to enlarge the features receptive fields from the encoder. Besides, a canny edge fusion (CEF) module is introduced to aforementioned architecture, which can fuse the priory edge information from segmentation task to boundary detection branch for a better predication. Furthermore, we model each boundary as a vertical coordinates distribution for boundary detection. Based on this distribution, a topology guarantee loss with combined A-scan regression loss and structure loss is proposed to make an accurate and guaranteed topological boundary set. The method is evaluated on two public datasets and the results demonstrate that the BAU-Net achieves promising performance than other state-of-the-art methods.

Blood vessel segmentation in fundus images is a critical procedure in the diagnosis of ophthalmic diseases. Recent deep learning methods achieve high accuracy in vessel segmentation but still face the challenge to segment the microvascular and detect the vessel boundary. This is due to the fact that common Convolutional Neural Networks (CNN) are unable to preserve rich spatial information and a large receptive field simultaneously. Besides, CNN models for vessel segmentation usually are trained by equal pixel level cross-entropy loss, which tend to miss fine vessel structures. In this paper, we propose a novel Context Spatial U-Net (CSU-Net) for blood vessel segmentation. Compared with the other U-Net based models, we design a two-channel encoder: a context channel with multi-scale convolution to capture more receptive field and a spatial channel with large kernel to retain spatial information. Also, to combine and strengthen the features extracted from two paths, we introduce a feature fusion module (FFM) and an attention skip module (ASM). Furthermore, we propose a structure loss, which adds a spatial weight to cross-entropy loss and guide the network to focus more on the thin vessels and boundaries. We evaluated this model on three public datasets: DRIVE, CHASE-DB1 and STARE. The results show that the CSU-Net achieves higher segmentation accuracy than the current state-of-the-art methods.

Electroencephalogram (EEG) has been widely used in emotion recognition due to its high temporal resolution and reliability. Since the individual differences of EEG are large, the emotion recognition models could not be shared across persons, and we need to collect new labeled data to train personal models for new users. In some applications, we hope to acquire models for new persons as fast as possible, and reduce the demand for the labeled data amount. To achieve this goal, we propose a multisource transfer learning method, where existing persons are sources, and the new person is the target. The target data are divided into calibration sessions for training and subsequent sessions for test. The first stage of the method is source selection aimed at locating appropriate sources. The second is style transfer mapping, which reduces the EEG differences between the target and each source. We use few labeled data in the calibration sessions to conduct source selection and style transfer. Finally, we integrate the source models to recognize emotions in the subsequent sessions. The experimental results show that the three-category classification accuracy on benchmark SEED improves by 12.72% comparing with the nontransfer method. Our method facilitates the fast deployment of emotion recognition models by reducing the reliance on the labeled data amount, which has practical significance especially in fast-deployment scenarios.

Existing recommender systems place emphasis on personalization to achieve promising accuracy. However, in the context of multiple domain, users are likely to seek the same behaviors as domain authorities. This conformity effect provides a wealth of prior knowledge when it comes to multi-domain recommendation, but has not been fully exploited. In particular, users whose behaviors are significant similar with the public tastes can be viewed as domain authorities. To detect these users meanwhile embed conformity into recommendation, a domain-specific similarity matrix is intuitively employed. Therefore, a collective similarity is obtained to leverage the conformity with personalization. In this paper, we establish a Collective Structure Sparse Representation(CSSR) method for multi-domain recommendation. Based on adaptive $k$-Nearest-Neighbor framework, we impose the lasso and group lasso penalties as well as least square loss to jointly optimize the collective similarity. Experimental results on real-world data confirm the effectiveness of the proposed method.

Recently, some recommendation methods try to improvethe prediction results by integrating informationfrom user’s multiple types of behaviors. How to modelthe dependence and independence between differentbehaviors is critical for them. In this paper, we proposea novel recommendation model, the Group-Sparse MatrixFactorization (GSMF), which factorizes the ratingmatrices for multiple behaviors into the user and itemlatent factor space with group sparsity regularization.It can (1) select out the different subsets of latent factorsfor different behaviors, addressing that users’ decisionson different behaviors are determined by differentsets of factors;(2) model the dependence and independencebetween behaviors by learning the sharedand private factors for multiple behaviors automatically; (3) allow the shared factors between different behaviorsto be different, instead of all the behaviors sharingthe same set of factors. Experiments on the real-world dataset demonstrate that our model can integrate users’multiple types of behaviors into recommendation better,compared with other state-of-the-arts.

This paper explores the problem of automatically naming faces in TV series or films. A novel method is proposed to build association between the faces in the video and the names in the script by a global sequence alignment algorithm. We firstly build two heterogenous sequences: a face sequence and a name sequence. The elements of the two sequences are cluster labels, computed from the clustering process, and speaking names, respectively. Then the alignment of the two sequences is considered as a problem of surjection between the cluster set and the name set. The optimal solution is obtained by minimizing the Levenshtein Distance between the two sequences which is constrained by the temporal order information. Experiments on public videos demonstrate the effectiveness of our method.

Recommendation for new users is a key challenge due to the lack of prior information from them, which is the well-known cold-start problem. Preference elicitation has been proposed as an efficient strategy for eliciting new users preference through an initial interview where new users are queried by elaborately selected items. In this paper, we propose a novel community discovering guided discriminative selection (CDDS) model for constructing query set. We exploit the community as an effective information which is not fully used in existing approaches. By integrating item selection and community discovery into one framework, our model selects most discriminative items for preference elicitation, with guidance of unsupervised community discovering process. To perform community discovering process, the model utilizes rating similarity graph and social network as a graph regular-ization. Experimental results on real-world datasets Flixster and Douban demonstrate that the proposed method outperforms traditional preference elicitation methods for cold-start recommendation.

Collaborative filtering with implicit feedbacks has been steadily receiving more attention, since the abundant implicit feedbacks are more easily collected while explicit feedbacks are not necessarily always available. Several recent work address this problem well utilizing pairwise ranking method with a fundamental assumption that a user prefers items with positive feedbacks to the items without observed feedbacks, which also implies that the items without observed feedbacks are treated equally without distinction. However, users have their own preference on different items with different degrees which can be modeled into a ranking relationship. In this paper, we exploit this prior information of a user's preference from the nearest neighbor set by the neighbors' implicit feedbacks, which can split items into different item groups with specific ranking relations. We propose a novel PRIGP(Personalized Ranking with Item Group based Pairwise preference learning) algorithm to integrate item based pairwise preference and item group based pairwise preference into the same framework. Experimental results on three real-world datasets demonstrate the proposed method outperforms the competitive baselines on several ranking-oriented evaluation metrics.