Motor imagery-based brain-computer interface (MI-BCI) faces a critical challenge in achieving effective spatial-temporal feature modeling while maintaining a compact model parameterization. Herein, a lightweight model was proposed, termed as Dual-Attention-EEGNet (DA-EEGNet), which extends the EEGNet backbone by integrating a channel attention module and a depth attention module to selectively emphasize informative electrodes and temporally discriminative features. Two widely used MI benchmark datasets and three evaluation strategies, i.e. subject-dependent scenario, subject-independent scenario, and dataset-independent classification scenario, were utilized to verify the model's performance. Despite its compact design, DA-EEGNet contains merely 3.97[Formula: see text]k trainable parameters and achieves average classification accuracies of [Formula: see text] and [Formula: see text], outperforming or matching existing deep learning approaches that rely on substantially larger parameter counts. Ablation studies further confirm the complementary contributions of the channel and depth attention modules. In addition, visualization analyses, including temporal attention heatmaps and motor-area topographies, demonstrate that DA-EEGNet captures neurophysiologically meaningful spatial-temporal patterns consistent with MI-related brain activity. These results indicate that DA-EEGNet provides a favorable parameter-accuracy trade-off and serves as an efficient and interpretable baseline for MI-BCI applications.

In recent years, deep learning based malicious traffic detection (MTD) systems have demonstrated remarkable success. However, their effectiveness tend to decrease because most malicious traffic datasets are suffered from noisy-labeled and long-tailed problems. While numerous approaches have been developed to address these two problems individually, they become inefficient when confronting the combined challenge of noisy long-tailed data, as they typically tackle only a single adverse factor at a time. This paper proposes a two-stage method called Distribution-aware sample Selection and Dynamic Instance-based Relabeling (DSDIR), which simultaneously addresses the impacts of noisy-labeled and long-tailed problems. In the first stage, a noise-independent clean sample selection method is designed to obtain a clean dataset, which converts negative effects of the long-tailed problem into positive ones. In the second stage, dynamic instance-based relabeling is designed to train a model and improve the dataset’s quality simultaneously. Eventually, DSDIR not only produces a balanced and noise-tolerant model but also obtains a clean dataset. Experimental results demonstrate that in the high noise condition of 60% and 80%, the accuracy rate of DSDIR is 5% higher than the state-of-the-art methods. Our code is available at https://github.com/nku-ligl/DSDIR.

Effectively utilizing extensive unlabeled high-density EEG data to improve performance in scenarios with limited labeled low-density EEG data presents a significant challenge. In this paper, we address this challenge by formulating it as a graph transfer learning and knowledge distillation problem. We propose a Unified Pre-trained Graph Contrastive Masked Autoencoder Distiller, named EEG-DisGCMAE, to bridge the gap between unlabeled and labeled as well as high- and low-density EEG data. Our approach introduces a novel unified graph self-supervised pre-training paradigm, which seamlessly integrates the graph contrastive pre-training with the graph masked autoencoder pre-training. Furthermore, we propose a graph topology distillation loss function, allowing a lightweight student model trained on low-density data to learn from a teacher model trained on high-density data during pre-training and fine-tuning. This method effectively handles missing electrodes through contrastive distillation. We validate the effectiveness of EEG-DisGCMAE across four classification tasks using two clinical EEG datasets with abundant data.

Electroencephalography (EEG) source imaging (ESI) methods aim to reconstruct cortical sources from scalp EEG signals, a crucial task for understanding the normal brain as well as brain disorders. Traditional model-driven ESI methods face challenges in real-time reconstruction, while deep neural network (DNN)-based ESI methods often struggle with generalization to new data. To address these issues, we propose ADMM-ESINet, a novel deep unfolding neural network for robust and efficient reconstruction of EEG extended sources. ADMM-ESINet leverages a structured sparsity constraint within a regularization framework and employs the Alternating Direction Method of Multipliers (ADMM) to achieve iterative solutions. By unrolling the ADMM algorithm into a cascaded network architecture, ADMM-ESINet effectively integrates prior knowledge, enabling end-to-end, real-time ESI. Crucially, both the regularization parameters and the spatial transform operator are learned directly from the training data. Numerical results demonstrate that ADMM-ESINet surpasses traditional DNN-based methods in generalization ability and accurately reconstructs the location, extent, and temporal dynamics of extended sources, establishing ADMM-ESINet as a promising method for real-time ESI.

The interconnection between brain regions in neurological disease encodes vital information for the advancement of biomarkers and diagnostics. Although graph convolutional networks are widely applied for discovering brain connection patterns that point to disease conditions, the potential of connection patterns that arise from multiple imaging modalities has yet to be fully realized. In this paper, we propose a multi-modal sparse interpretable GCN framework (SGCN) for the detection of Alzheimer's disease (AD) and its prodromal stage, known as mild cognitive impairment (MCI). In our experimentation, SGCN learned the sparse regional importance probability to find signature regions of interest (ROIs), and the connective importance probability to reveal disease-specific brain network connections. We evaluated SGCN on the Alzheimer's Disease Neuroimaging Initiative database with multi-modal brain images and demonstrated that the ROI features learned by SGCN were effective for enhancing AD status identification. The identified abnormalities were significantly correlated with AD-related clinical symptoms. We further interpreted the identified brain dysfunctions at the level of large-scale neural systems and sex-related connectivity abnormalities in AD/MCI. The salient ROIs and the prominent brain connectivity abnormalities interpreted by SGCN are considerably important for developing novel biomarkers. These findings contribute to a better understanding of the network-based disorder via multi-modal diagnosis and offer the potential for precision diagnostics. The source code is available at https://github.com/Houliang-Zhou/SGCN.

This manuscript presents a novel approach for decoding pre-movement patterns from brain signals using a two-stage-training temporal–spectral neural network (TTSNet). The TTSNet employs a combination of filter bank task-related component analysis (FBTRCA) and convolutional neural network (CNN) techniques to enhance the classification of single-upper limb movements in non-invasive brain–computer interfaces (BCIs). In our previous work, we introduced the FBTRCA method which utilized filter banks and spatial filters to handle spectral and spatial information, respectively. However, we observed limitations in the temporal decoding phase, where correlation features failed to effectively utilize temporal information because of misaligned onset and noisy spikes. To address this issue, our proposed method focuses on analyzing multi-channel signals in the temporal–spectral domain. The TTSNet first divides the signals into various filter banks, employing task-related component analysis to reduce dimensionality and eliminate noise, respectively. Subsequently, a CNN is employed to optimize the temporal characteristics of the signals and extract class-related features. Finally, the class-related features from all filter banks are concatenated and classified using the fully connected layer. To evaluate the effectiveness of our proposed method, we conducted experiments on two publicly available datasets. In binary classification tasks, the TTSNet achieved an improved accuracy of 0.7707 ± 0.1168, surpassing the performance of EEGNet (accuracy: 0.7340 ± 0.1246) and FBTRCA (accuracy: 0.7487 ± 0.1250). In multi-class tasks, TTSNet achieved an accuracy of 0.4588 ± 0.0724, exhibiting a 4.27% and 3.95% accuracy increase over EEGNet and FBTRCA, respectively. The findings of this study suggest that the proposed TTSNet method holds promise for detecting limb movements and assisting in the rehabilitation of stroke patients. The classification of single-side limb movements is expected to facilitate the interaction between patients and external environment by increasing the number of control commands in BCIs.

Deep reinforcement learning (DRL) has become prevalent for decision-making task assignments in mobile crowdsensing (MCS). However, when facing sensing scenarios with varying numbers of workers or task attributes, existing DRL-based task assignment schemes fail to generate matching policies continuously and are susceptible to environmental fluctuations. To overcome these issues, a twin-delayed deep stochastic policy gradient (TDDS) approach is presented for balanced and low-latency MCS task decomposition and parallel subtask allocation. A masked attention mechanism is incorporated into the policy network to enable TDDS to adapt to task-attribute and subtask variations. To enhance environmental adaptability, an off-policy DRL algorithm incorporating experience replay is developed to eliminate sample correlation during training. Gumbel-Softmax sampling is integrated into the twin-delayed deep deterministic policy gradient (TD3) to support discrete action space decisions and a customized reward strategy to reduce task completion delay and balance workloads. Extensive simulation results confirm that the proposed scheme outperforms mainstream DRL baselines in terms of environmental adaptability, task completion delay, and workload balancing.

The emerging matrix learning methods have achieved promising performances in electroencephalogram (EEG) classification by exploiting the structural information between the columns or rows of feature matrices. Due to the intersubject variability of EEG data, these methods generally need to collect a large amount of labeled individual EEG data, which would cause fatigue and inconvenience to the subjects. Insufficient subject-specific EEG data will weaken the generalization capability of the matrix learning methods in neural pattern decoding. To overcome this dilemma, we propose an adaptive multimodel knowledge transfer matrix machine (AMK-TMM), which can selectively leverage model knowledge from multiple source subjects and capture the structural information of the corresponding EEG feature matrices. Specifically, by incorporating least-squares (LS) loss with spectral elastic net regularization, we first present an LS support matrix machine (LS-SMM) to model the EEG feature matrices. To boost the generalization capability of LS-SMM in scenarios with limited EEG data, we then propose a multimodel adaption method, which can adaptively choose multiple correlated source model knowledge with a leave-one-out cross-validation strategy on the available target training data. We extensively evaluate our method on three independent EEG datasets. Experimental results demonstrate that our method achieves promising performances on EEG classification.

Integrating brain images and genetic data provides a great opportunity to discover potential biomarkers for neurological disorder diagnosis. However, learning genetic information and brain network dysfunction remains a challenging task. In this paper, we propose an interpretable multi-modal imaging and genetic graph convolution network (GCN) for Alzheimer’s disease diagnosis. Our genetic network uses hierarchical GCN to mimic a gene ontology-based graph of biological processes and learn the information flow in this graph. In parallel, our imaging network uses a sparse interpretable GCN with node and edge importance probabilities to learn the brain network from multi-modal images. After multi-modal fusion, the final representation guided by a cluster-based consistency constraint is used to predict the disease-related clinical measures. We evaluate our method on the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. Our result shows that our imaging-genetics framework achieves superior prediction performance compared to all state-of-the-art methods. The interpretation demonstrated that the salient SNPs, and salient regions interpreted by important probabilities were significantly correlated with AD-related clinical symptoms, and considerably important for developing novel biomarkers. The code is available at https://github.com/Houliang-Zhou/IG-GCN.

High annotation costs serve as a significant hurdle in deploying modern deep learning architectures for clinically relevant medical applications, especially when dealing with the inherent heterogeneity of multimodal data, proving the critical need for innovative algorithms that can effectively utilize unlabeled data. In this paper, we propose a model named MCLCA, which integrates multimodal contrastive learning and cross-modal attention to diagnose Alzheimer’s Disease (AD) and identify biomarkers using both labeled and unlabeled multimodal brain imaging genetics data. Through multimodal contrastive learning, MCLCA can effectively learn representations even in the absence of sufficient labels. By utilizing cross-modal attention blocks, the model captures deep connections between different modalities, providing a more comprehensive view of diagnosis. Our proposed MCLCA model is evaluated using the ADNI database with three imaging modalities (VBM-MRI, FDG-PET, and AV45-PET) and genetic SNP data. The results demonstrate that MCLCA can identify important biomarkers with better prediction accuracy compared to the existing methods. The source code is available at https://github.com/MCLCA.

With sufficient centralized training data coming from multiple subjects, deep learning methods have achieved powerful EEG decoding performance. However, sending each individuals’ EEG data directly to a centralized server might cause privacy leakage. To overcome this issue, we present an inter-subject structure matching-based federated EEG decoding (FedEEG) framework. First, we introduce a center loss to each client (subject), which can learn multiple virtual class centers by averaging the corresponding class-specific EEG features. To mitigate the client drift issue, we then explicitly connect the learning across multiple clients by aligning their corresponding virtual class centers, thus helping to correct the local training for individual subject. The proposed FedEEG can promote the discriminative feature learning while preventing the privacy leakage issue. The experimental results on benchmark EEG datasets show that FedEEG outperforms state-of-the-art federated learning methods.

The protection of model intellectual property is becoming an increasingly important issue. However, the existing methods for protecting model ownership, although effective, have limitations. Firstly, they primarily focus on classification models, and secondly, most of the proposed methods reduce the model's utility. To overcome these shortcomings, this paper proposes a task-agnostic model ownership verification framework based on feature fingerprint, called TMOVF, which separates ownership verification from model task. Our key idea is that model knowledge can be uniquely characterized by the extracted features, which may be high-dimensional, complicated, and difficult to compare for each input sample. Nevertheless, these features contain inherent information that cannot be ignored in cases of piracy. To measure the inheritance of our fingerprint, we introduce outlier detection into model ownership verification, which is a first in the field. By reconstructing the outlier detection algorithm, we extract the feature fingerprints of the victim model and the suspicious model, and compute the outliers of their feature fingerprints. By comparing the results, we can verify the ownership of the models. We conduct extensive experiments to evaluate our framework and demonstrate the inheritability of feature fingerprints in stolen models. Our experiments show that the framework is effective in verifying ownership, regardless of the model task. Additionally, our results demonstrate that our framework is more effective than existing methods.

Decoding brain activity from non-invasive electroencephalography (EEG) is crucial for brain-computer interfaces (BCIs) and the study of brain disorders. Notably, end-to-end EEG decoding has gained widespread popularity in recent years owing to the remarkable advances in deep learning research. However, many EEG studies suffer from limited sample sizes, making it difficult for existing deep learning models to effectively generalize to highly noisy EEG data. To address this fundamental limitation, this paper proposes a novel end-to-end EEG decoding algorithm that utilizes a low-rank weight matrix to encode both spatio-temporal filters and the classifier, all optimized under a principled sparse Bayesian learning (SBL) framework. Importantly, this SBL framework also enables us to learn hyperparameters that optimally penalize the model in a Bayesian fashion. The proposed decoding algorithm is systematically benchmarked on five motor imagery BCI EEG datasets ( N=192) and an emotion recognition EEG dataset ( N=45), in comparison with several contemporary algorithms, including end-to-end deep-learning-based EEG decoding algorithms. The classification results demonstrate that our algorithm significantly outperforms the competing algorithms while yielding neurophysiologically meaningful spatio-temporal patterns. Our algorithm therefore advances the state-of-the-art by providing a novel EEG-tailored machine learning tool for decoding brain activity.

Great progress has been made in diagnosing medical diseases based on deep learning. Large-scale medical data are expected to improve deep learning performance further. It is almost impossible for a single institution to collect so much data due to the time-consuming and costly collection and labeling of medical data. Many studies have turned attention to data sharing among multiple medical institutions. However, due to different data acquiring and processing procedures, multiple institutions' medical data is characterized by distribution heterogeneity. Besides, the protection of patient privacy in medical data sharing has also been a common concern. To simultaneously address the problems of heterogeneous data distribution and privacy protection, we propose a novel multi-source source free domain adaptation. When aligning distributed heterogeneous data, our method only require to transfer the pre-trained source models rather than the direct source domain data, thus protecting patients' privacy. In addition, it has the advantages of being efficient and less costly in network resources. The proposed method is evaluated on the multi-site fMRI database Autism Brain Imaging Data Exchange (ABIDE) and yields an average accuracy of 69.37%. We also analyzed its effectiveness on network resource-saving and conducted additional experiments on Camelyon17 to validate the generalization.

The classification of limb movements can provide with control commands in non-invasive brain-computer interface. Previous studies on the classification of limb movements have focused on the classification of left/right limbs; however, the classification of different types of upper limb movements has often been ignored despite that it provides more active-evoked control commands in the brain-computer interface. Nevertheless, few machine learning method can be used as the state-of-the-art method in the multi-class classification of limb movements. This work focuses on the multi-class classification of upper limb movements and proposes the multi-class filter bank task-related component analysis (mFBTRCA) method, which consists of three steps: spatial filtering, similarity measuring and filter bank selection. The spatial filter, namely the task-related component analysis, is first used to remove noise from EEG signals. The canonical correlation measures the similarity of the spatial-filtered signals and is used for feature extraction. The correlation features are extracted from multiple low-frequency filter banks. The minimum-redundancy maximum-relevance selects the essential features from all the correlation features, and finally, the support vector machine is used to classify the selected features. The proposed method compared against previously used models is evaluated using two datasets. mFBTRCA achieved a classification accuracy of 0.4193 ± 0.0780 (7 classes) and 0.4032 ± 0.0714 (5 classes), respectively, which improves on the best accuracies achieved using the compared methods (0.3590 ± 0.0645 and 0.3159 ± 0.0736, respectively). The proposed method is expected to provide more control commands in the applications of non-invasive brain-computer interfaces.

Improving sample efficiency has been a longstanding goal in reinforcement learning. This paper proposes VRMPO algorithm: a sample efficient policy gradient method with stochastic mirror descent. In VRMPO, a novel variance-reduced policy gradient estimator is presented to improve sample efficiency. We prove that the proposed VRMPO needs only O(ε−3) sample trajectories to achieve an ε-approximate first-order stationary point, which matches the best sample complexity for policy optimization. Extensive empirical results demonstrate that VRMP outperforms the state-of-the-art policy gradient methods in various settings.

Functional connectivity (FC) networks built from resting-state functional magnetic resonance imaging (rs-fMRI) has shown promising results for the diagnosis of Alzheimer's disease and its prodromal stage, that is, mild cognitive impairment (MCI). FC is usually estimated as a temporal correlation of regional mean rs-fMRI signals between any pair of brain regions, and these regions are traditionally parcellated with a particular brain atlas. Most existing studies have adopted a predefined brain atlas for all subjects. However, the constructed FC networks inevitably ignore the potentially important subject-specific information, particularly, the subject-specific brain parcellation. Similar to the drawback of the "single view" (versus the "multiview" learning) in medical image-based classification, FC networks constructed based on a single atlas may not be sufficient to reveal the underlying complicated differences between normal controls and disease-affected patients due to the potential bias from that particular atlas. In this study, we propose a multiview feature learning method with multiatlas-based FC networks to improve MCI diagnosis. Specifically, a three-step transformation is implemented to generate multiple individually specified atlases from the standard automated anatomical labeling template, from which a set of atlas exemplars is selected. Multiple FC networks are constructed based on these preselected atlas exemplars, providing multiple views of the FC network-based feature representations for each subject. We then devise a multitask learning algorithm for joint feature selection from the constructed multiple FC networks. The selected features are jointly fed into a support vector machine classifier for multiatlas-based MCI diagnosis. Extensive experimental comparisons are carried out between the proposed method and other competing approaches, including the traditional single-atlas-based method. The results indicate that our method significantly improves the MCI classification, demonstrating its promise in the brain connectome-based individualized diagnosis of brain diseases.

Non-negative Tucker decomposition (NTD) is one of the most popular techniques for tensor data representation. To enhance the representation ability of NTD by multiple intrinsic cues, that is, manifold structure and supervisory information, in this article, we propose a generalized graph regularized NTD (GNTD) framework for tensor data representation. We first develop the unsupervised GNTD (UGNTD) method by constructing the nearest neighbor graph to maintain the intrinsic manifold structure of tensor data. Then, when limited must-link and cannot-link constraints are given, unlike most existing semisupervised learning methods that only use the pregiven supervisory information, we propagate the constraints through the entire dataset and then build a semisupervised graph weight matrix by which we can formulate the semisupervised GNTD (SGNTD). Moreover, we develop a fast and efficient alternating proximal gradient-based algorithm to solve the optimization problem and show its convergence and correctness. The experimental results on unsupervised and semisupervised clustering tasks using four image datasets demonstrate the effectiveness and high efficiency of the proposed methods.

As a kind of biological network, the brain network conduces to understanding the mystery of high-efficiency information processing in the brain, which will provide instructions to develop efficient brain-like neural networks. Large-scale dynamical functional network connectivity (dFNC) provides a more context-sensitive, dynamical, and straightforward sight at a higher network level. Nevertheless, dFNC analysis needs good enough resolution in both temporal and spatial domains, and the construction of dFNC needs to capture the time-varying correlations between two multivariate time series with unmatched spatial dimensions. Effective methods still lack. With well-developed source imaging techniques, electroencephalogram (EEG) has the potential to possess both high temporal and spatial resolutions. Therefore, we proposed to construct the EEG large-scale cortical dFNC based on brain atlas to probe the subtle dynamic activities in the brain and developed a novel method, that is, wavelet coherence-S estimator (WTCS), to assess the dynamic couplings among functional subnetworks with different spatial dimensions. The simulation study demonstrated its robustness and availability of applying to dFNC. The application in real EEG data revealed the appealing "Primary peak" and "P3-like peak" in dFNC network properties and meaningful evolutions in dFNC network topology for P300. Our study brings new insights for probing brain activities at a more dynamical and higher hierarchical level and pushing forward the development of brain-inspired artificial neural networks. The proposed WTCS not only benefits the dFNC studies but also gives a new solution to capture the time-varying couplings between the multivariate time series that is often encountered in signal processing disciplines.

Camouflaged object detection (COD) aims to detect/segment camouflaged objects embedded in the environment, which has attracted increasing attention over the past decades. Although several COD methods have been developed, they still suffer from unsatisfactory performance due to the intrinsic similarities between the foreground objects and background surroundings. In this paper, we propose a novel Feature Aggregation and Propagation Network (FAP-Net) for camouflaged object detection. Specifically, we propose a Boundary Guidance Module (BGM) to explicitly model the boundary characteristic, which can provide boundary-enhanced features to boost the COD performance. To capture the scale variations of the camouflaged objects, we propose a Multi-scale Feature Aggregation Module (MFAM) to characterize the multi-scale information from each layer and obtain the aggregated feature representations. Furthermore, we propose a Cross-level Fusion and Propagation Module (CFPM). In the CFPM, the feature fusion part can effectively integrate the features from adjacent layers to exploit the cross-level correlations, and the feature propagation part can transmit valuable context information from the encoder to the decoder network via a gate unit. Finally, we formulate a unified and end-to-end trainable framework where cross-level features can be effectively fused and propagated for capturing rich context information. Extensive experiments on three benchmark camouflaged datasets demonstrate that our FAP-Net outperforms other state-of-the-art COD models. Moreover, our model can be extended to the polyp segmentation task, and the comparison results further validate the effectiveness of the proposed model in segmenting polyps. The source code and results will be released at https://github.com/taozh2017/FAPNet.

Accurate electroencephalogram (EEG) pattern decoding for specific mental tasks is one of the key steps for the development of brain-computer interface (BCI), which is quite challenging due to the considerably low signal-to-noise ratio of EEG collected at the brain scalp. Machine learning provides a promising technique to optimize EEG patterns toward better decoding accuracy. However, existing algorithms do not effectively explore the underlying data structure capturing the true EEG sample distribution and, hence, can only yield a suboptimal decoding accuracy. To uncover the intrinsic distribution structure of EEG data, we propose a clustering-based multitask feature learning algorithm for improved EEG pattern decoding. Specifically, we perform affinity propagation-based clustering to explore the subclasses (i.e., clusters) in each of the original classes and then assign each subclass a unique label based on a one-versus-all encoding strategy. With the encoded label matrix, we devise a novel multitask learning algorithm by exploiting the subclass relationship to jointly optimize the EEG pattern features from the uncovered subclasses. We then train a linear support vector machine with the optimized features for EEG pattern decoding. Extensive experimental studies are conducted on three EEG data sets to validate the effectiveness of our algorithm in comparison with other state-of-the-art approaches. The improved experimental results demonstrate the outstanding superiority of our algorithm, suggesting its prominent performance for EEG pattern decoding in BCI applications.

We study the convergence of Expected Sarsa(λ) with function approximation. We show that with off-line es- timate (multi-step bootstrapping) to ExpectedSarsa(λ) is unstable for off-policy learning. Furthermore, based on convex-concave saddle-point framework, we propose a con- vergent Gradient Expected Sarsa(λ) (GES(λ)) algorithm. The theoretical analysis shows that the proposed GES(λ) converges to the optimal solution at a linear convergence rate under true gradient setting. Furthermore, we develop a Lyapunov function technique to investigate how the step- size influences finite-time performance of GES(λ). Addition- ally, such a technique of Lyapunov function can be poten- tially generalized to other gradient temporal difference algo- rithms. Finally, our experiments verify the effectiveness of our GES(λ). For the details of proof, please refer to https: //arxiv.org/pdf/2012.07199.pdf.

Three-dimensional scanning by means of structured light illumination is an active imaging technique involving projecting and capturing a series of striped patterns and then using the observed warping of stripes to reconstruct the target object’s surface through triangulating each pixel in the camera to a unique projector coordinate corresponding to a particular feature in the projected patterns. The undesirable phenomenon of multi-path occurs when a camera pixel simultaneously sees features from multiple projector coordinates. Bimodal multi-path is a particularly common situation found along step edges, where the camera pixel sees both a foreground and background surface. Generalized from bimodal multi-path, this paper examines the phenomenon of sparse or N-modal multi-path as a more general case, where the camera pixel sees no fewer than two reflective surfaces, resulting in decoding errors. Using fringe projection profilometry, our proposed solution is to treat each camera pixel as an underdetermined linear system of equations and to find the sparsest (least number of paths) solution by taking an application-specific Bayesian learning approach. We validate this algorithm with both simulations and a number of challenging real-world scenarios, demonstrating that it outperforms state-of-the-art techniques.

How to learn dimension-reduced representations of image data for clustering has been attracting much attention. Motivated by that the clustering accuracy is affected by both the prior-known label information of some of the images and the sparsity feature of the representations, we propose a non-negative matrix factorization (NMF) method with dual constraints in this paper. In our model, one constraint is used to keep the label feature and the other constraint is utilized to enhance the sparsity of the representations. Notably that these two constraints are embedded naturally into the traditional NMF model, refraining from the usage of the balance parameters which are hard to choose. Meantime, for solving the proposed model, the alternative iteration scheme is employed, and an efficient algorithm based on convex optimization is designed to conduct each iteration operation. It is proved that this algorithm achieves a nonlinear convergence rate, much faster than existing methods with linear rate. Simulation results demonstrate the advantages of the proposed method.

Nonnegative Tucker Decomposition (NTD) is one of the most popular technique for feature extraction and representation from nonnegative tensor data with preserving internal structure information. From the perspective of geometry, highdimensional data are usually drawn in low-dimensional submanifold of the ambient space. In this paper, we propose a novel Graph reguralized Nonnegative Tucker Decomposition (GNTD) method which is able to extract the low-dimensional parts-based representation and preserve the geometrical information simultaneously from high-dimensional tensor data. We also present an effictive algorithm to solve the proposed GNTD model. Experimental results demonstrate the effectiveness and high efficiency of the proposed GNTD method.

Event-related potentials (ERPs) especially P300 are popular effective features for brain-computer interface (BCI) systems based on electroencephalography (EEG). Traditional ERP-based BCI systems may perform poorly for small training samples, i.e. the undersampling problem. In this study, the ERP classification problem was investigated, in particular, the ERP classification in the high-dimensional setting with the number of features larger than the number of samples was studied. A flexible group sparse discriminative analysis algorithm based on Moreau-Yosida regularization was proposed for alleviating the undersampling problem. An optimization problem with the group sparse criterion was presented, and the optimal solution was proposed by using the regularized optimal scoring method. During the alternating iteration procedure, the feature selection and classification were performed simultaneously. Two P300-based BCI datasets were used to evaluate our proposed new method and compare it with existing standard methods. The experimental results indicated that the features extracted via our proposed method are efficient and provide an overall better P300 classification accuracy compared with several state-of-the-art methods.

With the rapid development of the Internet, social media is affecting and changing people’s lives. The research of network community structure based on a large number of complex network data sets is increasingly popular. Due to the large scale of existing social network data and privacy issues, it is hard to analyze the entire network data directly. Therefore, a reliable and effective network sampling method is very important for the actual estimation of online social networks attributes. Existing network sampling methods like metropolis–hasting random walk (MHRW) can obtain unbiased sample sets from relatively large-scale social networks such as Facebook and describe the key features of the original network. Moreover, MHRW uses a proposed distribution function for sampling control, which can guarantee a well-balanced nature of resulting Markov chain. However, MHRW has the defect of partial graph over entry. In this paper, we proposed a new hybrid jump (HJ) sample by introducing an HJ strategy into MHRW during the sampling progress. First, we use a breadth-first search to obtain a data set without repeated node quickly from a list of jump nodes. Moreover, we adopted uniform sample (UNI) to get the average degree of the original network. Then, a 3-D average degree distribution model is designed to determine the optimal value of the jump parameter in HJ. Finally, we set the condition to execute the HJ strategy in each step of sampling progress. The experimental results demonstrate the performance of HJ is better than the other representation sampling methods both in strong-tie networks and weak-tie networks.

Common spatial pattern (CSP)-based spatial filtering has been most popularly applied to electroencephalogram (EEG) feature extraction for motor imagery (MI) classification in brain-computer interface (BCI) application. The effectiveness of CSP is highly affected by the frequency band and time window of EEG segments. Although numerous algorithms have been designed to optimize the spectral bands of CSP, most of them selected the time window in a heuristic way. This is likely to result in a suboptimal feature extraction since the time period when the brain responses to the mental tasks occurs may not be accurately detected. In this paper, we propose a novel algorithm, namely temporally constrained sparse group spatial pattern (TSGSP), for the simultaneous optimization of filter bands and time window within CSP to further boost classification accuracy of MI EEG. Specifically, spectrum-specific signals are first derived by bandpass filtering from raw EEG data at a set of overlapping filter bands. Each of the spectrum-specific signals is further segmented into multiple subseries using sliding window approach. We then devise a joint sparse optimization of filter bands and time windows with temporal smoothness constraint to extract robust CSP features under a multitask learning framework. A linear support vector machine classifier is trained on the optimized EEG features to accurately identify the MI tasks. An experimental study is implemented on three public EEG datasets (BCI Competition III dataset IIIa, BCI Competition IV datasets IIa, and BCI Competition IV dataset IIb) to validate the effectiveness of TSGSP in comparison to several other competing methods. Superior classification performance (averaged accuracies are 88.5%, 83.3%, and 84.3% for the three datasets, respectively) based on the experimental results confirms that the proposed algorithm is a promising candidate for performance improvement of MI-based BCIs.

A potential limitation of a motor imagery (MI) based brain-computer interface (BCI) is that it usually requires a relatively long time to record sufficient electroencephalogram (EEG) data for robust classifier training. The calibration burden during data acquisition phase will most probably cause a subject to be reluctant to use a BCI system. To alleviate this issue, we propose a novel sparse group representation model (SGRM) for improving the efficiency of MI-based BCI by exploiting the intersubject information. Specifically, preceded by feature extraction using common spatial pattern, a composite dictionary matrix is constructed with training samples from both the target subject and other subjects. By explicitly exploiting within-group sparse and group-wise sparse constraints, the most compact representation of a test sample of the target subject is then estimated as a linear combination of columns in the dictionary matrix. Classification is implemented by calculating the class-specific representation residual based on the significant training samples corresponding to the nonzero representation coefficients. Accordingly, the proposed SGRM method effectively reduces the required training samples from the target subject due to auxiliary data available from other subjects. With two public EEG data sets, extensive experimental comparisons are carried out between SGRM and other state-of-the-art approaches. Superior classification performance of our method using 40 trials of the target subject for model calibration (Averaged accuracy = 78.2%, Kappa = 0.57 and Averaged accuracy = 77.7%, Kappa = 0.55 for the two data sets, respectively) indicates its promising potential for improving the practicality of MI-based BCI.

Multiset canonical correlation analysis (MsetCCA) has been successfully applied to optimize the reference signals by extracting common features from multiple sets of electroencephalogram (EEG) for steady-state visual evoked potential (SSVEP) recognition in brain-computer interface application. To avoid extracting the possible noise components as common features, this study proposes a sophisticated extension of MsetCCA, called multilayer correlation maximization (MCM) model for further improving SSVEP recognition accuracy. MCM combines advantages of both CCA and MsetCCA by carrying out three layers of correlation maximization processes. The first layer is to extract the stimulus frequency-related information in using CCA between EEG samples and sine-cosine reference signals. The second layer is to learn reference signals by extracting the common features with MsetCCA. The third layer is to re-optimize the reference signals set in using CCA with sine-cosine reference signals again. Experimental study is implemented to validate effectiveness of the proposed MCM model in comparison with the standard CCA and MsetCCA algorithms. Superior performance of MCM demonstrates its promising potential for the development of an improved SSVEP-based brain-computer interface.

Effective common spatial pattern (CSP) feature extraction for motor imagery (MI) electroencephalogram (EEG) recordings usually depends on the filter band selection to a large extent. Subband optimization has been suggested to enhance classification accuracy of MI. Accordingly, this study introduces a new method that implements sparse Bayesian learning of frequency bands (named SBLFB) from EEG for MI classification. CSP features are extracted on a set of signals that are generated by a filter bank with multiple overlapping subbands from raw EEG data. Sparse Bayesian learning is then exploited to implement selection of significant features with a linear discriminant criterion for classification. The effectiveness of SBLFB is demonstrated on the BCI Competition IV IIb dataset, in comparison with several other competing methods. Experimental results indicate that the SBLFB method is promising for development of an effective classifier to improve MI classification.

High accuracy of electroencephalogram (EEG) classification can hardly be achieved if the signals are contaminated by severe artefacts. One helpless way to avoid such artefacts is usually to directly discard the severely disturbed EEG segments. This study considers a more elegant way that tries to recover the disturbed segments from other undisturbed segments. The possible artefacts in EEG are treated as missing values. A Bayesian tensor factorization (BTF) based method is proposed to implement EEG completion for artefact removal. By specifying a sparsity-inducing hierarchical prior, the underlying low-rank tensor is discovered from incomplete EEG tensor with automatically inferred model parameters. The EEG missing values are effectively predicted with robustness to overfitting. Effectiveness of the BTF algorithm is demonstrated on EEG data recorded from seven subjects in a brain-computer interface paradigm based on event-related potentials.

With the increasing availability of various sensor technologies, we now have access to large amounts of multiblock (also called multiset, multirelational, or multiview) data that need to be jointly analyzed to explore their latent connections. Various component analysis methods have played an increasingly important role for the analysis of such coupled data. In this article, we first provide a brief review of existing matrix-based (two-way) component analysis methods for the joint analysis of such data with a focus on biomedical applications. Then, we discuss their important extensions and generalization to multiblock multiway (tensor) data. We show how constrained multiblock tensor decomposition methods are able to extract similar or statistically dependent common features that are shared by all blocks, by incorporating the multiway nature of data. Special emphasis is given to the flexible common and individual feature analysis of multiblock data with the aim to simultaneously extract common and individual latent components with desired properties and types of diversity. Illustrative examples are given to demonstrate their effectiveness for biomedical data analysis.

Regularization has been one of the most popular approaches to prevent overfitting in electroencephalogram (EEG) classification of brain-computer interfaces (BCIs). The effectiveness of regularization is often highly dependent on the selection of regularization parameters that are typically determined by cross-validation (CV). However, the CV imposes two main limitations on BCIs: 1) a large amount of training data is required from the user and 2) it takes a relatively long time to calibrate the classifier. These limitations substantially deteriorate the system's practicability and may cause a user to be reluctant to use BCIs. In this paper, we introduce a sparse Bayesian method by exploiting Laplace priors, namely, SBLaplace, for EEG classification. A sparse discriminant vector is learned with a Laplace prior in a hierarchical fashion under a Bayesian evidence framework. All required model parameters are automatically estimated from training data without the need of CV. Extensive comparisons are carried out between the SBLaplace algorithm and several other competing methods based on two EEG data sets. The experimental results demonstrate that the SBLaplace algorithm achieves better overall performance than the competing algorithms for EEG classification.

Real-world data are often acquired as a collection of matrices rather than as a single matrix. Such multiblock data are naturally linked and typically share some common features while at the same time exhibiting their own individual features, reflecting the underlying data generation mechanisms. To exploit the linked nature of data, we propose a new framework for common and individual feature extraction (CIFE) which identifies and separates the common and individual features from the multiblock data. Two efficient algorithms termed common orthogonal basis extraction (COBE) are proposed to extract common basis is shared by all data, independent on whether the number of common components is known beforehand. Feature extraction is then performed on the common and individual subspaces separately, by incorporating dimensionality reduction and blind source separation techniques. Comprehensive experimental results on both the synthetic and real-world data demonstrate significant advantages of the proposed CIFE method in comparison with the state-of-the-art.

The P300-based brain-computer interface (BCI) is an extension of the oddball paradigm, and can facilitate communication for people with severe neuromuscular disorders. It has been shown that, in addition to the P300, other event-related potential (ERP) components have been shown to contribute to successful operation of the P300 BCI. Incorporating these components into the classification algorithm can improve the classification accuracy and information transfer rate (ITR). In this paper, a single character presentation paradigm was compared to a presentation paradigm that is based on the visual mismatch negativity. The mismatch negativity paradigm showed significantly higher classification accuracy and ITRs than a single character presentation paradigm. In addition, the mismatch paradigm elicited larger N200 and N400 components than the single character paradigm. The components elicited by the presentation method were consistent with what would be expected from a mismatch paradigm and a typical P300 was also observed. The results show that increasing the signal-to-noise ratio by increasing the amplitude of ERP components can significantly improve BCI speed and accuracy. The mismatch presentation paradigm may be considered a viable option to the traditional P300 BCI paradigm.

Recent research has shown that a new face paradigm is superior to the conventional "flash only" approach that has dominated P300 brain-computer interfaces (BCIs) for over 20 years. However, these face paradigms did not study the repetition effects and the stability of evoked event related potentials (ERPs), which would decrease the performance of P300 BCI. In this paper, we explored whether a new "multi-faces (MF)" approach would yield more distinct ERPs than the conventional "single face (SF)" approach. To decrease the repetition effects and evoke large ERPs, we introduced a new stimulus approach called the "MF" approach, which shows different familiar faces randomly. Fifteen subjects participated in runs using this new approach and an established "SF" approach. The result showed that the MF pattern enlarged the N200 and N400 components, evoked stable P300 and N400, and yielded better BCI performance than the SF pattern. The MF pattern can evoke larger N200 and N400 components and more stable P300 and N400, which increase the classification accuracy compared to the face pattern.

Canonical correlation analysis (CCA) has been one of the most popular methods for frequency recognition in steady-state visual evoked potential (SSVEP)-based brain-computer interfaces (BCIs). Despite its efficiency, a potential problem is that using pre-constructed sine-cosine waves as the required reference signals in the CCA method often does not result in the optimal recognition accuracy due to their lack of features from the real electro-encephalo-gram (EEG) data. To address this problem, this study proposes a novel method based on multiset canonical correlation analysis (MsetCCA) to optimize the reference signals used in the CCA method for SSVEP frequency recognition. The MsetCCA method learns multiple linear transforms that implement joint spatial filtering to maximize the overall correlation among canonical variates, and hence extracts SSVEP common features from multiple sets of EEG data recorded at the same stimulus frequency. The optimized reference signals are formed by combination of the common features and completely based on training data. Experimental study with EEG data from 10 healthy subjects demonstrates that the MsetCCA method improves the recognition accuracy of SSVEP frequency in comparison with the CCA method and other two competing methods (multiway CCA (MwayCCA) and phase constrained CCA (PCCA)), especially for a small number of channels and a short time window length. The superiority indicates that the proposed MsetCCA method is a new promising candidate for frequency recognition in SSVEP-based BCIs.

Two main issues for event-related potential (ERP) classification in brain-computer interface (BCI) application are curse-of-dimensionality and bias-variance tradeoff, which may deteriorate classification performance, especially with insufficient training samples resulted from limited calibration time. This study introduces an aggregation of sparse linear discriminant analyses (ASLDA) to overcome these problems. In the ASLDA, multiple sparse discriminant vectors are learned from differently l1-regularized least-squares regressions by exploiting the equivalence between LDA and least-squares regression, and are subsequently aggregated to form an ensemble classifier, which could not only implement automatic feature selection for dimensionality reduction to alleviate curse-of-dimensionality, but also decrease the variance to improve generalization capacity for new test samples. Extensive investigation and comparison are carried out among the ASLDA, the ordinary LDA and other competing ERP classification algorithms, based on different three ERP datasets. Experimental results indicate that the ASLDA yields better overall performance for single-trial ERP classification when insufficient training samples are available. This suggests the proposed ASLDA is promising for ERP classification in small sample size scenario to improve the practicability of BCI.

Electrooculogram (EOG) signals are potential responses generated by eye movements, and event related potential (ERP) is a special electroencephalogram (EEG) pattern which evoked by external stimuli. Both EOG and ERP have been used separately for implementing human-machine interfaces which can assist disabled patients in performing daily tasks. In this paper, we present a novel EOG/ERP hybrid human-machine interface which integrates the traditional EOG and ERP interfaces together. Eye movements like the blink, wink, gaze, and frown are detected from EOG signals using double threshold algorithm. Multiple ERP components, i.e., N170, VPP and P300 are evoked by inverted face stimuli and classified by linear discriminant analysis (LDA). Based on this hybrid interface, we also design a control scheme for the humanoid robot NAO (Aldebaran robotics, Inc). On-line experiment results show that the proposed hybrid interface can effectively control the robot's basic movements and order it to make various behaviors. While normally operating the robot by hands takes 49.1 s to complete the experiment sessions, using the proposed EOG/ERP interface, the subject is able to finish the sessions in 54.1 s.