The motor imagery (MI) based brain–computer interface (BCI) holds broad application prospects in human–machine interaction. However, current MI recognition approaches primarily utilize complex attention modules for higher recognition accuracy, consequently hindering real-time BCI implementation. Furthermore, existing methods often overlook inter-subject variability, leading to inadequate generalization of model. Additionally, traditional BCI systems lack closed-loop feedback from the machine to the brain. To address these limitations, we develop a novel closed-loop motor imagery BCI system, which encompasses a spectral-temporal refined attention network via contrastive mutual learning (STRA-CML) and a brain-controlled perceived hand exoskeleton. Specifically, we first design a spectral temporal refined attention block to capture the most discriminative spectral and temporal features. Second, we investigate a contrastive mutual learning strategy incorporating supervised-contrastive learning to enhance the generalization of our STRA-CML. Finally, a brain–machine closed-loop interaction platform based on perceived hand exoskeleton is developed to validate the feasibility of the proposed STRA-CML and provide kinesthetic and visual feedback synchronized with MI. Competitive experimental results on two public datasets and a self-collected dataset demonstrate the effectiveness of our STRA-CML, indicating that our STRA-CML achieves superior classification performance of 83.89% on BCI IV 2a dataset, 86.93% on BCI IV 2b dataset, and 82.79% on self-collected dataset.

Individual-specific calibration is a major bottleneck in motor imagery (MI) electroencephalogram (EEG) decoding, limiting real-world neural-feedback rehabilitation. Transfer learning, particularly Test-Time Adaptation (TTA), offers a promising solution for direct online cross-subject adaptation, handling sequentially arriving unlabeled MI-EEG data. However, existing TTA methods, primarily designed for domains such as computer vision, face challenges when applied to MI-EEG data due to its scarcity and non-stationary nature. To address the challenges in direct online MI-EEG decoding, this paper proposes MI-IASW, a novel framework combining Information-Aggregation (IA) and Source-Guided Pseudo-Label Weighting (SW). IA leverages Mixed and Adaptive Batch Normalization (MABN) to ensure effective aggregation of statistical and gradient information. Additionally, IA adopts a Weight Aggregation (WA) strategy to improve generalization under limited data. Meanwhile, SW first evaluates the overconfident pseudo-labels with the guidance of source centers and then employs Class-Aware Weighting (CAW) to adjust sample contributions to the loss function. Experimental evaluations on two public MI-EEG datasets demonstrate that our proposed framework outperforms various competitive baselines, achieving an average performance gain of 3.17% over the baseline TTA methods and 6.80% over the source model. By eliminating the need for individual-specific offline calibration, MI-IASW enables practical deployment in real-world rehabilitation and improves cross-subject decoding.

Evaluating review helpfulness is pivotal in assessing the caliber of airline reviews, instigating lively debates in both academic and practical spheres. This study endeavors to construct a comprehensive conceptual framework grounded in signaling theory, recognizing two factors as indicators shaping the perceived helpfulness of reviews. Empirical analysis was conducted using 82,539 reviews from nine airlines on TripAdvisor. Initially, the study scrutinizes the combined impact of review sentiment and consumer rating, followed by exploring the influence of review inconsistency on review helpfulness. Our experimental results show that most variables achieved a significance of one thousandth. Additionally, we shed light on the moderating effects of several heuristic clues in the model, including text length, seat class, and region. These findings underscore those heuristic clues that collectively influence the helpfulness of reviews. The outcomes of this research can aid airlines in identifying the most helpful reviews, thereby mitigating consumer search costs and empowering reviewers to contribute more valuable insights.

Transfer learning is one of the popular methods to solve the problem of insufficient data in subject-specific electroencephalogram (EEG) recognition tasks. However, most existing approaches ignore the difference between subjects and transfer the same feature representations from source domain to different target domains, resulting in poor transfer performance. To address this issue, we propose a novel subject-specific EEG recognition method named deep multiview module adaption transfer (DMV-MAT) network. First, we design a universal deep multiview (DMV) network to generate different types of discriminative features from multiple perspectives, which improves the generalization performance by extensive feature sets. Second, module adaption transfer (MAT) is designed to evaluate each module by the feature distributions of source and target samples, which can generate an optimal weight sharing strategy for each target subject and promote the model to learn domain-invariant and domain-specific features simultaneously. We conduct extensive experiments in two EEG recognition tasks, i.e., motor imagery (MI) and seizure prediction, on four datasets. Experimental results demonstrate that the proposed method achieves promising performance compared with the state-of-the-art methods, indicating a feasible solution for subject-specific EEG recognition tasks. Implementation codes are available at https://github.com/YangLibuaa/DMV-MAT.

Industrial control systems rely on wireless devices and sensors, necessitating critical security. Physical layer authentication (PLA) is a promising mechanism for device authentication, utilizing its unique spatiotemporal characteristics and channel state randomness, which offers unforgeability and high informatics security with low computational overhead and efficiency in resource-constrained scenarios. However, existing PLA mechanisms face challenges in complex industrial wireless environments, including insufficient accuracy, computational complexity, inadequate noise consideration, and poor performance. To address these challenges, we propose a convolutional denoising autoencoder (CDAE) model that reduces feature dimensions, eliminates noise, and extracts key vectors. The weighted$k$-nearest neighbor algorithm classifies the extracted vectors for comprehensive authentication in control system networks. Accurate authentication enables efficient detection of malicious attacks. Simulation experiments show that using CDAE-extracted feature vectors achieves over 95% accuracy with only 1% training samples, surpassing channel state information-based authentication by 46.15%, validating the proposed mechanism’s effectiveness.

Unlike conventional IT systems, industrial control systems (ICSs) requires tailored attack detection methods due to its unique communication protocols. Existing attack detection methods lack the ability to consider both detection accuracy and time performance, particularly for highly stealthy fake data injection attacks (FDIAs). To address these challenges, this work proposes an online parallel attack detection method for ICS based on multibandpass filter. By building multiple adaptive filters based on energy equilibrium and time–frequency domain data transformation, we implement multifrequency band data segmentation. Hierarchical temporal memory (HTM) models are employed to parallelly fit the segmented data and detect anomalies. Simulation experiments demonstrate that our method outperforms the state-of-the-art Numenta method, achieving a 9% higher detection accuracy while reducing detection time to just 1/14 of Numenta’s. These results highlight the significant advantages of our method in striking a balance between detection accuracy and time performance. Our proposed method fills the gap in ICS attack detection and offers substantial improvements over existing techniques.

Emotion recognition in conversation (ERC) based on multiple modalities has attracted enormous attention. However, most research simply concatenated multimodal representations, generally neglecting the impact of cross-modal correspondences and uncertain factors, and leading to the cross-modal misalignment problems. Furthermore, recent methods only considered simple contextual features, commonly ignoring semantic clues and resulting in an insufficient capture of the semantic consistency. To address these limitations, we propose a novel multi-level alignment and cross-modal unified semantic graph refinement network (MA-CMU-SGRNet) for ERC task. Specifically, a multi-level alignment (MA) is first designed to bridge the gap between acoustic and lexical modalities, which can effectively contrast both the instance-level and prototype-level relationships, separating the multimodal features in the latent space. Second, a cross-modal uncertainty-aware unification (CMU) is adopted to generate a unified representation in joint space considering the ambiguity of emotion. Finally, a dual-encoding semantic graph refinement network (SGRNet) is investigated, which includes a syntactic encoder to aggregate information from near neighbors and a semantic encoder to focus on useful semantically close neighbors. Extensive experiments on three multimodal public datasets show the effectiveness of our proposed method compared with the state-of-the-art methods, indicating its potential application in conversational emotion recognition. Implementation codes can be available athttps://github.com/zxiaohen/MA-CMU-SGRNet.

Detecting and diagnosing major depressive disorder (MDD) is greatly crucial for appropriate treatment and support. In recent years, there have been efforts to develop automated methods for depression detection using machine learning techniques, which mainly analyze various data sources such as text, speech, and social media posts. However, the effectiveness and reliability of these methods may vary and more importantly, they fail to provide timely intervention and treatment to MDD patients. To address these challenges, we propose a novel electroencephalogram (EEG)-based MDD detection framework, which is named as multiview sparse dynamic graph convolution-based region-attention feature fusion network (MV-SDGC-RAFFNet). Specifically, we first design a multiview (MV) feature extractor to concurrently characterize EEG signals from temporal, spectral, and time-frequency views, providing rich semantic information on the emotional status of patients. Secondly, we introduce a sparse dynamic graph convolution network (SDGCN) to map the multidomain features into high-level representations, which avoids the limitation of over-smoothing and redundant edges existing in the conventional graph neural networks (GNNs). Finally, to efficiently fuse multidomain features, we propose a region-attention feature fusion network (RAFFNet), which applies different attention weights for brain regions and is greatly beneficial to boost the accuracy (ACC) of MDD detection. We validate the efficacy of the proposed MV-SDGC-RAFFNet framework on two public MDD datasets, and it achieves more promising detection performance against the state-of-the-art methods, indicating that our method has a prospect on clinical MDD detection.

Recognizing visual objects from single-trial electroencephalograph (EEG) signals is a promising brain-computer interface technology. However, due to the redundant features from noisy multichannel EEG signals, it is still a challenging task to achieve high precision recognition. Recent deep learning approaches commonly extract spatio-temporal features of EEG signals, which neglect important spectral-temporal features and may degrade the EEG recognition performance. To address the deficiency, we propose a novel channel attention weighting and multilevel adaptive spectral aggregation based dual-branch spatio-temporal-spectral transformer feature fusion network (CAW-MASA-STST) for EEG-based visual recognition. Specially, we first develop a channel attention weighting (CAW) to automatically learn the channel weights of EEG signals. Then, a graph convolution-based MASA is employed to aggregate spectral-temporal features of different sub-bands. Finally, an STST is designed to fuse spatio-temporal and spectral-temporal features, which enhances the comprehensive learning ability by modeling the temporal dependencies of the fused features. Competitive experimental results on two public datasets demonstrate that the proposed method is able to achieve superior recognition performance compared with the state-of-the-art methods, indicating a feasible solution for visual recognition-based BCI technology. The code of our proposed method will be available athttps://github.com/ljbuaa/VisualDecoding.

Conventional functional connectivity network (FCN) based on resting-state fMRI (rs-fMRI) can only reflect the relationship between pairwise brain regions. Thus, the hyper-connectivity network (HCN) has been widely used to reveal high-order interactions among multiple brain regions. However, existing HCN models are essentially spatial HCN, which reflect the spatial relevance of multiple brain regions, but ignore the temporal correlation among multiple time points. Furthermore, the majority of HCN construction and learning frameworks are limited to using a single template, while the multi-template carries richer information. To address these issues, we first employ multiple templates to parcellate the rs-fMRI into different brain regions. Then, based on the multi-template data, we propose a spatio-temporal weighted HCN (STW-HCN) to capture more comprehensive high-order temporal and spatial properties of brain activity. Next, a novel deep fusion model of multi-template called spatio-temporal weighted multi-hypergraph convolutional network (STW-MHGCN) is proposed to fuse the STW-HCN of multiple templates, which extracts the deep interrelation information between different templates. Finally, we evaluate our method on the ADNI-2 and ABIDE-I datasets for mild cognitive impairment (MCI) and autism spectrum disorder (ASD) analysis. Experimental results demonstrate that the proposed method is superior to the state-of-the-art approaches in MCI and ASD classification, and the abnormal spatio-temporal hyper-edges discovered by our method have significant significance for the brain abnormalities analysis of MCI and ASD.

Functional connectivity (FC) networks based on resting-state functional magnetic imaging (rs-fMRI) are reliable and sensitive for brain disorder diagnosis. However, most existing methods are limited by using a single template, which may be insufficient to reveal complex brain connectivities. Furthermore, these methods usually neglect the complementary information between static and dynamic brain networks, and the functional divergence among different brain regions, leading to suboptimal diagnosis performance. To address these limitations, we propose a novel multi-graph cross-attention based region-aware feature fusion network (MGCA-RAFFNet) by using multi-template for brain disorder diagnosis. Specifically, we first employ multi-template to parcellate the brain space into different regions of interest (ROIs). Then, a multi-graph cross-attention network (MGCAN), including static and dynamic graph convolutions, is developed to explore the deep features contained in multi-template data, which can effectively analyze complex interaction patterns of brain networks for each template, and further adopt a dual-view cross-attention (DVCA) to acquire complementary information. Finally, to efficiently fuse multiple static-dynamic features, we design a region-aware feature fusion network (RAFFNet), which is beneficial to improve the feature discrimination by considering the underlying relations among static-dynamic features in different brain regions. Our proposed method is evaluated on both public ADNI-2 and ABIDE-I datasets for diagnosing mild cognitive impairment (MCI) and autism spectrum disorder (ASD). Extensive experiments demonstrate that the proposed method outperforms the state-of-the-art methods. Our source code is available at https://github.com/mylbuaa/MGCA-RAFFNet.

Recently, the use of wireless channel state information (CSI) to generate encryption keys in the physical layer has gained significant attention from researchers. Unlike classical cryptography, this approach relies on the variability of the wireless channel, channel reciprocity, and spatial decorrelation to ensure security, making it more lightweight and providing strong randomness. This article proposes a physical layer key generation scheme for wireless LAN MIMO systems based on feature fusion autoencoder (FFAEncoder) to address the issue of a high key disagreement rate (KDR). Our approach involves extracting amplitude and phase features separately, fusing them through multiplication operator in a neural network, and using an autoencoder to extract common features. The proposed scheme was evaluated on multiple data sets in different real-world scenarios, and it was found that the transmitter and receiver codeword’s mean squared error (MSE) and mean absolute error (MAE) were smaller than those of the current models, indicating better key generation performance. Additionally, the proposed scheme’s KDR was smaller, with a decay rate faster than that of the other two models in the same environment, and the primary key bits were 1/2 and 1/3 that of the other models, respectively, as the signal-to-noise ratio (SNR) increased.

Physical-layer secret key generation (PLKG) is a new research area that has emerged in recent years. It is aimed at scenarios where legitimate IoT devices communicate directly, interacting with confidential information for lower overhead and higher security by using wireless channel. When applying it to wireless feature extraction, noise removal is not taken into account in current deep learning networks. To address these problems, the PLKG scheme based on bidirectional convergence feature learning convolutional network (BCFL-based scheme) is proposed, which consists of neural network called BCFL and a new quantization method to achieve better secret key generation. Unlike existing PLKG schemes that enabling both parties to communicate for obtaining higher channel feature similarities, when training it, channel state information (CSI) obtained by channel estimation for two legitimate devices during coherent time is used as inputs; and mean square error (MSE) between two outputs is used as a result of loss function for iterative training. Thus, it can obtain better denoising ability with guaranteed low computational resource consumption, and two legitimate devices can obtain highly correlated channel features. Multiple quantization method is also proposed to address low secret key generation rate (KGR) and low-secret key randomness (KR). The results show that the proposed BCFL-based scheme has a lower MSE than other schemes in different scenarios, indicating that it has better capability to learn channel reciprocity; and secret key error rate (KER) and time consumption are only about 50% of other schemes, which is a significant performance improvement.

With the development of big data and blockchain, an increasing number of scholars have begun to study blockchain for data sharing. By studying data sharing models that are based on blockchain, we find that almost all of them have the following problems: 1) it is difficult to protect the privacy and integrity of users’ data, along with users’ data ownership; 2) the storage burden of blockchain is heavy, and blockchain lacks a mechanism for dealing with data with diverse types and inconsistent formats; and 3) the consensus mechanism has low fairness or low efficiency. Therefore, we propose a data-sharing privacy protection model (DS2PM) that is based on blockchain and a federated learning mechanism for solving these problems. The safety analysis and experimental results show that the DS2PM outperforms the previously established schemes.

Nowadays, Industrial Internet of Things (IIoT) technology has made a great progress and the industrial control systems (ICSs) have been used extensively, which has brought more and more serious information security threats to the ICS at the same time. The authenticated key agreement (AKA) protocol is a common method to ensure the communication security. This work proposes a lightweight AKA protocol based on the elliptic curve cryptography (ECC) algorithm to adapt to the resource-constrained environment. We only employ hash operation, XOR operation, and ECC algorithm to encrypt the data in the authentication and key agreement phase, and avoid involving the register center while proceeding the key agreement, to give consideration to both performance and security. The security analyses indicate that our protocol can meet nine critical security requirements, more than all of the existing protocols, and the performance analysis carried out indicates that our protocol has less computational and communication overheads in contrast to other corelative protocols.

Clinically, retinal vessel segmentation is a significant step in the diagnosis of fundus diseases. However, recent methods generally neglect the difference of semantic information between deep and shallow features, which fail to capture the global and local characterizations in fundus images simultaneously, resulting in the limited segmentation performance for fine vessels. In this article, a global transformer (GT) and dual local attention (DLA) network via deep-shallow hierarchical feature fusion (GT-DLA-dsHFF) are investigated to solve the above limitations. First, the GT is developed to integrate the global information in the retinal image, which effectively captures the long-distance dependence between pixels, alleviating the discontinuity of blood vessels in the segmentation results. Second, DLA, which is constructed using dilated convolutions with varied dilation rates, unsupervised edge detection, and squeeze-excitation block, is proposed to extract local vessel information, consolidating the edge details in the segmentation result. Finally, a novel deep-shallow hierarchical feature fusion (dsHFF) algorithm is studied to fuse the features in different scales in the deep learning framework, respectively, which can mitigate the attenuation of valid information in the process of feature fusion. We verified the GT-DLA-dsHFF on four typical fundus image datasets. The experimental results demonstrate our GT-DLA-dsHFF achieves superior performance against the current methods and detailed discussions verify the efficacy of the proposed three modules. Segmentation results of diseased images show the robustness of our proposed GT-DLA-dsHFF. Implementation codes will be available on https://github.com/YangLibuaa/GT-DLA-dsHFF.

Advanced metering infrastructure (AMI) is a vital component of the smart grid (SG) for real-time data access and bidirectional communication. An authentication and key agreement (AKA) protocol is needed for AMI systems to ensure the confidentiality and integrity of communication data. Since the devices are connected to the open network and generally deployed outdoors with limited computation, communication, and storage, designing a suitable AKA protocol is a challenging task. Researchers are still looking for good ways to make the SG secure and efficient simultaneously. To remedy the situation, in this article, we advance a security-enhanced elliptic-curve-cryptography-based AKA protocol, and the security has been proven rigorously under the random oracle model and verified with the ProVerif tool. Furthermore, performance comparison validates the proposed protocol in affording improved security features with lower computation and communication cost. In addition, the proposed scheme is implemented practically on a testbed, which is deployed usingRaspberry Pi 3 Model B+for smart meters.

Facial expression recognition (FER) in the wild is a challenging task for affective computing in human–machine interaction fields. However, most of the existing methods fail to learn the most prominent regions of facial images by simple cross-entropy loss due to the imbalance problem commonly existing in FER datasets, which limits the robustness and interpretability of the model. In addition, these methods only capture local features of original images with multisize shallow convolution and ignore facial texture characteristics, leading to a suboptimal recognition performance. To address these issues, in this article, we propose a novel FER network, named the attention-rectified and texture-enhanced cross-attention transformer feature fusion network (AR-TE-CATFFNet). Specifically, an attention-rectified convolution block is first designed to assist multiple convolution heads to focus on the critical areas of human faces and improve the model generalization. Second, we investigate a texture enhancement block to capture texture features through local binary pattern and gray-level co-occurrence matrix, which solves the limitation of insufficient texture information. Finally, a cross-attention transformer feature fusion block is employed to deeply integrate red, green, blue (RGB) features and texture features globally, which is beneficial to boost the accuracy of recognition. Competitive experimental results on three public datasets validate the efficacy of the proposed method, indicating that our proposed method achieves superior classification performance of 89.50% on real-world affective faces database (RAF-DB) dataset, 65.66% on AffectNet dataset, and 74.84% on FER2013 dataset against the existing methods.

Synthetic digital twins based on medical data accelerate the acquisition, labelling and decision making procedure in digital healthcare. A core part of digital healthcare twins is model-based data synthesis, which permits the generation of realistic medical signals without requiring to cope with the modelling complexity of anatomical and biochemical phenomena producing them in reality. Unfortunately, algorithms for cardiac data synthesis have been so far scarcely studied in the literature. An important imaging modality in the cardiac examination is three-directional CINE multi-slice myocardial velocity mapping (3Dir MVM), which provides a quantitative assessment of cardiac motion in three orthogonal directions of the left ventricle. The long acquisition time and complex acquisition produce make it more urgent to produce synthetic digital twins of this imaging modality. In this study, we propose a hybrid deep learning (HDL) network, especially for synthetic 3Dir MVM data. Our algorithm is featured by a hybrid UNet and a Generative Adversarial Network with a foreground-background generation scheme. The experimental results show that from temporally down-sampled magnitude CINE images (six times), our proposed algorithm can still successfully synthesise high temporal resolution 3Dir MVM CMR data (PSNR=42.32) with precise left ventricle segmentation (DICE=0.92). These performance scores indicate that our proposed HDL algorithm can be implemented in real-world digital twins for myocardial velocity mapping data simulation. To the best of our knowledge, this work is the first one investigating digital twins of the 3Dir MVM CMR, which has shown great potential for improving the efficiency of clinical studies via synthesised cardiac data.

Magnetic resonance imaging (MRI) is an important non-invasive clinical tool that can produce high-resolution and reproducible images. However, a long scanning time is required for high-quality MR images, which leads to exhaustion and discomfort of patients, inducing more artefacts due to voluntary movements of the patients and involuntary physiological movements. To accelerate the scanning process, methods by k-space undersampling and deep learning based reconstruction have been popularised. This work introduced SwinMR, a novel Swin transformer based method for fast MRI reconstruction. The whole network consisted of an input module (IM), a feature extraction module (FEM) and an output module (OM). The IM and OM were 2D convolutional layers and the FEM was composed of a cascaded of residual Swin transformer blocks (RSTBs) and 2D convolutional layers. The RSTB consisted of a series of Swin transformer layers (STLs). The shifted windows multi-head self-attention (W-MSA/SW-MSA) of STL was performed in shifted windows rather than the multi-head self-attention (MSA) of the original transformer in the whole image space. A novel multi-channel loss was proposed by using the sensitivity maps, which was proved to reserve more textures and details. We performed a series of comparative studies and ablation studies in the Calgary-Campinas public brain MR dataset and conducted a downstream segmentation experiment in the Multi-modal Brain Tumour Segmentation Challenge 2017 dataset. The results demonstrate our SwinMR achieved high-quality reconstruction compared with other benchmark methods, and it shows great robustness with different undersampling masks, under noise interruption and on different datasets. The code is publicly available at https://github.com/ayanglab/SwinMR.

With the development of big data, data sharing has become a hot topic. According to the previous research on data sharing, there is a problem with regard to how to design an effective incentive mechanism to make users willing to share data. First, we integrate the incentives based on reputation and payment and introduce “credibility coins” as a cryptocurrency for data-sharing transactions, to encourage users to participate honestly in the data-sharing process based on federated learning. Second, we propose a dynamic incentive model based on the evolutionary game theory to model the game process of users in data sharing and analyze the stability of their strategies. Finally, based on the results of this analysis, we use the blockchain-based smart contract technology to dynamically adjust the participation benefits of users under different conditions in order to promote users to join consortium blockchains more often and steadily to participate in model training for federated learning and obtain better model accuracy. Our work is the first to apply the evolutionary game theory to the study of incentives in federated learning, and plays a leading role in the study of incentives in federated learning. Experimental simulation validation shows that our DIM-DS model can adequately motivate users to participate in the collaborative task of data sharing and maintain stability. The model can maximize the effectiveness of the federated learning model.

Electroencephalograph (EEG) emotion recognition plays an important role in the brain-computer interface (BCI) field. However, most of recent methods adopted shallow graph neural networks using a single temporal feature, leading to the limited emotion classification performance. Furthermore, the existing methods generally ignore the individual divergence between different subjects, resulting in poor transfer performance. To address these deficiencies, we propose a dual-branch dynamic graph convolution based adaptive transformer feature fusion network with adapter-finetuned transfer learning (DBGC-ATFFNet-AFTL) for EEG emotion recognition. Specifically, a dual-branch graph convolution network (DBGCN) is firstly designed to effectively capture the temporal and spectral characterizations of EEG simultaneously. Second, the adaptive Transformer feature fusion network (ATFFNet) is conducted by integrating the obtained feature maps with the channel-weight unit, leading to significant difference between different channels. Finally, the adapter-finetuned transfer learning method (AFTL) is applied in cross-subject emotion recognition, which proves to be parameter-efficient with few samples of the target subject. The competitive experimental results on three datasets have shown that our proposed method achieves the promising emotion classification performance compared with the state-of-the-art methods. The code of our proposed method will be available at:https://github.com/smy17/DANet.

Graph theory analysis using electroencephalogram (EEG) signals is currently an advanced technique for seizure prediction. Recent deep learning approaches, which fail to fully explore both the characterizations in EEGs themselves and correlations among different electrodes simultaneously, generally neglect the spatial or temporal dependencies in an epileptic brain and, thus, produce suboptimal seizure prediction performance consequently. To tackle this issue, in this article, a patient-specific EEG seizure predictor is proposed by using a novel spatio-temporal-spectral hierarchical graph convolutional network with an active preictal interval learning scheme (STS-HGCN-AL). Specifically, since the epileptic activities in different brain regions may be of different frequencies, the proposed STS-HGCN-AL framework first infers a hierarchical graph to concurrently characterize an epileptic cortex under different rhythms, whose temporal dependencies and spatial couplings are extracted by a spectral-temporal convolutional neural network and a variant self-gating mechanism, respectively. Critical intrarhythm spatiotemporal properties are then captured and integrated jointly and further mapped to the final recognition results by using a hierarchical graph convolutional network. Particularly, since the preictal transition may be diverse from seconds to hours prior to a seizure onset among different patients, our STS-HGCN-AL scheme estimates an optimal preictal interval patient dependently via a semisupervised active learning strategy, which further enhances the robustness of the proposed patient-specific EEG seizure predictor. Competitive experimental results validate the efficacy of the proposed method in extracting critical preictal biomarkers, indicating its promising abilities in automatic seizure prediction.

Dynamic functional connectivity (dFC) network inferred from resting-state fMRI reveals macroscopic dynamic neural activity patterns for brain disease identification. However, dFC methods ignore the causal influence between the brain regions. Furthermore, due to the complex non-Euclidean structure of brain networks, advanced deep neural networks are difficult to be applied for learning high-dimensional representations from brain networks. In this paper, a group constrained Kalman filter (gKF) algorithm is proposed to construct dynamic effective connectivity (dEC), where the gKF provides a more comprehensive understanding of the directional interaction within the dynamic brain networks than the dFC methods. Then, a novel virtual adversarial training convolutional neural network (VAT-CNN) is employed to extract the local features of dEC. The VAT strategy improves the robustness of the model to adversarial perturbations, and therefore avoids the overfitting problem effectively. Finally, we propose the high-order connectivity weight-guided graph attention networks (cwGAT) to aggregate features of dEC. By injecting the weight information of high-order connectivity into the attention mechanism, the cwGAT provides more effective high-level feature representations than the conventional GAT. The high-level features generated from the cwGAT are applied for binary classification and multiclass classification tasks of mild cognitive impairment (MCI). Experimental results indicate that the proposed framework achieves the classification accuracy of 90.9%, 89.8%, and 82.7% for normal control (NC) vs. early MCI (EMCI), EMCI vs. late MCI (LMCI), and NC vs. EMCI vs. LMCI classification respectively, outperforming the state-of-the-art methods significantly.

Retinal vessel segmentation with deep learning technology is a crucial auxiliary method for clinicians to diagnose fundus diseases. However, the deep learning approaches inevitably lose the edge information, which contains spatial features of vessels while performing down-sampling, leading to the limited segmentation performance of fine blood vessels. Furthermore, the existing methods ignore the dynamic topological correlations among feature maps in the deep learning framework, resulting in the inefficient capture of the channel characterization. To address these limitations, we propose a novel dual encoder-based dynamic-channel graph convolutional network with edge enhancement (DE-DCGCN-EE) for retinal vessel segmentation. Specifically, we first design an edge detection-based dual encoder to preserve the edge of vessels in down-sampling. Secondly, we investigate a dynamic-channel graph convolutional network to map the image channels to the topological space and synthesize the features of each channel on the topological map, which solves the limitation of insufficient channel information utilization. Finally, we study an edge enhancement block, aiming to fuse the edge and spatial features in the dual encoder, which is beneficial to improve the accuracy of fine blood vessel segmentation. Competitive experimental results on five retinal image datasets validate the efficacy of the proposed DE-DCGCN-EE, which achieves more remarkable segmentation results against the other state-of-the-art methods, indicating its potential clinical application.

Multi-atlas parcellation (MAP) is carried out on a brain image by propagating and fusing labelled regions from brain atlases. Typical nonlinear registration-based label propagation is time-consuming and sensitive to inter-subject differences. Recently, deep learning parcellation (DLP) has been proposed to avoid nonlinear registration for better efficiency and robustness than MAP. However, most existing DLP methods neglect using brain atlases, which contain high-level information (e.g., manually labelled brain regions), to provide auxiliary features for improving the parcellation accuracy. In this paper, we propose a novel multi-atlas DLP method for brain parcellation. Our method is based on fully convolutional networks (FCN) and squeeze-and-excitation (SE) modules. It can automatically and adaptively select features from the most relevant brain atlases to guide parcellation. Moreover, our method is trained via a generative adversarial network (GAN), where a convolutional neural network (CNN) with multi-scale l1loss is used as the discriminator. Benefiting from brain atlases, our method outperforms MAP and state-of-the-art DLP methods on two public image datasets (LPBA40 and NIREP-NA0).

Dynamic functional connectivity (dFC) analysis using resting-state functional Magnetic Resonance Imaging (rs-fMRI) is currently an advanced technique for capturing the dynamic changes of neural activities in brain disease identification. Most existing dFC modeling methods extract dynamic interaction information by using the sliding window-based correlation, whose performance is very sensitive to window parameters. Because few studies can convincingly identify the optimal combination of window parameters, sliding window-based correlation may not be the optimal way to capture the temporal variability of brain activity. In this paper, we propose a novel adaptive dFC model, aided by a deep spatial-temporal feature fusion method, for mild cognitive impairment (MCI) identification. Specifically, we adopt an adaptive Ultra-weighted-lasso recursive least squares algorithm to estimate the adaptive dFC, which effectively alleviates the problem of parameter optimization. Then, we extract temporal and spatial features from the adaptive dFC. In order to generate coarser multi-domain representations for subsequent classification, the temporal and spatial features are further mapped into comprehensive fused features with a deep feature fusion method. Experimental results show that the classification accuracy of our proposed method is reached to 87.7%, which is at least 5.5% improvement than the state-of-the-art methods. These results elucidate the superiority of the proposed method for MCI classification, indicating its effectiveness in the early identification of brain abnormalities.

Glaucoma is one of the leading causes of irreversible vision loss. Many approaches have recently been proposed for automatic glaucoma detection based on fundus images. However, none of the existing approaches can efficiently remove high redundancy in fundus images for glaucoma detection, which may reduce the reliability and accuracy of glaucoma detection. To avoid this disadvantage, this paper proposes an attention-based convolutional neural network (CNN) for glaucoma detection, called AG-CNN. Specifically, we first establish a large-scale attention-based glaucoma (LAG) database, which includes 11 760 fundus images labeled as either positive glaucoma (4878) or negative glaucoma (6882). Among the 11 760 fundus images, the attention maps of 5824 images are further obtained from ophthalmologists through a simulated eye-tracking experiment. Then, a new structure of AG-CNN is designed, including an attention prediction subnet, a pathological area localization subnet, and a glaucoma classification subnet. The attention maps are predicted in the attention prediction subnet to highlight the salient regions for glaucoma detection, under a weakly supervised training manner. In contrast to other attention-based CNN methods, the features are also visualized as the localized pathological area, which are further added in our AG-CNN structure to enhance the glaucoma detection performance. Finally, the experiment results from testing over our LAG database and another public glaucoma database show that the proposed AG-CNN approach significantly advances the state-of-the-art in glaucoma detection.

Mild cognitive impairment (MCI) detection is important, such that appropriate interventions can be imposed to delay or prevent its progression to severe stages, including Alzheimer's disease (AD). Brain connectivity network inferred from the functional magnetic resonance imaging data has been prevalently used to identify the individuals with MCI/AD from the normal controls. The capability to detect the causal or effective connectivity is highly desirable for understanding directed functional interactions between brain regions and further helping the detection of MCI. In this paper, we proposed a novel sparse constrained effective connectivity inference method and an elastic multilayer perceptron classifier for MCI identification. Specifically, a ultra-group constrained structure detection algorithm is first designed to identify the parsimonious topology of the effective connectivity network, in which the weak derivatives of the observable data are considered. Second, based on the identified topology structure, an effective connectivity network is then constructed by using an ultra-orthogonal forward regression algorithm to minimize the shrinking effect of the group constraint-based method. Finally, the effective connectivity network is validated in MCI identification using an elastic multilayer perceptron classifier, which extracts lower to higher level information from initial input features and hence improves the classification performance. Relatively high classification accuracy is achieved by the proposed method when compared with the state-of-the-art classification methods. Furthermore, the network analysis results demonstrate that MCI patients suffer a rich club effect loss and have decreased connectivity among several brain regions. These findings suggest that the proposed method not only improves the classification performance but also successfully discovers critical disease-related neuroimaging biomarkers.

The electroencephalogram (EEG) signal analysis is a valuable tool in the evaluation of neurological disorders, which is commonly used for the diagnosis of epileptic seizures. This paper presents a novel automatic EEG signal classification method for epileptic seizure detection. The proposed method first employs a continuous wavelet transform (CWT) method for obtaining the time-frequency images (TFI) of EEG signals. The processed EEG signals are then decomposed into five sub-band frequency components of clinical interest since these sub-band frequency components indicate much better discriminative characteristics. Both Gaussian Mixture Model (GMM) features and Gray Level Co-occurrence Matrix (GLCM) descriptors are then extracted from these sub-band TFI. Additionally, in order to improve classification accuracy, a compact feature selection method by combining the ReliefF and the support vector machine-based recursive feature elimination (RFE-SVM) algorithm is adopted to select the most discriminative feature subset, which is an input to the SVM with the radial basis function (RBF) for classifying epileptic seizure EEG signals. The experimental results from a publicly available benchmark database demonstrate that the proposed approach provides better classification accuracy than the recently proposed methods in the literature, indicating the effectiveness of the proposed method in the detection of epileptic seizures.

A novel method of meaningful image encryption is proposed in this paper. A secret image is encrypted into another meaningful image using the algorithm of reversible data hiding (RDH). High covertness can be ensured during the communication, and the possibility of being attacked of the secret image would be reduced to a very low level. The key innovation of the proposed method is that RDH is applied to compressive sensing (CS) domain, which brings a variety of benefits in terms of image sampling, communication and security. The secret image after preliminary encryption is embedded into the sparse representation coefficients of the host image with the help of the dictionary. The embedding rate could reach 2 bpp, which is significantly higher than those of other state-of-art schemes. In addition, the computational complexity of receiver is reduced. Simulations verify our proposal.

The automatic detection of epileptic seizures from electroencephalography (EEG) signals is crucial for the localization and classification of epileptic seizure activity. However, seizure processes are typically dynamic and nonstationary, and thus, distinguishing rhythmic discharges from nonstationary processes is one of the challenging problems. In this paper, an adaptive and localized time-frequency representation in EEG signals is proposed by means of multiscale radial basis functions (MRBF) and a modified particle swarm optimization (MPSO) to improve both time and frequency resolution simultaneously, which is a novel MRBF-MPSO framework of the time-frequency feature extraction for epileptic EEG signals. The dimensionality of extracted features can be greatly reduced by the principle component analysis algorithm before the most discriminative features selected are fed into a support vector machine (SVM) classifier with the radial basis function (RBF) in order to separate epileptic seizure from seizure-free EEG signals. The classification performance of the proposed method has been evaluated by using several state-of-art feature extraction algorithms and other five different classifiers like linear discriminant analysis, and logistic regression. The experimental results indicate that the proposed MRBF-MPSO-SVM classification method outperforms competing techniques in terms of classification accuracy, and shows the effectiveness of the proposed method for classification of seizure epochs and seizure-free epochs.

A new parametric approach is proposed for nonlinear and nonstationary system identification based on a time-varying nonlinear autoregressive with exogenous input (TV-NARX) model. The TV coefficients of the TV-NARX model are expanded using multiwavelet basis functions, and the model is thus transformed into a time-invariant regression problem. An ultra-orthogonal forward regression (UOFR) algorithm aided by mutual information (MI) is designed to identify a parsimonious model structure and estimate the associated model parameters. The UOFR-MI algorithm, which uses not only the observed data themselves but also weak derivatives of the signals, is more powerful in model structure detection. The proposed approach combining the advantages of both the basis function expansion method and the UOFR-MI algorithm is proved to be capable of tracking the change of TV parameters effectively in both numerical simulations and the real EEG data.

Electroencephalogram (EEG) is a noninvasive method to record electrical activity of brain and it has been used extensively in research of brain function due to its high time resolution. However raw EEG is a mixture of signals, which contains noises such as Ocular Artifact (OA) that is irrelevant to the cognitive function of brain. To remove OAs from EEG, many methods have been proposed, such as Independent Components Analysis (ICA), Discrete Wavelet Transform (DWT), Adaptive Noise Cancellation (ANC) and Wavelet Packet Transform (WPT). In this paper, we present a novel hybrid de-noising method which uses Discrete Wavelet Transform (DWT) and Kalman Filtering to remove OAs in EEG. Firstly, we used this method on simulated data. The Mean Squared Error (MSE) of DWT-Kalman method was 0.0017, significantly lower compared to results using WPT-ICA and DWT-ANC, which were 0.0468 and 0.0052, respectively. Meanwhile, the Mean Absolute Error (MAE) using DWT-Kalman achieved an average of 0.0052, which also performed better than WPT-ICA and DWT-ANC, which were 0.0218 and 0.0115, respectively. Then we applied the proposed approach to the raw data collected by our prototype three-channel EEG collector and 64-channel Braincap from BRAIN PRODUCTS. On both data, our method achieved satisfying results. This method does not rely on any particular electrode or the number of electrodes in certain system, so it is recommended for ubiquitous applications.

Modeling and prediction of Electroencephalogram (EEG) signals is very important for Portable applications; EEG signals are however widely regarded as being chaotic in nature. An adaptive modeling technique that combines Discrete Wavelet Transformation (DWT) to predict contaminated EEG signals for removal of ocular artifacts (OAs) from EEG records is proposed as an effective a data processing tool for Interventions in Mental Illness Based on Bio-feedback. The proposed method is well suited for use in portable environments where constraints with respect to acceptable wearable sensor attachments usually dictate single channel devices. Using simulated and measured data the accuracy of the proposed model is compared to the accuracy of other pre-existing methods based on Wavelet Packet Transform (WPT) and independent component analysis (ICA) using DWT and adaptive noise cancellation (ANC) for Portable applications. The results show that the our new model not only demonstrates an improved performance with respect to the recovery of true EEG signals, achieves improved computational speed, and demonstrates better tracking performance.

Owning to its clinical accessibility, T1-weighted MRI has been extensively studied for the prediction of mild cognitive impairment (MCI) and Alzheimer's disease (AD). The tissue volumes of GM, WM and CSF are the most commonly used measures for MCI and AD prediction. We note that disease-induced structural changes may not happen at isolated spots, but in several inter-related regions. Therefore, in this paper we propose to directly extract the inter-region connectivity based features for MCI prediction. This involves constructing a brain network for each subject, with each node representing an ROI and each edge representing regional interactions. This network is also built hierarchically to improve the robustness of classification. Compared with conventional methods, our approach produces a significant larger pool of features, which if improperly dealt with, will result in intractability when used for classifier training. Therefore based on the characteristics of the network features, we employ Partial Least Square analysis to efficiently reduce the feature dimensionality to a manageable level while at the same time preserving discriminative information as much as possible. Our experiment demonstrates that without requiring any new information in addition to T1-weighted images, the prediction accuracy of MCI is statistically improved.