Grade classification of gliomas is critical in clinical diagnosis and treatment decisions. Although histological images are commonly used for grading and as an important factor in prognostic prediction, their results are prone to inter-observer variability. Recent advancements in molecular genetics have significantly improved tumor classification, but challenges persist in effective feature selection and multi-modal data fusion. This letter proposes a multi-modal hybrid encoding method based on information bottleneck (MHEIB), combining histological images and genetic data to enhance glioma grading. MHEIB effectively fuses multi-modal features through the information bottleneck module and the self-attention mechanism, which compresses and filters the key features and dynamically adjusts the weights of multi-modal features to improve the classification accuracy. Experimental results on The Cancer Genome Atlas (TCGA) glioma dataset demonstrate that MHEIB outperforms several fusion methods in terms of F1-score, AUC, and AP. In particular, MHEIB significantly improved the classification AUC to 89.3% and 83.7% for similar categories of Grades II and III respectively. Overall, the MHEIB method provides an efficient multi-modal data fusion solution for glioma grading.

Electroencephalography (EEG) has been widely used for the detection of major depressive disorder (MDD). Currently, several methods have been proposed to process EEG signals for MDD detection using deep learning algorithms, and some advantages have been achieved. However, the extraction of EEG features for MDD detection remains challenging and most methods have difficulty in extracting common features of EEG signals across subjects. We propose a graph domain adaptation with global message-passing (GMP-GDA) for MDD detection. In addition, an adjacency matrix weighting algorithm is designed in the global message-passing module to learn the weight combinations of different adjacency matrices instead of feature transformation matrices to achieve cross-domain global message-passing between multi-source and target domains. Experimental results demonstrate that the proposed method achieves the superior detection result in each frequency band compared to the baseline systems. Meanwhile, ablation experiments demonstrate the effectiveness of our proposed method.

Despite recent advances, scene text recognition remains a challenging problem due to the significant variability, irregularity and distortion in text appearance and localization. Attention-based methods have become the mainstream due to their superior vocabulary learning and observation ability. Nonetheless, they are susceptible to attention drift which can lead to word recognition errors. Most works focus on correcting attention drift in decoding but completely ignore the error accumulated during the encoding process. In this paper, we propose a novel scheme, called the Attention Guidance by Cross-Domain Supervision Signals for Scene Text Recognition (ACDS-STR), which can mitigate the attention drift at the feature encoding stage. At the heart of the proposed scheme is the cross-domain attention guidance and feature encoding fusion module (CAFM) that uses the core areas of characters to recursively guide attention to learn in the encoding process. With precise attention information sourced from CAFM, we propose a non-attention-based adaptive transformation decoder (ATD) to guarantee decoding performance and improve decoding speed. In the training stage, we fuse manual guidance and subjective learning to learn the core areas of characters, which notably augments the recognition performance of the model. Experiments are conducted on public benchmarks and show the state-of-the-art performance. The source will be available at https://github.com/xuefanfu/ACDS-STR.

Semi-supervised learning (SSL) has emerged as a powerful technique to mitigate the scarcity of labeled data. However, the effectiveness of most SSL methods relies on the assumption of a balanced class distribution, which often proves unrealistic, especially in medical imaging scenarios. To address this challenge, we propose Class-Specific Thresholding (CST) for imbalanced SSL. Specifically, CST dynamically integrates the model's learning status, class-specific learning effects, and data class distribution to estimate confidence thresholds for each class. These thresholds enhance the reliability of selected unlabeled data, particularly for minority classes. Additionally, we introduce a class-sensitive unsupervised loss function that further intensifies the model's focus on minority classes at minimal computational cost by leveraging predicted class distributions from previous inputs. Extensive experiments on highly imbalanced skin disease and endoscopy image datasets demonstrate that CST significantly outperforms state-of-the-art SSL methods, particularly in improving the accuracy for minority classes. These results validate the effectiveness of CST in addressing class imbalance.

In clinical settings, the implementation of deep neural networks is impeded by the prevalent problems of label scarcity and class imbalance in medical images. To mitigate the need for labeled data, semi-supervised learning (SSL) has gained traction. However, existing SSL schemes exhibit certain limitations. 1) They commonly fail to address the class imbalance problem. Training with imbalanced data makes the model's prediction biased towards majority classes, consequently introducing prediction bias. 2) They usually suffer from training bias arising from unreasonable training strategies, such as strong coupling between the generation and utilization of pseudo labels. To address these problems, we propose a novel SSL framework called Tri-Net with Cross-Balanced pseudo supervision (TNCB). Specifically, two student networks focusing on different learning tasks and a teacher network equipped with an adaptive balancer are designed. This design enables the teacher model to pay more focus on minority classes, thereby reducing prediction bias. Additionally, we propose a virtual optimization strategy to further enhance the teacher model's resistance to class imbalance. Finally, to fully exploit valuable knowledge from unlabeled images, we employ cross-balanced pseudo supervision, where an adaptive cross loss function is introduced to reduce training bias. Extensive evaluation on four datasets with different diseases, image modalities, and imbalance ratios consistently demonstrate the superior performance of TNCB over state-of-the-art SSL methods. These results indicate the effectiveness and robustness of TNCB in addressing imbalanced medical image classification challenges.

Accurate and fully automated brain structure examination and prediction from 3D volumetric magnetic resonance imaging (MRI) is a necessary step in medical imaging analysis, which can assist greatly in clinical diagnosis. Traditional deep learning models suffer from severe performance degradation when applied to clinically acquired unlabeled data. The performance degradation is mainly caused by domain discrepancy such as different device types and parameter settings for data acquisition. However, existing approaches focus on the reduction of domain discrepancies but ignore the entanglement of semantic features and domain information. In this article, we explore the feature invariance of categories and domains in different projection spaces and propose a Siamese-Transport Domain Adaptation (STDA) method using a joint optimal transport theory and contrastive learning for automatic 3D MRI classification and glioma multi-grade prediction. Specifically, the learning framework updates the distribution of features across domains and categories by Siamese transport network training with an Optimal Cost Transfer Strategy (OCTS) and a Mutual Invariant Constraint (MIC) in two projective spaces to find multiple invariants in potential heterogeneity. We design three sets of transfer task scenarios with different source and target domains, and demonstrate that STDA yields substantially higher generalization performance than other state-of-the-art unsupervised domain adaptation (UDA) methods. The method is applicable on 3D MRI data from glioma to Alzheimer's disease and has promising applications in the future clinical diagnosis and treatment of brain diseases.

Accurate identification of lesions is a key step in surgical planning. However, this task mainly exists two challenges: 1) Due to the complex anatomical shapes of different lesions, most segmentation methods only achieve outstanding performance for a specific structure, rather than other lesions with location differences. 2) The huge number of parameters limits existing transformer-based segmentation models. To overcome these problems, we propose a novel slight dual-path network (SDPN) to segment variable location lesions or organs with significant differences accurately. First, we design a dual-path module to integrate local with global features without obvious memory consumption. Second, a novel Multi-spectrum attention module is proposed to pay further attention to detailed information, which can automatically adapt to the variable segmentation target. Then, the compression module based on tensor ring decomposition is designed to compress convolutional and transformer structures. In the experiment, four datasets, including three benchmark datasets and a clinical dataset, are used to evaluate SDPN. Results of the experiments show that SDPN performs better than other start-of-the-art methods for brain tumor, liver tumor, endometrial tumor and cardiac segmentation. To ensure the generalizability, we train the network on Kvasir-SEG and test on CVC-ClinicDB which collected from a different institution. The quantitative analysis shows that the clinical evaluation results are consistent with the experts. Therefore, this model may be a potential candidate for the segmentation of lesions and organs segmentation with variable locations in clinical applications.

The brain tumor is one of the primary diseases that endanger human life and health. Multi-modality magnetic resonance imaging (MRI) for tumor analysis is one of the key techniques for clinical diagnosis. Experienced experts artificially segment the classical method of brain tumor segmentation according to their anatomical and pathological knowledge, which is time-consuming. In this paper, we propose a novel deep neural network structure, namely Feature Mining Networks (FMNet), for brain tumor segmentation. The proposed FMNet adopts three innovative structure, including semantic information mining unit (SIMU), macro information mining unit (MIMU), and a feature correction unit (FCU). These three units can enhance the mining of semantic information and spatial information, and further, modify the information in a direction that is conducive to segmentation results. Each unit can bring significant improvement in segmentation performance. We evaluate the proposed framework on BraTS2017 and BraTS2018 dataset. The experimental results show that our FMNet performs better than state-of-the-art networks such as fully convolutional networks (FCN), U-Net, VGG, and Hybrid Pyramid U-Net (HPUNet).

Event-related potentials (ERPs) especially P300 are popular effective features for brain-computer interface (BCI) systems based on electroencephalography (EEG). Traditional ERP-based BCI systems may perform poorly for small training samples, i.e. the undersampling problem. In this study, the ERP classification problem was investigated, in particular, the ERP classification in the high-dimensional setting with the number of features larger than the number of samples was studied. A flexible group sparse discriminative analysis algorithm based on Moreau-Yosida regularization was proposed for alleviating the undersampling problem. An optimization problem with the group sparse criterion was presented, and the optimal solution was proposed by using the regularized optimal scoring method. During the alternating iteration procedure, the feature selection and classification were performed simultaneously. Two P300-based BCI datasets were used to evaluate our proposed new method and compare it with existing standard methods. The experimental results indicated that the features extracted via our proposed method are efficient and provide an overall better P300 classification accuracy compared with several state-of-the-art methods.

Appearance-based gaze estimation methods have received increasing attention in the field of human-computer interaction (HCI). These methods tried to estimate the accurate gaze point via Convolutional Neural Network (CNN) model, but the estimated accuracy can't reach the requirement of gaze-based HCI when the regression model is used in the output layer of CNN. Given the popularity of button-touch-based interaction, we propose an appearance-based gaze block estimation method, which aims to estimate the gaze block, not the gaze point. In the proposed method, we relax the estimation from point to block, so that the gaze block can be estimated by CNN-based classification instead of the previous regression model. We divide the screen into square blocks to imitate the button-touch interface, and build an eye-image dataset, which contains the eye images labelled by their corresponding gaze blocks on the screen. We train the CNN model according to this dataset to estimate the gaze block by classifying the eye images. The experiments on 6- and 54-block classifications demonstrate that the proposed method has high accuracy in gaze block estimation without any calibration, and it is promising in button-touch-based interaction.

Depth propagation plays an important role to improve the quality of converted video in 2D-to-3D conversion. During depth propagation, the propagation path is usually along the time. In this paper, a new depth propagation algorithm is proposed to improve the quality of depth propagation by changing the propagation path according to frame clustering. The frames are clustered via K-means clustering based on HSV color histogram. The frame at the center of each cluster is selected as the keyframe of the cluster and the other frames are taken as the non-keyframes attaching to the keyframe. The depth information of keyframe is propagated to the non-keyframes within the same cluster directly. The performance of the proposed depth propagation algorithm is evaluated by MSE of the propagated depth map comparing with several existing algorithms. The comparison results show that the depth propagation errors can be reduced a lot by the proposed clustering based propagation.

In this paper, we proposed a Nonnegative Shifted Tensor Factorization (NSTF) model considering multiple component delays by time frequency analysis. Explicit mathematical representation for the delays is presented to recover the patterns from the original data. In order to explore multilinear shifted component in different modes, we use fast fourier transform (FFT) to transform the non-integer delays into frequency domain by gradients search. The ALS algorithm for NSTF is developed by alternating least square procedure to estimate the nonnegative factor matrices in each mode and enforce the sparsity of model. Simulation results indicate that ALS-NSTF algorithm can extract the shift-invariance sparse features and improve the recognition performance of robust speaker identification and structural magnetic resonance imaging (sMRI) diagnosis for Alzheimer's Disease.

Non-negative tensor factorization (NTF) has been successfully applied to analyze event-related potentials (ERPs), and shown superiority in terms of capturing multi-domain features. However, the time-frequency representation of ERPs by higher-order tensors are usually large-scale, which prevents the popularity of most tensor factorization algorithms. To overcome this issue, we introduce a non-negative canonical polyadic decomposition (NCPD) based on low-rank approximation (LRA) and hierarchical alternating least square (HALS) techniques. We applied NCPD (LRAHALS and benchmark HALS) and CPD to extract multi-domain features of a visual ERP. The features and components extracted by LRAHALS NCPD and HALS NCPD were very similar, but LRAHALS NCPD was 70 times faster than HALS NCPD. Moreover, the desired multi-domain feature of the ERP by NCPD showed a significant group difference (control versus depressed participants) and a difference in emotion processing (fearful versus happy faces). This was more satisfactory than that by CPD, which revealed only a group difference.

There have been many attempts to design brain-computer interfaces (BCIs) for wheelchair control based on steady state visual evoked potential (SSVEP), event-related desynchronization/synchronization (ERD/ERS) during motor imagery (MI) tasks, P300 evoked potential, and some hybrid signals. However, those BCI systems cannot implement the wheelchair navigation flexibly and effectively. In this paper, we propose a hybrid BCI scheme based on two-class MI and four-class SSVEP tasks. It cannot only provide multi-degree control for its user, but also allow the user implement the different types of commands in parallel. In order for the subject to learn the hybrid mental strategies effectively, we design a visual and auditory cues and feedback-based training paradigm. Furthermore, an algorithm based on entropy of classification probabilities is proposed to detect intentional control (IC) state for hybrid tasks, and ensure that multi-degree control commands are accurately and quickly generated. The experiment results attest to the efficiency and flexibility of the hybrid BCI for wheelchair control in the real-world.

Non-negative Canonical Polyadic decomposition (NCPD) and non-negative Tucker decomposition (NTD) were compared for extracting the multi-domain feature of visual mismatch negativity (vMMN), a small event-related potential (ERP), for the cognitive research. Since signal-to-noise ratio in vMMN is low, NTD outperformed NCPD. Moreover, we proposed an approach to select the multi-domain feature of an ERP among all extracted features and discussed determination of numbers of extracted components in NCPD and NTD regarding the ERP context.

The performance of speech recognition systems relies on the consistency and adaptation of the speech feature in complex conditions during the training and testing stages. Traditional systems usually perform poorly under adverse noisy conditions and are not applicable to most real world problems. In this paper, we investigate the speech feature extraction problem in a noisy environment and propose a novel approach based on Gabor filtering and tensor factorization. Recent physiological and psychoacoustic experimental results suggest that the localized spectro-temporal features are essential for auditory perception. To explore this property, we represent the speech signal by using a general higher order tensor and employ two-dimensional Gabor functions with different scales and directions to analyze the localized patches of the power spectrogram. Then the Nonnegative Tensor PCA with sparse constraints is proposed to learn the projection matrices from multiple interrelated feature subspaces. The objective of the sparse constraints is to preserve the statistical characteristic of clean speech data by finding projection matrices of speech subspaces and reduce the noise components which have distributions different from those of clean speech. A multifactor analysis method is proposed to extract robust sparse features by processing the data samples in tensor structure. The simulation results indicate that our proposed method is able to improve the speech recognition performance, especially in noisy environments, compared with the traditional speech feature extraction methods.

In this paper, we investigate the speech feature extraction problem in the noisy environment. A novel approach based on Gabor filtering and tensor factorization is proposed. From recent physiological and psychoacoustic experimental results, localized spectro-temporal features are essential for auditory perception. We employ 2D-Gabor functions with different scales and directions to analyze the localized patches of power spectrogram, by which speech signal can be encoded as a general higher order tensor. Then nonnegative tensor PCA with sparse constraint is used to learn the projection matrices from multiple interrelated feature subspaces and extract the robust features. Experimental results confirm that our proposed method can improve the speech recognition performance, especially in noisy environment, compared with traditional speech feature extraction methods.