Alzheimer's Disease (AD) is a prevalent and severe neurodegenerative disorder, and early diagnosis is essential for managing disease progression. Recently, multimodal graph learning has demonstrated significant potential in integrating both medical imaging and non-imaging data, as well as uncovering relationships between patients. However, the high-dimensional nature of multimodal medical data poses significant challenges for constructing and learning modality graph structures. Moreover, existing methods are often imprecise in modeling graph structures for continuous data. To address these issues, this paper introduces a novel multimodal multi-graph fusion learning method for Alzheimer's disease diagnosis. Specifically, multimodal state space networks (multimodal SSNs) are proposed to capture the dependencies between multimodal and high-dimensional features. Furthermore, a novel graph structure learning (KGSL) based on an initial K-nearest neighbors graph is proposed to separately construct graph structures for each modality. This method is particularly suitable for modeling the graph structures of Euclidean data. Finally, multimodal graph fusion integrates various modal graph structures into a single graph, leading to enhanced multimodal integration. In addition, this paper uses a learnable Chebyshev Graph Convolutional Network for the classification network, which enables end-to-end optimization. Experimental results demonstrate that our approach achieves excellent performance on public datasets.

Multimodal image fusion aims to merge features from different modalities to create a comprehensively representative image. However, existing medical image fusion methods often struggle to handle noise generated during image acquisition, significantly diminishing their impact on visual quality. To address these challenges, we propose a semantically text-guided medical image fusion model, named TDMF. Specifically, TDMF guides classical image fusion through textual semantics and effectively coordinates the resolution of degradation and interaction issues during the fusion process. By integrating text encoders and interactive fusion modules, TDMF establishes a unified framework for denoising and interactive fusion of medical images. Extensive experiments have demonstrated that our proposed text-guided image fusion strategy offers significant advantages over state-of-the-art methods in medical image fusion performance.

Few-shot learning (FSL) has gained increasing attention in hyperspectral image (HSI) classification due to its ability to perform cross-domain classification with minimal labeled samples. However, existing FSL methods overlook the continuity of HSI spectral sequences and fail to utilize the large amount of unlabeled samples in the target domain. To address these issues, we introduce a novel cross-domain HSI classification method that combines self-supervised learning with FSL (SSFSL). This approach uses self-supervised learning and FSL to extract transferable knowledge from the source domain and introduces an adaptive soft label generation algorithm to leverage unlabeled samples in the target domain. Compared to existing cross-domain FSL classification methods, the proposed approach considers the spectral sequence continuity of HSI and effectively extracts useful information from unlabeled samples in the target domain. Extensive experiments conducted on three datasets demonstrate that SSFSL outperforms state-of-the-art methods in both quantitative and qualitative aspects.

Multimodal image fusion integrates information from different modalities to generate images with complementary features. However, existing medical image fusion methods often lack interactive guidance tailored to specific subjective and objective needs, resulting in a uniform processing approach for all image regions within a single model. This limitation does not fully account for region-specific characteristics, restricting the ability of the model to effectively address degradation issues encountered during image acquisition.To overcome these challenges, we propose a novel semantic text-guided medical image fusion and interactive degradation handling method called STGFuse. On one hand, the text-guided semantic encoder enables integrated processing of both image degradation and fusion tasks. It allows for text-based selection of regions and effects, focusing on preserving critical features from different images and producing visually compelling fusion results. On the other hand, we introduce a dedicated interactive fusion module, enhancing the synergistic interaction between visual and textual information and facilitating more profound cross-modal integration. Extensive experiments demonstrate the effectiveness of STGFuse, with both qualitative and quantitative evaluations proving its superiority over existing methods in addressing fusion challenges caused by image degradation.

Automatic skin lesion segmentation is a critical tool in clinical diagnosis, which significantly enhances the accuracy of early diagnosis. However, due to the diverse shapes, blurred boundaries and interference from hair in the samples, skin lesion segmentation remains a challenging task. To overcome these challenges, we propose a parallel-branch codec structure called CM-Net. Specifically, we design a feature extraction module that uses parallel Mamba and CNN branches to perform global modeling and local feature extraction. This module effectively captures global lesion area and detailed boundary information. Additionally, a Multi-Scale Attention Cross Fusion module (MACF) is designed to replace traditional skip connections, thereby enhancing the interaction between shallow and deep features. It helps mitigate semantic gap while suppressing background interference. Extensive experiments conducted on two public datasets demonstrate that our method achieves superior segmentation performance compared to most state-of-the-art approaches.

Infrared and visible image fusion is an image enhancement technique that generates a single image with rich textures and significant objectives in a variety of scenarios, providing great convenience for human discrimination and computer recognition. However, in low-light environments, low-intensity visible images tend to blur valuable information, and these details are often ignored during image fusion, resulting in the loss of important information. Although existing methods take into account the damage of low illumination and highlight the illumination in the fusion process, a large amount of structural information is lost in the process of adjusting illumination, resulting in the lack of texture details and poor performance in high-level vision tasks. To address the above challenges, this paper proposes a structure-aware image fusion method for low illumination scenes, called SLFusion, which enhances the illumination while reducing the loss of structural information, leading to a fused image with richer texture details. We first design an illumination enhancement module to separate the degraded illumination from the scene information in the visible image, and mine more details from the low-intensity regions. Based on the fact that image edge information has a good capability of modeling structures, we design an edge extraction network for low-light visible images to model the structural information, which can accurately highlight important structural information and inject it into the fusion image. The proposed method produces fusion results that not only have good visual perception, but also minimize the loss of structural information. Extensive experiments on benchmark datasets demonstrate that the proposed method outperforms state-of-the-art (SOTA) methods in terms of visual quality, quantitative metrics as well as advanced vision tasks.

Most existing hyperspectral image (HSI) classification methods primarily focus on capturing subtle spectral variations by leveraging local spectral-spatial cues derived from patch-level representations. However, limited attention has been given to exploring the global spatial contextual correlations among pixels of HSI. In this study, we propose the Superpixel Segmentation and Classification Mutual Enhancement Network (S2CMEN), a novel framework that integrates global spatial correlations with spectral information through the mutual enhancement of superpixel segmentation and classification. Specifically, a global spatial adaptive module (GSAM) is designed to obtain the direct correlation of the global classes in HSI. It consists of an Adaptive Spectral-Superpixel Network (ASSN) and a Graph Convolutional Network (GCN), forming a synergistic architecture that effectively captures global spatial relationships by adaptively deriving superpixel results from HSIs. Notably, GSAM offers a transferable global spatial representation for HSI tasks, enabling integration with other spectral feature extraction models. Furthermore, we develop a Spatial-Spectral Fusion Module (SSFM) to obtain comprehensive spectral features and fuse them with the extracted global spatial features. Finally, under the constraint of a unit loss, the Mutual Enhancement Strategy (MES) can make the superpixel segmentation loss and the classification loss mutually enhance each other for better performance. We conducted extensive experiments on three public datasets. The proposed S2CMEN achieves overall classification accuracies of 97.38%, 92.33%, and 91.38% on Indian Pines, Pavia University, and Houston, respectively, consistently surpassing existing state-of-the-art methods.

Domain adaptation (DA)-based cross-domain hyperspectral image (HSI) classification methods have garnered significant attention. The majority of DA techniques utilize models based on convolutional neural networks (CNNs) and Transformers for feature extraction. However, Transformers may struggle to capture local details in HSIs, while CNNs often underperform in handling long-range dependencies. Furthermore, many methods focus only on aligning marginal distributions while ignoring the consistency of inter-class features, which may lead to feature confusion and degraded classification accuracy. To overcome the challenges mentioned, we propose a Mamba-CNN and knowledge distillation network (MKDnet). Firstly, the network employs a feature extractor that integrates Mamba and CNN frameworks for cross-domain HSI classification, enabling the capture of both global and local features while effectively capturing long-range dependencies. Secondly, domain alignment is achieved through distribution alignment and graph alignment. In the distribution alignment phase, we design a knowledge distillation architecture that utilizes soft labels to enhance the understanding of relationships between classes, thereby improving the consistency of inter-class features. In the graph alignment phase, we use graph convolution to capture connections between nodes and edges and transfer class-level topological relationships across domains. Finally, the classifier is used to obtain classification results, with consistency constraints applied to balance features between classes more effectively. Extensive experiments have demonstrated that MKDnet outperforms other state-of-the-art methods on three public cross-domain HSI datasets.

The spread of COVID-19 has brought great disaster to the world, and automatic segmentation of the infected region can help doctors make a quick diagnosis and reduce workload. However, automatic segmentation of infected regions in computed tomography (CT) images still faces many challenges. The most central problem at the moment is that when capturing low-frequency signals and high-frequency signals, the use of Transformer blocks for capturing low-frequency signals may deteriorate the high-frequency signals (local texture information) to some extent. For this reason, in this paper, we propose a new network for automatic segmentation of COVID-19 lung infections from CT images, called NextSeg. NextSeg follows an encoder-decoder architecture, where the encoder part uses NCB and NTB convolutional attention modules to extract better features while reducing the deterioration of texture profile information. The decoder part uses an attention fusion module to learn the weight relationship between features and retain more important feature information. In addition to the difficulty of extracting the boundary details of the COVID-19 infected region as well as the loss of details the deeper the network is, we designed a new hybrid strategy. Traditional and state-of-the-art approaches to the COVID-19 segmentation domain are compared. Extensive experiments show that our proposed framework outperforms existing competitors in both qualitative and quantitative results.

The integration of multi-modality images significantly enhances the clarity of critical details for object detection. Valuable semantic data from object detection enriches the fusion process of these images. However, the potential reciprocal relationship that could enhance their mutual performance remains largely unexplored and underutilized, despite some semantic-driven fusion methodologies catering to specific application needs. To address these limitations, this study proposes a mutually reinforcing, dual-task-driven fusion architecture. Specifically, our design integrates a feature-adaptive interlinking module into both image fusion and object detection components, effectively managing the inherent feature discrepancies. The core idea is to channel distinct features from both tasks into a unified feature space after feature transformation. We then design a feature-adaptive selection module to generate features rich in target semantic information and compatible with the fusion network. Finally, effective combination and mutual enhancement of the two tasks are achieved through an alternating training process. A diverse range of swift evaluations is performed across various datasets to corroborate the potential efficiency of our framework, actualizing visible advancements in both fusion effectiveness and detection accuracy.

The fusion classification of hyperspectral image (HSI) and light detection and ranging (LiDAR) data has gained widespread attention because of its ability to obtain more comprehensive spatial and spectral information. However, the heterogeneous gap between HSI and LiDAR data also adversely affects the classification performance. Despite the excellent performance of traditional multimodal fusion classification models, language information containing much linguistic priori knowledge to enrich visual representations needs to be addressed. Therefore, we design a Spectral-Spatial-Language fusion network (S2LFNet), which can fuse visual and language features to broaden the semantic space using linguistic priori knowledge commonly shared between spectral features and spatial features. First, we propose a dual-channel cascaded image fusion encoder (DCIFencoder) for visual feature extraction and progressive feature fusion of different levels for HSI and LiDAR data. Then, three aspects of Text data are designed to extract linguistic priori knowledge using the Text encoder. Finally, contrastive learning is utilized to construct a unified semantic space, and Spectral-Spatial-Language fusion features are obtained for classification tasks. We evaluate the classification performance of the proposed S2LFNet on three datasets through extensive experiments, and the results show that it outperforms the state-of-the-art fusion classification methods.

Epilepsy is a neurological disease that occurs in all ages and seriously threatens physical and mental health. There are two problems in the present study. One is the limitation of the amount of publicly available medical data. And the other is that the distributions of the data are different but correlated. Conventional machine learning methods are not applicable. But transfer learning method has shown promising performance in solving both problems. In this paper, a multi-source domain transfer learning method called MDTL for epilepsy diagnosis is proposed. In order to fully exploit the specific features and common features of the dataset, we propose a domain specific feature extractor and a common feature extractor. For enhancing data, we transform the signals into time-frequency diagrams to rotate and crop. The three types of electrocardiogram (ECG) time-frequency diagram are put to train model, and the model is transferred to electroencephalogram (EEG) time-frequency diagrams. The results confirm that MDTL is effective in epilepsy diagnosis.

In practical hyperspectral image (HSI) classification tasks, we often encounter the problems of few-shot classification and domain misalignment between source domains and target domains. To solve this classification paradigm, a meta-learning method of few-shot learning (FSL) is usually used. However, most existing FSL methods address the problem for domain alignment and neglect the exploration of semantic relationships of objects across domains. In this paper, we propose the cross-domain spectral semantic transformer FSL (SSTFSL), which can fully extract semantic features and spectral detail features for the cross-domain few-shot HSI classification task. Specifically, the multi-head self-attention (MSA) mechanism with enhancement process (EP) of the transformer is used to map out semantically relevant local regions and can enhance the ability of the model to distinguish subtle feature differences in the spectrum. In addition, the matching degree of different branches is computed by relational network learning, which ultimately enables cross-domain few-shot HSI classification. Through extensive experiments, we evaluate the classification performance of SSTFSL on HSI datasets. The results demonstrate that SSTFSL outperforms existing FSL methods and deep learning methods on HSI classification.

Since the discovery of COVID-19 in late 2019, the viral pneumonia crisis has begun to spread rapidly around the world. Lesion segmentation can remove unnecessary background areas and help doctors diagnose the condition. However, the infected areas showed differences at different stages, and the border between the infected areas and the surrounding tissue was blurred. To solve this problem, a novel COVID-19 lung infection segmentation network (Mix-Net) is designed for the automatic identification of infected areas from chest CT slices. Specifically, first, the local and global features of the infected areas are extracted and interacted with using the mixing block. Then, the features extracted from multiple layers of the encoder are fused and connected to the decoder. Experiments show that Mix-Net outperforms most cutting-edge segmentation models and achieves good segmentation results.

Infrared and visible image fusion aims to integrate salient targets and abundant texture information into a single fused image. Existing methods typically ignore the issue of illumination, so that there are problems of weak texture details and poor visual perception in case of low illumination. To address this issue, we propose a low-light oriented infrared and visible image fusion network, named L2Fusion. In particular, we first design a decomposition network according to Retinex theory to obtain the reflectance features of a visible image with low-light. Then, these features are integrated with the features extracted from the corresponding infrared image by a residual network. The finally fused image largely eliminates the negative impact caused by low illumination, and contains both salient targets and abundant texture information. Extensive experiments demonstrate the superiority of our L2Fusion over the state-of-the-art methods, in terms of both visual effect and quantitative metrics.

In the few-shot hyperspectral image (HSI) classification, most current models don't fully utilize the advantage of spectral-spatial feature fusion, resulting in low classification accuracy. Therefore, we propose a few-shot HSI classification model with spectral-spatial feature fusion based on fuzzy broad learning system (FBLS) (FSFBLS). Firstly, we use a Gaussian filter to suppress noise while smoothing spectral features based on spatial information to achieve the first fusion of spectral-spatial features. Secondly, we use FBLS with fuzzy rules to fully model the complex mapping relationship between spectral-spatial features and HSI labels to complete HSI classification. The fuzzy processing can extract rich discriminative features to enhance the recognition of different categories. Finally, the guided filter corrects the misclassified samples of FBLS based on the guided image to achieve the second fusion of spectral-spatial features. Extensive experimental results on three public datasets demonstrate that FSFBLS achieves state-of-the-art classification performance compared to nine popular models.

With the development of deep learning in various fields, a deep learning method that can optimize multiple target tasks at the same time, that is, deep multi-task learning (MTL), has attracted more and more attention. We aim to find a multitask learning method to optimize a single goal. For image classification tasks, it is difficult to find multiple tasks with task relevance. We use an unsupervised clustering algorithm to construct multiple related auxiliary tasks in the dataset to solve this problem in order to achieve a kind of data enhancement. The purpose is to improve the accuracy of the main task. While these newly constructed auxiliary tasks may exhibit semantic features that are not relevant to the main task, in order to reduce the impact of such non-ideal auxiliary tasks on the main task, we use a multi-task learning based on learning to learn (MTL-LTL) approach with a spatially dependent attention function embedded in an underlying joint model with hard parameter sharing that allows our model to have pixelated modeling capabilities. In addition, we also use the method of learning to learn to randomly sample multiple auxiliary tasks. It can train these tasks on the shared hidden layer, and at the same time minimize the loss of the main task, and ensure that the optimization direction leads to the improvement of the main task. In order to verify the classification performance of the model, three image dataset are used for experimental analysis. The experimental results show that the model proposed in this paper is better than the current popular benchmark methods and can effectively improve the accuracy of image classification.

Many traditional semi-supervised learning algorithms not only train on the labeled samples but also incorporate the unlabeled samples in the training sets through an automated labeling process such as manifold preserving. If some labeled samples are falsely labeled, the automated labeling process will generally propagate negative impact on the classifier in quite a serious manner. In order to avoid such an error propagating effect, the unlabeled samples should not be directly incorporated into the training sets during the automated labeling strategy. In this paper, a new semi-supervised support vector machine with extended hidden features (SSVM-EHF) is presented to address this issue. According to the maximum margin principle and the minimum integrated squared error between the probability distributions of the labeled and unlabeled samples, the dimensionality of the labeled and unlabeled samples is extended through an orthonormal transformation to generate the corresponding hidden features shared by the labeled and unlabeled samples. After doing so, the last step in the process of training of SSVM-EHF is done only on the labeled samples with their original and hidden features, and the unlabeled samples are no longer explicitly used. Experimental results confirm the effectiveness of the proposed method.