Currently, hyperspectral and multispectral image fusion methods based on local and global feature learning (e.g., CNN and Transformer) have achieved promising results. However, as the core part of transformer, the computational cost of the self-attention is quadratic with the image size, which severely limits its practical application. In this paper, we propose a spatial-spectral cross mamba network (SSCM) for hyperspectral and multispectral image fusion. By using the mamba structure, our model is able to obtain long-range spatial-spectral information with less computational complexity in comparison with the transformer structure. Specifically, we introduce a spatial-spectral cross mamba block to facilitate the interaction between hyperspectral and multispectral features, effectively enhancing the spatial-spectral feature representation ability of the network. In addition, a cross-scale spatial-spectral learning module based on the U-shaped structure is proposed to effectively extract the long-range high-frequency feature information at different scales. Extensive experimental results demonstrate that our method achieves comparable performance in comparison with some state-of-the-art image fusion methods.

Hyperspectral image (HSI) target detection is an advanced technology within the remote sensing society, widely applied in both civilian and military domains. However, subpixel target detection poses a significantly challenging problem in HSI processing. Existing methods for HSI target detection mostly rely on constructing detectors based on linear mixing model (LMM). Nevertheless, due to multiple scattering, LMM fails to capture the nonlinear features and extensive interactions among various materials commonly present in real HSIs, potentially resulting in poor performance. To address this issue, this work proposes a novel method for HTD based on bilinear mixing model (BMM) in HSI. Specifically, we designed a spectral decomposition network to separate the background and target. To further extract and combine spectral and spatial information of the image for more accurate learning of target and background distributions, a deep network model based on a multiscale feature extraction module is proposed. Finally, to enhance the discrimination between targets and backgrounds, a customized CEM loss is formulated by minimizing the response of the background and maximizing the response of the target. Experimental comparisons on six real HSI datasets validate the superiority of the proposed detector.

Multi-class change detection from hyperspectral image (HSI) leverages the rich spectral information of HSIs to detect and classify subtle changes of interest in an imaged scene. However, challenges arise due to limited samples in small categories, which hinder the accurate differentiation of changes. This study proposes a spatial-spectral feature-enhanced Mamba and SAM-guided hyperspectral multi-class change detection (SFMS) method. To address the challenges, a tri-plane gated Mamba is designed to complement spatial information by utilizing the abundant spectral information in HSIs. Additionally, frequency domain features are combined with state space models, enabling the detection of more accurate semantic and texture changes using integrated information from frequency domains. This approach effectively mitigates the problem of inaccurate detection in small-sample categories. Furthermore, the segment anything model (SAM) is adapted, with the features of change areas being enhanced through prior knowledge obtained from segmentation, thereby improving the multi-class change detection accuracy. The experimental results demonstrate that the proposed SFMS method outperforms state-of-the-art techniques, achieving superior multi-class change detection while overcoming the challenges associated with detecting small-sample categories.

Images captured under low-light conditions suffer from poor visibility and clarity due to insufficient light. The emergence of deep learning has greatly boosted the development of low-light enhancement techniques and achieved promising results. However, while these low-light enhancement methods have enhanced the perceptual effects of human vision, their results in high-level visual tasks (e.g., object detection and semantic segmentation) are still unstable and even sometimes bring negative effects. Therefore, in this work, we propose a new model, KANformer, which uses a semantic-gradient prior as a guide to recover pixels relevant to the image subject from both high-frequency and low-frequency perspectives. Specifically, our model consists of three key components: Low-Frequency Enhancement (LFE) module, which aims to enhance the restoration of the image subject via the semantic prior obtained from SAM; Low-Frequency-Based High-Frequency Enhancement (LFHE) module, which utilizes the KAN module to obtain information from the low-frequency features conducive to the enhancement of high-frequency features; and Gradient-Based High-Frequency Enhancement (GHE) module, which aims to utilize the original gradient as prior to further enhance the structural information of the image and reduce the effect of noise. In addition, we introduce the discrete wavelet transform as down-sampling method while transforming the spatial domain features to the frequency domain for processing. Experiments on multiple paired and unpaired datasets show that our method achieves better visualization and image fidelity compared to other state-of-the-art methods. In addition, experiments on object detection and segmentation show that our method provides better enhancement in improving low-light high-level vision tasks.

The classification of land cover based on spectral-spatial joint features is a focus in the current domain of hyperspectral image (HSI) classification. However, existing methods might not fully leverage both spectral and spatial characteristics. Additionally, approaches that separately extract and later fuse spectral and spatial features often encounter challenges with suboptimal fusion outcomes. To address these issues, we propose a spectral-spatial dual-branch cross-enhanced transformer method. This approach first utilizes distinct shallow convolutional modules tailored to the attributes of spectral and spatial data for feature extraction. Subsequently, a cross-attention strategy is designed to better align spectral and spatial features and smoothly merge these features through attention operations. Furthermore, by introducing the fused spectral-spatial features, we extend the traditional multi-head self-attention (MHSA) mechanism. This richness in feature sequences aids the model in better capturing long-range dependencies between features. The experimental results on two benchmark datasets validate the superiority of the proposed method.

Abstract Deep convolutional neural networks based remote sensing change detection has recently shown significant performance improvement. However, small region changes and global‐local features in high‐resolution remote sensing images are not fully explored. This paper introduces a hybrid U‐shaped and transformer network for change detection in high‐resolution remote sensing images. Specifically, a UNet++‐based backbone to facilitate feature learning across different scales. In addition, we introduce a transformer‐based feature fusion module for extracting long‐range dependencies, which can enhance the representation ability of the network. Furthermore, the introduced efficient channel attention mechanism can efficiently calibrate the feature representation and concentrate on more important feature information. Thanks to the above designs, the proposed method enjoys a strong ability to extract local and global features for remote sensing change detection. Extensive experimental results on different remote sensing images show that our method can achieve superior performance in comparison with state‐of‐the‐art change detection methods.

In recent years, deep learning has shown promising results in change detection (CD) of hyperspectral images (HSIs). However, these algorithms often require a large number of labeled samples, which can be costly in practical applications. This article proposes a few-shot HSI CD method based on a 3-D convolution neural network and normalizing flow. To mitigate the issue of limited training samples, we develop a low-parameter baseline CD model utilizing 3-D convolution neural networks to extract spectral-spatial features while preventing overfitting. In addition, the proposed method analyzes the spectral-spatial distribution of hyperspectral variation by normalizing flow and generates hyperspectral tensor samples that approximate the distribution, thus enhancing CD performance. Furthermore, to improve the sample generation speed, we employ soft labels and label smoothing techniques to assign high-quality labels to the generated samples, thereby increasing the number of available samples. The method is evaluated on three datasets, and the experimental results demonstrate the efficacy of the proposed approach for detecting changes with limited training samples. The proposed method offers a promising solution to the challenge of detecting changes with limited training samples in HSIs.

The detection of oriented objects in remote sensing images remains a daunting challenge due to their complex backgrounds, various sizes, and especially arbitrary orientations. However, most of the existing methods only model the structural features of the images in the spatial domain, while the horizontal convolution kernels limit the model’s ability to perceive object direction information. Furthermore, the frequency features contain rich information about scale, texture, and angle, which can be a good complement to the spatial features. Inspired by this, we propose a multiscale spatial-frequency domain network (MSFN) to utilize spatial-frequency information for oriented object detection, which can be integrated into any convolutional neural network (CNN) architectures seamlessly and perform end-to-end training easily. Firstly, multiscale Haar wavelet transforms are leveraged to extract the multiscale frequency domain features from the image. Subsequently, channel alignment feature fusion module (CA-FFM) is proposed to fuse the high-level semantic features extracted by CNN with the low-level texture features extracted by the wavelet transform in multiscale. Finally, a channel self-attention (CSA)-based spatial-frequency feature perception module (SFPM) is designed to perform self-attention weighted aggregation on the fused features along the channel dimension, thereby constructing a novel spatial-frequency feature extraction backbone network for oriented object detector in remote sensing images. Experimental results on the DOTA and HRSC2016 datasets validate the effectiveness and universality of the proposed method.

Change detection (CD) is a hot issue in the field of remote sensing. Hyperspectral images (HSI) contain rich spectral information, and have gradually become an important data source in CD. Spectral-spatial combination is a commonly used strategy for suppressing the influence of noise on the spectrum. However, it is difficult to find a feature space that allows both spectral and spatial features to be optimally expressed. Therefore, this letter proposes a bidirectional reconstruction coding network and enhanced residual network for HSI CD (i.e., BRCN-ERN) based on the strategy of completely extracting spectral and spatial features separately and then fusing them together. In the spectral module, we use the spectrum of unchanged pixels at two time points to construct a bidirectional reconstruction network, and use the reconstruction error as a new source of spectral features. In the spatial module, we use advanced band selection algorithms to filter the bands with good spatial information, and design an enhanced two-dimensional residual network to extract the spatial features of the change tensor. Finally, the obtained spectral and spatial feature vectors are fused and inputted into the fully connected classification network to obtain the final CD map. Real HSI experiments show that our proposed BRCN-ERN has a better CD effect and is more effective than most existing algorithms.

In this letter, using the sparse unmixing framework, a weighted collaborative sparse and$L_{1/2}$low-rank regularization with superpixel segmentation method is proposed for hyperspectral unmixing. The method outlined here first uses superpixel segmentation to obtain local homogeneous regions. The reason for this approach is that the shape and size of superpixels are adaptive, which are better for obtaining homogeneous regions than square patches. Next, the weighted collaborative sparse term and$L_{1/2}$low-rank regularization were utilized to exploit the spatial and spectral correlation of each superpixel. In addition, the smoothness between adjacent pixels is enforced by total variation regularization. Finally, the proposed method and several state-of-the-art methods were tested on two simulated data sets and two real data sets. The results demonstrate the superiority of the method proposed here.

Locality-preserving projection (LPP) has been widely used in feature extraction. However, LPP does not use data category information and uses the${L}_{2}$-norm for distance measurement, which is highly sensitive to outliers. In this paper, we consider the LPP weight matrix from a supervised perspective and combine the low-rank regression method to propose a new model to discover and extract features. By using the${L}_{2,1}$-norm to constrain the loss function and the regression matrix, not only is the sensitivity to outliers reduced but the low-rank condition of the regression matrix is also restricted. Then, we propose a solution to the optimization problem. Finally, we apply the method to a series of face databases, handwriting digital datasets and palmprint datasets to test the performance, and the experimental results show that this method is effective compared with some existing methods.

The traditional methods of hyperspectral image (HSI) classification are based on the sufficient labeled data. In real life, we often encounter that the target domain corresponding to the classification task has only a small amount of labeled data, but the source domain has enough labeled data. However, the distribution of the source domain is different from the distribution of the target domain. Thus, the labeled data of the source domain cannot be applied to the target domain directly. This paper proposes a new method to solve the cross-domain few shot problem of HSI classification. In the proposed method, the local spatial alignment and the spectral alignment are simultaneously introduced to transfer the knowledge from the source domain to the target domain. Besides, to extract the domain specific features, we balance the domain invariant features and the domain specific features by a weakly parameter-shared mechanism. The two modules together can narrow the distance between two domains and make the model perform well on the target domain. Experiments conducted on four different target domain data sets demonstrate the effectiveness of our method.

Recently, deep convolutional neural network-based low-resolution hyperspectral image (LR-HSI) and high-resolution multispectral image (HR-MSI) fusion methods have achieved significant performance improvement. However, the rich spatial and spectral information in HSIs is not fully explored. In this article, we propose a novel cross-scale octave network (CSONet) for hyperspectral and multispectral image fusion. Specifically, we adopt a progressive image fusion structure to effectively extract the spatial and spectral information of HR-MSI at multiple resolutions, thereby efficiently complementing LR-HSI’s information. In addition, the proposed cross-scale octave convolution module can extract rich multiscale spatial feature information and concentrate on more important spatial–spectral features at different scales with the multiscale spatial–spectral attention mechanism. Finally, a multisupervised loss function is used to improve the gradient propagation and enhance the representation ability of the network. Ablation analysis on the benchmark datasets shows the effectiveness of each component in the proposed method. Extensive experimental results on different hyperspectral images demonstrate that the proposed CSONet can achieve superior results and strong generalization ability in comparison with some state-of-the-art LR-HSI and HR-MSI fusion methods.

Anomaly detection (AD) has become a hot topic in hyperspectral image (HSI) analysis. Anomalies are samples that are significantly different from the surrounding background in space or spectrum. However, the rich spectral-spatial features in HSI are not fully discovered by most traditional AD methods. In this paper, a 3D-convolutional Variational Au-toencoder (3D-CVAE) based AD method is proposed to make full use of the spectral-spatial information. The spectral-spatial features are extracted by the 3D-CVAE encoder and the background is reconstructed using these features through 3D-CVAE decoder. The residual between the original input and the reconstructed background contains the anomalies which can be easily detected by the Reed-Xiaoli(RX) detector in the residual. Experimental results on two HSI datasets demonstrate the advantage of the proposed method.

Classification plays an important role in the field of hyperspectral image (HSI) remote sensing. In this letter, a novel multiscale superpixel kernel-based low-rank representation (MSKLRR) classifier is proposed for HSI classification. A multiscale superpixel segmentation method is first used to generate several homogeneous regions at different scales. Then, the multiscale superpixel spectral-spatial kernel (SSK) is generated using the radial basis function (RBF) kernel on the multiscale superpixels. Finally, the multiscale superpixel kernel is integrated into a low-rank representation (LRR) to generate the MSKLRR classifier for HSI classification. The experimental results with two widely used HSIs suggest an advantage of the proposed method over other classical classification methods.

Spatial-spectral kernel (SSK) has proven to be one of state-of-the-art tools for producing precise classification results for hyperspectral images (HSIs). However, how to exactly identify the neighborhood pixels within a given cubic patch of HSI is one of the critical tasks for constructing an accurate spatial-spectral kernel (SSK). In this paper, a novel low-rank component induced SSK (LRCISSK) method is proposed to deliver more accurate classification results for HSI. It explores the low-rank properties within each HSI patch in spectral domain to adaptively identify the precise neighborhood pixels with regards to the centroid pixel. Then, the neighborhood pixels associated with the centroid pixel are embedded into the SSK framework to easily map the spectra into the nonlinear complex manifolds and enable the support vector machine (SVM) classifier to effectively discriminate them. Experiments on Indian Pines and Pavia University datasets demonstrate the superiority of the proposed LRCISSK classifier when compared to other state-of-the-art approaches.

Resolution in medical images is limited by diverse physical, technological and economical considerations. In conventional medical practice, resolution enhancement is usually performed with bicubic or B-spline interpolations, strongly affecting the accuracy of subsequent processing steps such as segmentation or registration. In this paper, we propose a coupled dictionary learning approach for super resolution of medical images, in which canonical correlation analysis (CCA) is applied to construct a common feature space. Then a pair of coupled dictionaries are learned on the derived space. At last, we seek a sparse representation for each patch of the low-resolution input, and use the sparse coefficients to generate the high-resolution output. The experimental results show that the proposed method is competitive or even superior to the other state-of-the-art SR methods.

Target detection plays an important role in the field of hyperspectral image (HSI) remote sensing. In this letter, a novel matched subspace detector based on low-rank regularized least squares (LRLS-MSD) is proposed for hyperspectral target detection. As pixels in an HSI have global correlation and can be represented in subspace, the low-rank regularization is introduced in the least squares model. An effective algorithm is presented to solve the problem. Then, the detection results are generated according to the generalized likelihood ratio test with statistical hypotheses. The experimental results suggest an advantage of the low-rank regularization over other classical target detection methods.

In this paper, we propose a new method for hyperspectral image (HSI) classification using multi-layer superpixel graph and loopy belief propagation. A merging algorithm using graph based representation of image is applied to generate multi-scale superpixels in hyperspectral image at first. Then, we build a multi-layer superpixel graph and use loopy belief propagation to transmit messages between the superpixels and compute beliefs at each superpixel in our multi-layer graph for HSI classification. Experimental results with real hyperspectral data set demonstrate that our proposed method provides good performance and is competitive with some of the best available spectral-spatial methods for hyperspectral image classification.

The authors propose a fast and effective solution for automatic segmentation of white matter lesions by using T1 and fluid‐attenuated inversion recovery (FLAIR) image modalities with no need for manual segmentation and atlas registration. Initially, a brain tissue segmentation method is used to segment the T1 image into cerebrospinal fluid (CSF), grey matter and white matter. Based on the obtained tissue segmentation results, the region of interest (ROI) of the FLAIR image is created by subtracting the CSF from the FLAIR image. Subsequently, the authors calculate the z ‐score of the intensities in the ROI and define a threshold to perform a preliminary identification of abnormalities from normal tissues. The abnormalities obtained at this stage are used as the prior knowledge for the modified level‐set technique. The proposed level set method here is applied based on local Gaussian distribution to precisely detect the boundaries of the white matter lesions in the ROI. The level set method based on local Gaussian distribution fitting energy is robust to the intensity inhomogeneity of MR data and therefore capable of precisely extracting the boundaries of white matter lesions. Experimental analysis and quantitative comparisons with the peak‐seeking and state‐of‐the‐art white matter lesion segmentation (WMLS) techniques demonstrate that the algorithm is a stable and effective approach which significantly outperforms other trusted solutions for white matter lesion segmentation.