Hyperspectral unmixing (HU) aims to decompose mixed pixels into their constituent spectral signatures and estimate their corresponding fractional abundances. Recently, the nonlinear spectral mixing model (NLMM) has advanced significantly and offered strong physical interpretability. However, the effective integration of physics-driven NLMM with data-driven deep learning (DL) approaches still remains a critical challenge. To address this, we propose a diffusion-enhanced autoencoder (DEAE), a novel unsupervised framework that innovatively incorporates the diffusion model (DM) into nonlinear HU. DEAE introduces the residual second-order attention mechanism to capture global spectral information, adaptively weighting informative bands while compressing redundant bands. Subsequently, we integrate the extended multilinear mixing model (EMLM) into the DM-enhanced decoder, which extracts latent features from the linear autoencoder’s output and generates an enhanced reconstructed image while simultaneously estimating the transition probabilities of EMLM. Finally, a nonlinear decoder outputs the ultimate reconstructed image based on both the enhanced reconstructed image and transition probabilities. Experiments conducted on synthetic and three real-world datasets demonstrate the superior performance of DEAE compared to the state-of-the-art methods based on both LMM and NLMM.

The challenges in semantic segmentation of remote sensing images arise primarily from significant variations in target sizes, diverse shapes, and complex background environments. To address these challenges, we have developed a feature-guided semantic segmentation model for remote sensing images, further enhancing its accuracy with high-fidelity masks generated by the SAM vision large model. The principal contributions of this study are outlined as follows: We introduce a semantic segmentation model that employs feature guidance, ingeniously integrating CNN and Transformer architectures to effectively capture both the local details and the global context of remote sensing images. We propose an improved method based on the SAM vision large model, featuring a dual-branch structure that incorporates two key modules: semantic voting and mask refinement. The semantic voting module corrects semantic errors in both branches. Subsequently, a mask refinement module is utilized to further enhance these masks. In experimental validations using the iSAID dataset, the model significantly improved the mean Intersection over Union (mIoU) to 61.54% and the meanF1 score to 74.69%, underscoring the significance and applicability of the proposed methodology.

The availability of spectral library makes hyperspectral sparse unmixing an attractive unmixing scheme, and the powerful feature extraction capability of deep learning meets the requirements of estimating abundances with hundreds of channels in sparse unmixing. However, few related researches have been carried out. In this letter, we propose a window transformer convolutional autoencoder (WiTCAE) to address the sparse unmixing problem. In our method, a well-designed transformer encoder for hyperspectral images is applied before convolutional neural network (CNN), aiming at exploring non-local information by a new attention mechanism called window-based pixel-level multihead self-attention (WP-MSA). Three consecutive CNN blocks focus on further joint spatial-spectral feature extraction, and adjust the number of channels to the number of endmembers contained in the spectral library. Moreover, CNN establishes the connections among windows, and smooths out the discontinuities caused by window partition. The decoder is a convolutional layer with the kernel size of 1, and its weights are fixed to a known spectral library. Comparative experiments on both simulated and real datasets confirm the superiority of our proposed network.

Hyperspectral unmixing (HSU), which aims to identify constituent materials and estimate the corresponding proportions in a scene, is an essential research topic in remote sensing. Most deep learning-based methods are data-inspired, relying on massive amounts of data to train black-box-like networks. While a few model-inspired unmixing networks only consider the spectral features of the pixel, ignoring the exploration of spatial information between pixels. In this paper, we design a network topology according to the classical iterative algorithm, and the large number of learnable parameters contained in the network are continuously updated through data fitting. In other words, we integrate the concepts of both model-driven and data-driven and propose a deep interpretable fully convolutional neural network (DIFCNN). The iteration of the classic sparse unmixing algorithm is unfolded to provide guidance for the network structure and incorporate prior knowledge into the network. Meanwhile, two-dimensional (2D) convolutional layers are employed to automatically learn the spatial information at different scales. A known spectral library is used as a prior to initialize network parameters and reconstruct the image. The DIFCNN adopts an end-to-end training strategy, in addition, we establish a new loss function that adds a joint sparse constraint on the abundance result to the cross-entropy loss. Experiments on both synthetic and real datasets show that the performance of the DIFCNN not only outperforms the SUnSAL and its improved algorithms, but also is highly competitive in the state-of-the-art methods of deep learning.

Due to the Hughes phenomenon, hyperspectral image (HSI) classification under small sample size situation is still a key challenging problem. To alleviate this issue, we propose a novel spectral–spatial prototype learning-based nearest neighbor classifier (SSPLNN) for HSI in this article. The local spectral–spatial neighbor set is first constructed for each sample based on both spectral similarity and spatial structural context to accurately explore the local spectral–spatial information. Then, a spectral–spatial prototype learning model is designed to learn a set of spectral–spatial prototypes, which can optimally utilize both the similarity and variance of samples within each spectral–spatial set and excavate the unseen spectral–spatial variations. The learned spectral–spatial prototypes offer more complementary information to improve the classification accuracy remarkably under small sample size situation. In addition, a linear discriminative projection is simultaneously learned to make each test local spectral–spatial set to be optimally classified to the same class with its nearest neighbor (NN) spectral–spatial prototype set in the projected target subspace. Finally, the NN classifier based on measuring the minimum geometric distance between the projected test spectral–spatial set and the projected spectral–spatial prototype sets is employed to determine the label. Experimental results demonstrate that the proposed SSPLNN method outperforms several well-known classification methods by a large margin on three widely analyzed HSI datasets.

To improve the classification accuracies of hyperspectral images (HSIs), especially when using a limited number of training samples, a novel superpixel-based multiple statistical feature extraction (SPMSFE) method is proposed in this article. For each dimension-reduced pixel obtained by maximum noise fraction (MNF), the most similar superpixel-based neighbors of different sizes are first identified based on the spatial structures of the HSIs to exploit contextual spatial information accurately. Then, multiple statistical features, including the mean, covariance descriptor, and the Gaussian feature, are extracted for the set of superpixel-based neighbors to fully explore the spatial geometry information, tight correlations between different spectral bands, and spatial–spectral variations from different perspectives, respectively. In addition, these three statistical features of the pixels share the same size and can be utilized for uniform classification without any dimensionality obstacles even though the sizes of the superpixel-based neighbors for different pixels may be different. Next, we construct multiple kernels to map these multiple statistical features in the Euclidean and Riemannian manifold spaces to a uniform Hilbert space and embed them into a multitask kernelized sparse representation classification (MTKSRC) model. The constructed MTKSRC model provides a natural method to effectively fuse the multiple statistical features for excellent classification performance and robustness, especially when using limited numbers of training samples. The experimental results for three widely used HSI data sets demonstrate that the classification accuracy of the proposed SPMSFE method outperforms several latest and state-of-the-art classification methods by a large margin.

Conventional low-rank (LR)-based hyperspectral image (HSI) denoising models generally convert high-dimensional data into 2-D matrices or just treat this type of data as 3-D tensors. However, these pure LR or tensor low-rank (TLR)-based methods lack flexibility for considering different correlation information from different HSI directions, which leads to the loss of comprehensive structure information and inherent spatial-spectral relationship. To overcome these shortcomings, we propose a novel multidirectional LR modeling and spatial-spectral total variation (MLR-SSTV) model for removing HSI mixed noise. By incorporating the weighted nuclear norm, we obtain the weighted sum of weighted nuclear norm minimization (WSWNNM) and the weighted sum of weighted tensor nuclear norm minimization (WSWTNNM) to estimate the more accurate LR tensor, especially, to remove the dead-line noise better. Gaussian noise is further denoised and the local spatial-spectral smoothness is preserved effectively by SSTV regularization. We develop an efficient algorithm for solving the derived optimization based on the alternating direction method of multipliers (ADMM). Extensive experiments on both synthetic data and real data demonstrate the superior performance of the proposed MLR-SSTV model for HSI mixed noise removal.