Advanced interpretation of hyperspectral remote sensing images benefits many precise Earth observation tasks. Recently, visual foundation models have promoted the remote sensing interpretation but concentrating on RGB and multi-spectral images. Due to the varied hyperspectral channels, existing foundation models would face image-by-image tuning situation, imposing great pressure on hardware and time resources. In this paper, we propose a tuning-free hyper-spectral foundation model called HyperFree, by adapting the existing visual prompt engineering. To process varied channel numbers, we design a learned weight dictionary covering full-spectrum from 0.4 ∼ 2.5 μm, supporting to build the embedding layer dynamically. To make the prompt design more tractable, HyperFree can generate multiple semantic-aware masks for one prompt by treating feature distance as semantic-similarity. After pre-training HyperFree on constructed large-scale high-resolution hyperspectral images, HyperFree (1 prompt) has shown comparable results with specialized models (5 shots) on 5 tasks and 11 datasets. Code and dataset are accessible at https://rsidea.whu.edu.cn/hyperfree.htm.

Hyperspectral anomaly detection (HAD) aims to find small targets deviating from surroundings in an unsupervised manner. Recently, various deep models have been applied to HAD, such as autoencoder series and generative adversarial networks (GAN) series, which mainly use a proxy task, i.e., iteratively reconstructing low-frequency components (backgrounds) to separate anomalies (two-step paradigm). However, in such an unsupervised manner, most deep HAD model is trained and tested on the same image. Since the learned low-frequency background varies from image to image and the trained model cannot be directly transferred to unseen images. In this paper, the one-step detection paradigm is first proposed, where the model is optimized directly for the HAD task and can be transferred to unseen datasets. The one-step paradigm is optimized to identify the spectral deviation relationship according to the anomaly definition. Compared to learning the specific background distribution in the two-step paradigm, the spectral deviation relationship is universal for different images and guarantees transferability. Further, we instantiated the one-step paradigm as an unsupervised transferred direct detection (TDD) model. To train the TDD model in an unsupervised manner, an anomaly sample simulation strategy is proposed to generate numerous pairs of anomaly samples. A global self-attention module and a local self-attention module are designed to help the model focus on the “spectrally deviating” relationship. The TDD model was validated on six public datasets. The results show that TDD is superior to the recent two-step methods in detection and transferability aspects.

Positive and unlabeled (PU) learning is aimed at building a binary classifier to distinguish the target from the background using only the known positive samples, which is an advanced solution for the hyperspectral target detection (HTD) task. However, when PU learning (PUL) meets complex hyperspectral scenarios, there are two main challenges: 1) How to estimate the class prior accurately? The class prior, i.e., the target proportion, is an important prior for PUL to learn the discriminant boundary, but it is difficult to estimate in hyperspectral imagery, due to the interclass spectral similarity and 2) How to remove redundancy and improve the discriminative features of the target? The diagnostic spectral feature extraction is important for the weakly supervised PUL models as it can help with separating the target from the background. In this article, to tackle these challenges, a robust PUL framework with key band selection (PU-KBS) is proposed, which is modeled as an end-to-end and class prior free PUL framework, where the accurate class prior and the most discriminative key band subset are jointly initialized and iteratively updated until reaching the optimal result by evolutionary search. Meanwhile, a deep PUL detector is introduced for guiding the subsequent search direction and discriminative deep feature extraction. The proposed PU-KBS framework was verified using different hyperspectral datasets, where accurate class prior estimation, diagnostic spectral characteristics, and robust detection results could be obtained simultaneously by the PU-KBS framework. Furthermore, the improvement in band selection interpretability and detection performance was proven experimentally.

Anomaly segmentation in high spatial resolution (HSR) remote sensing imagery is aimed at segmenting anomaly patterns of the earth deviating from normal patterns, which plays an important role in various Earth vision applications. However, it is a challenging task due to the complex distribution and the irregular shapes of objects, and the lack of abnormal samples. To tackle these problems, an anomaly segmentation model based on pixel descriptors (ASD) is proposed for anomaly segmentation in HSR imagery. Specifically, deep one-class classification is introduced for anomaly segmentation in the feature space with discriminative pixel descriptors. The ASD model incorporates the data argument for generating virtual abnormal samples, which can force the pixel descriptors to be compact for normal data and meanwhile to be diverse to avoid the model collapse problems when only positive samples participated in the training. In addition, the ASD introduced a multi-level and multi-scale feature extraction strategy for learning the low-level and semantic information to make the pixel descriptors feature-rich. The proposed ASD model was validated using four HSR datasets and compared with the recent state-of-the-art models, showing its potential value in Earth vision applications.

Deep neural networks have shown huge potential in hyperspectral unmixing (HU). However, the large function space increases the difficulty of obtaining the optimal solution with limited unmixing data. The autoencoder-based blind unmixing methods are sensitive to the hyperparameters, and the optimal solution can be difficult to obtain. Algorithm unrolling, which integrates deep learning and iterative algorithms, can shrink the search space and improve the efficiency of obtaining optimal results. Based on this, a model-driven deep neural network named the group sparsity regularized unmixing unrolling (GSUU) network, which unrolls a regularized matrix factorization objective function for blind HU, is proposed in this paper. Based on the nonnegative matrix factorization (NMF) optimization rules, the GSUU network contains two sub-networks—the A-Block and the S-Block—for alternately and iteratively estimating the optimal endmember spectra and abundance maps. The GSUU method incorporates the spatial group sparsity prior of the abundances, i.e., the fact that spatially adjacent mixed pixels share similar sparse abundances, into a deep unrolling network. The experimental results obtained with both synthetic and real hyperspectral data illustrate that the proposed algorithm can obtain a superior accuracy, compared to the other state-of-the-art unmixing algorithms.

Hyperspectral anomaly detection is aimed at detecting observations that differ from their surroundings, and is an active area of research in hyperspectral image processing. Recently, autoencoders (AEs) have been applied in hyperspectral anomaly detection; however, the existing AE-based methods are complicated and involve manual parameter setting and preprocessing and/or postprocessing procedures. In this article, an autonomous hyperspectral anomaly detection network (Auto-AD) is proposed, in which the background is reconstructed by the network and the anomalies appear as reconstruction errors. Specifically, through a fully convolutional AE with skip connections, the background can be reconstructed while the anomalies are difficult to reconstruct, since the anomalies are relatively small compared to the background and have a low probability of occurring in the image. To further suppress the anomaly reconstruction, an adaptive-weighted loss function is designed, where the weights of potential anomalous pixels with large reconstruction errors are reduced during training. As a result, the anomalies have a higher contrast with the background in the map of reconstruction errors. The experimental results obtained on a public airborne data set and two unmanned aerial vehicle-borne hyperspectral data sets confirm the effectiveness of the proposed Auto-AD method.

Hyperspectral anomaly detection is aimed at detecting observations that differ from their surroundings. To achieve this goal, low-rank models and autoencoders (AEs) have attracted a lot of attention. Although the low-rank model is self-explainable, a low-rank prior may not completely match real data. In contrast, AEs can automatically learn the discriminative features between anomalies and background, whereas AEs are not self-explainable. In this article, a deep low-rank prior-based method (DeepLR) is proposed, which combines a model-driven low-rank prior and a data-driven AE. To be specific, the low-rank prior and a fully convolutional AE architecture are incorporated through modeling an energy minimization problem solved by an iterative optimization framework, in which low-rank background estimation and network training serve as two subproblems. The low-rank background is input into the network to calculate a low-rank regularized loss, constraining the training of the network. Finally, the background can be approximately reconstructed, while the anomalies are reconstructed with significant reconstruction errors; thus, the reconstruction errors indicate the anomalous degree. The experimental results obtained on several public datasets and two large unmanned aerial vehicle (UAV)-borne datasets confirm the merit and viability of the proposed method.

Anomaly detection is an active area of research in hyperspectral information processing. Recently, low-rank representation has been applied in hyperspectral anomaly detection. However, the existing low-rank-based methods either involve a complicated dictionary construction process or the anomaly-background separation which is not sufficient. In this article, to solve these problems, a novel hyperspectral anomaly detection method based on a locally enhanced low-rank prior (LELRP-AD) is proposed. This article is inspired by the observation that, in local homogeneous regions, the background signals hold an enhanced low-rank property while the anomalies exhibit spatial sparsity. Based on this observation, the background pixels can be low-rank reconstructed by a set of basis background signals, whereas anomalies can be represented as sparse residuals. First, image segmentation is performed to enhance the homogeneity of the background, in which a Potts-based image segmentation algorithm is adopted with postprocessing, thus avoiding the need for a complicated spectral dictionary for the representation of the background. Furthermore, the original hyperspectral data matrix is augmented with extracted background endmembers for the low-rank and sparse matrix decomposition, to further achieve anomaly-background separation. The experimental results obtained on four real hyperspectral data sets demonstrate the merit and viability of the proposed method compared with the current state-of-the-art methods.