Identifying novel land cover classes in open-world scenarios is essential for achieving reliable hyperspectral image (HSI) classification. However, existing methods often depend heavily on label supervision, leading to biased feature spaces and the underutilization of unlabeled samples from unknown classes. To address these limitations, we propose Contrastive and Reciprocal Points Learning (CoRPL), a novel framework specifically designed for open-set HSI classification. In the pretraining phase, contrastive learning (CL) is employed at both instance and cluster levels to extract shared feature representations while effectively incorporating information from unlabeled samples of unknown classes. During the training phase, we introduce a Reciprocal Points Learning (RPL) strategy, which leverages three supervised losses to refine the feature space. This approach enables robust modeling of known and unknown classes, minimizing distribution overlap and enhancing open-set classification performance. Experiments on two widely-used datasets demonstrate that CoRPL consistently surpasses state-of-the-art methods across classification scenarios with varying degrees of openness.

Prompt tuning has emerged as a powerful approach in the era for foundation models, enabling efficient use of pretrained knowledge while minimizing resource demands. We introduce prompt tuning to the field of remote sensing (RS) and propose a novel prompt-based multimodal fusion framework called ProMF for RS multimodal image classification. ProMF incorporates a small number of learnable parameters into the input space, while keeping the parameters of pretrained networks frozen during model fine tuning. These additional parameters are prepended to the input sequence of each Transformer layer and trained alongside the linear classification head during fine-tuning. Furthermore, to enhance the feature interaction and fusion, we hierarchically incorporate useful prompts through a novel prompt-embedded multihead self-attention (MSA) mechanism. This approach allows for the learning of complementary representations from different modalities layer by layer, improving the model performance while reducing the risk of overfitting. Experimental results on three commonly used datasets demonstrate that the proposed method outperforms state-of-the-art approaches. The demonstrated effectiveness of multimodal prompt tuning offers a new perspective on adapting pretrained models for RS applications. The code will be publicly available at:https://github.com/zhaolin6/ProMF

Hyperspectral images (HSI) offer exceptional precision in distinguishing features due to their broad spectral dimensions. However, their high dimensionality gives rise to a phenomenon known as the ‘dimensional curse’, characterized by data sparsity in high-dimensional feature spaces. This issue is further exacerbated by the limited number of labeled samples, rendering it challenging to effectively define the decision boundary and increasing risk of over-fitting. To address the challenges, we propose a spectral-spatial representation learning framework based on hyperspectral image and light detection and ranging (LiDAR) data, which enhances the generalization of spectral features while reducing dimensionality through the optimization of spectral-wise information. Meanwhile, a local-global spatial feature fusion mechanism is designed for LiDAR spatial features to further alleviating the sparsity of spectral features and to effectively recognize complex land cover. The method fully leverages the complementary strengths of HSI and LiDAR data through self-supervised contrastive learning, effectively mitigates the challenge posed by data properties. Extensive experiments were conducted on three widely used HSI-LiDAR datasets, and the results demonstrate that the proposed algorithm outperforms state-of-art methods in classification accuracy.

Deep neural networks (DNNs) have achieved significant advancements in hyperspectral image (HSI) classification, enabling critical applications in environmental monitoring, medical imaging, and geological analysis. However, their vulnerability to adversarial attacks, particularly in the boundary regions between classes, remains a critical challenge. Existing defense methods often struggle to address these issues due to insufficient adaptability to irregular boundary patterns and larger perturbations in boundary regions. To bridge this gap, we propose a boundary-aware network (BAN), a novel adversarial defense network framework that integrates a multiscale deformable convolution (MSDC) module with a Kolmogorov–Arnold network (KAN). The MSDC module dynamically adjusts receptive fields across scales to capture discriminative spatial-spectral features, while the KAN architecture leverages its cubic spline-based smooth nonlinearity to suppress gradient-driven adversarial perturbations. By leveraging these components, BAN not only mitigates boundary-specific vulnerabilities but also enhances global robustness against diverse adversarial threats. Experimental results on three benchmark HSI datasets demonstrate that BAN outperforms state-of-the-art methods, maintaining high accuracy and robustness under various adversarial attack scenarios.

As an effective unsupervised learning method, contrastive learning (CL) has made remarkable progress in the hyperspectral image (HSI) classification. The core idea of CL is to learn representations by attracting positive samples and repelling negative samples. However, due to the patch sampling mode of HSI, the patches with the same semantic information might be undesirably considered as negative samples of each other, which are called “noisy negatives.” The noisy negatives deteriorate the performance of CL. To address the issue, a progressive CL based on noisy negatives cleaning (ProCoL) is proposed for HSI classification. In contrast to existing CL, an adjunct low-dimensional subspace is introduced. Additionally, encoder training was conceptually divided into two stages each with distinct roles. In the rough-training stage, CL is applied concurrently within two different dimensional subspaces to improve the discriminative ability of the encoder. Subsequently, as the training stabilizes, noisy negatives are gradually eliminated in the retraining stage based on the dynamically generated pseudo labels in the low-dimensional space, which further improves the latent representations of the encoder. Experiments show that the ProCoL achieves the best performance compared to the previous state-of-the-art methods.

Contrastive learning has emerged as a promising technique for hyperspectral image (HSI) classification. However, the inherent limitation of sliding window sampling in HSI results in partial samples within a mini-batch exhibiting extremely high similarity. Consequently, there is an increased number of negative sample pairs composed of similar samples, significantly reducing the effectiveness of contrastive learning. Moreover, prevailing classification models heavily depend on convolutional operations, emphasizing the extraction of local features but struggle to capture long-distance dependencies in both spatial and spectral dimensions. To address these problems and fully leverage the abundance of unlabeled samples, we propose a novel purified contrastive learning (PCL) framework for HSI classification. We design a complementary spatial-spectral representation encoder architecture that combines Convolutional Neural Network (CNN) and Transformer to capture local features and global dependencies. More importantly, a purified contrastive loss function is proposed based on super-pixel spatial prior. Extensive experiments on three public datasets demonstrate the superiority of PCL over state-of-the-art methods in HSI classification. The code for this work is available at https://github.com/zhaolin6/PCL for the sake of reproducibility.

Deep learning (DL) methods have achieved impressive performance in hyperspectral image (HSI) classification but are susceptible to adversarial attacks, which can lead to significant accuracy degradation. Although there has been encouraging progress in improving model robustness for HSI classification, the existing approaches primarily concentrate on capturing global pixel-level dependencies while overlooking the intrinsic superpixel priors of HSI—namely, spatial smoothness and spectral correlations within spatially coherent regions. In addition, these methods often lack effective noise suppression mechanisms. To address these challenges, we introduce a novel antiperturbation network, APNet, designed for robust HSI classification against adversarial attacks. APNet incorporates a noise suppression module that includes a U-shaped unit (UsU) and a feature cascade unit (FCU) to extract clear pixel-level features at multiple scales. These features are then combined with superpixel priors, which serve as robust tokens for a Transformer encoder to capture global structural features. By fusing denoised pixel-level features with coarse-grained superpixel-level features, APNet significantly enhances the robustness of feature representations and the network’s intrinsic resistance to adversarial attacks. Extensive experiments on three HSI benchmark datasets show that APNet outperforms the existing state-of-the-art techniques against across various attacks and perturbation intensities. In particular, APNet maintains stable classification performance even under high attack intensities.

Hyperspectral image (HSI) classification methods often follow an approach of patch-based learning framework. Recently, an image-based global deep learning framework has gained increasing attention for HSI classification tasks due to faster inference speed. However, such a framework exhibits deteriorated performance in modeling features on the region level while balancing local spatial structure information. In this letter, we propose a global learning method that includes graph-guided transformer (G2T) as the core tool. First, we extract pixel level features by convolution block and obtain an undirected graph by segmentation on superpixel scales for an input HSI. Then, to model global and local correlations among nodes of superpixels, a mechanism of graph-guided self-attention (G2SA) is developed and implemented. Finally, pixel level features integrated with superpixel features at regional level are used to generate classification results for the HSI. Experimental results demonstrate that the method of G2T outperforms state-of-the-art methods in classification accuracy and inference speed, in particular in the case of limited labeled sample. The source code for this work will be available at https://github.com/zhaolin6/G2T.

The transformer framework has shown great potential in the field of hyperspectral image (HSI) classification due to its superior global modeling capabilities compared to convolutional neural networks (CNNs). To utilize the transformer to model spatial–spectral information, a hybrid transformer that integrates multigranularity tokens and spatial–spectral attention (SSA) is proposed. Specifically, a token generator is designed to embed the multigranularity semantic tokens, which contributes richer image features to the model by exploiting CNN’s local representation capability. Moreover, a transformer encoder with an SSA mechanism is proposed to capture the global dependencies between different tokens, enabling the model to focus on more differentiated channels and spatial locations to improve the classification accuracy. Ultimately, adaptive weighted fusion is applied to different granularity transformer branches to boost HybridFormer’s classification performance. Experiments were conducted on four new challenging datasets, and the results indicate that HybridFormer achieves state-of-the-art results in terms of classification performance. The code of this work will be available athttps://github.com/zhaolin6/HybridFormerfor the sake of reproducibility.

Visual object tracking plays a fundamental role in computer vision. Extracting the unique spectral and spatial features of hyperspectral images (HSIs) can significantly improve tracking performance in complex scenarios, especially in hyperspectral object tracking. However, due to the limited training samples, handcrafted features are employed in most current hyperspectral trackers, although they cannot sufficiently describe the intrinsic nature of the object. This letter proposes a band regrouping and response-level fusion network (BRRF-Net) for hyperspectral object tracking based on deep transfer learning, employing a deep model trained on color videos to represent features to solve this problem. Specifically, a new band regrouping subnetwork that generates the band weights using hyperspectral feature information is proposed. The bands are divided into several groups using band weights and imported into the Siamese network. Finally, the response-level fusion strategy is adopted to integrate the tracker results for the precise location of objects. Experiments on hyperspectral video reveal that the accuracy of the BRRF-Net is up to 0.689, which is the state-of-the-art performance compared with the current hyperspectral object trackers and proves the effectiveness and superiority of the BRRF-Net.

Hyperspectral image (HSI) classification is an active research topic in remote sensing. Supervised learning-based methods have been widely used in HSI classification tasks due to their powerful feature extraction capabilities for cases of sufficiently labeled samples. However, practical applications often have limited samples with accurate labels due to the high cost of labeling or unreliable visual interpretation. We introduce a contrastive self-supervised learning (SSL) algorithm to achieve HSI classification for problems with few labeled samples. First, a new HSI-specific augmentation module is developed to generate sample pairs. Then, a contrastive SSL model based on Siamese networks is used to extract features from these easily accessible sample pairs. Finally, the labeled samples are taken to fine-tune the parameters of the classification model to boost classification performance. Tests of the contrastive self-supervised algorithm have been performed on two widely used HSI datasets. The experimental results reveal that the proposed algorithm requires a few labeled samples to achieve superior performance.

Hyperspectral image (HSI) data have large numbers of bands that probably not all bands are equally informative and predictive for an effective HSI classification. Effective algorithms are highly desired in many real-world HSI applications, especially in cases requiring rapid learning with limited computing power. To address the abovementioned case, we present in this article a novel plug-and-play compact band weighting (CBW) module based on the attention-driven mechanism that evaluates different spectral bands according to their contributions to a given classification task. Compared to existing band weighting (BW) modules with tens of thousands of network parameters by deep learning, the proposed CBW is a lightweight module with only 20 parameters. Both model complexity and time cost are significantly reduced. The CBW module implements BW by making full use of the correlation among the adjacent spectral bands and spectral statistic information and, thereby, leads to the effect of recalibrated HSI. The experimental study has been conducted on three widely used HSI data sets, and results show the superiority of the proposed algorithm over current state-of-the-art methods of BW. The source code is available athttps://github.com/JarvenYi/CBW.