Vision Transformer (ViT) has been thoroughly explored in hyperspectral image (HSI) classification (HIC). Nevertheless, current ViT-based approaches still acquire discriminative features, resulting in relatively limited generalization capabilities when confronted with the challenges posed by high intraclass variances and interclass similarities commonly observed in HSI data. Moreover, most of these methods fail to adequately emphasize the significance of the central pixel in HIC. To address the aforementioned challenges, we introduce a masked ViT (MViT) for HIC. First and foremost, MViT endeavors for the first time to introduce the masking operations in supervised models to learn more robust patterned features instead of distinguishable features, thereby bestowing it with outstanding generalization performance. Secondly, during the training phase of MViT, when conducting the random masking operations on the embedded features, we deliberately retain the embedding corresponding to the central pixel to guarantee the effectiveness of the model and emphasize the importance of the central pixel in HIC. Finally, MViT will deactivate the masking operations during the testing phase and utilize all the embedded features to accomplish the classification task, thereby enabling the model to fully exploit its recognition capabilities. On top of that, MViT is an extremely lightweight model, and by introducing the masking operations during the training phase, its training speed becomes unprecedentedly rapid. Experiments conducted on four publicly accessible datasets demonstrate that MViT can consistently achieve excellent or even the optimal classification results in comparison with the most advanced methods. The source code was powered by Jupyter and released at https://github.com/swiftest/MViT.

As the most prevalent self-supervised representation learning (SSRL) model, the masked auto-encoder (MAE) has been gradually investigated in semi-supervised hyperspectral image classification (SHIC). However, the majority of the current approaches augment MAE merely from the application perspective or by introducing a weak regularization term, and do not comprehensively consider the challenges posed by the high intraclass variances and interclass similarities that often appear in hyperspectral image (HSI) data. In this article, we present a regularized MAE (RMAE) to address the aforementioned problems. Specifically, within the framework of MAE, we introduce a self-designed induced transformer block, using a small number of visible patches to learn the embeddings of patches with larger receptive fields. The learned embeddings are used to reconstruct the corresponding patches and an induced reconstruction loss is calculated. This strategy creates a much harder task for masked image modeling (MIM), and the induced transformer block is lightweight and imposes negligible computational burden overhead the underlying MAE framework. In addition, by rethinking the masking operations, we develop a masked convolutional neural network (MCNN), uncovering the principle of MAE and affirming the efficacy of RMAE. Finally, we present two metrics: the mean intraclass distance, and the mean interclass distance. Based on the metrics we give two criteria to evaluate the performance of an SSRL model, providing a new coordinate for the research in SSRL-based SHIC. Experiments conducted on four publicly accessible datasets show that RMAE outperforms state-of-the-art methods. The source code was powered by Jupyter and released athttps://github.com/swiftest/RMAE.

Deep learning techniques have shown increasing potential in change detection (CD) in hyperspectral images (HSIs). However, most deep learning-based existing methods for HSI CD follow a patch-based local learning framework and concentrate on binary CD. In this letter, we propose an end-to-end semantic feature fused global learning (SFFGL) framework for HSI multiclass change detection (MCD). In SFFGL, a global spatial-wise fully convolutional network (FCN), which introduces a spatial attention mechanism (PAM) between encoder and decoder, is designed to effectively exploit the global spatial information from the whole HSIs and achieve patch-free inference. PAM can adaptively extract global spatial-wise feature representation. In the model training stage, a global hierarchical (GH) sampling strategy is introduced to obtain diverse gradients during backpropagation for more robust performance. The semantic-spatial feature fusion (S2F2) unit is designed to effectively fuse the enhanced spatial context information in the encoder and the semantic information in the decoder. More importantly, a semantic feature enhancement module (SEM) is proposed to weaken the influence of the unchanged regional background on the change regional foreground, thus further improving the accuracy. Experimental results on two benchmark HSI datasets demonstrate the effectiveness of the proposed SFFGL.

The very high spectral resolution in hyperspectral images (HSIs) presents an opportunity to detect subtle land-cover changes. However, availability of HSIs acquired from different platforms requires the development of change detection (CD) methods for HSIs capable to process images with different spatial resolutions. In this paper, we propose an end-to-end sub-pixel convolution-based residual network (SPCNet) to detect changes between high resolution (HR) and low resolution (LR) HSIs. First, an efficient sub-pixel convolution layer is introduced to upscale the LR feature maps into the HR one. Then, the super resolution (SR) block is designed to generate more discriminative representations in sub-pixel-based LR images. Moreover, the sub-pixel-based feature of LR image and pixel-based feature of HR image are concatenated as an input to the designed ResNet for HSI CD. Experimental results on two HSI datasets demonstrate the effectiveness of the proposed SPCNet.

Deep learning methods, especially convolutional neural network (CNN)-based methods, have shown promising performance for hyperspectral image (HSI) change detection (CD). It is acknowledged widely that different spectral channels and spatial locations in input image patches may contribute differently to CD. However, they are treated equally in existing CNN-based approaches. To increase the accuracy of HSI CD, we propose an end-to-end Siamese CNN (SiamNet) with a spectral–spatial-wise attention (SSA-SiamNet) mechanism. The proposed SSA-SiamNet method can emphasize informative channels and locations and suppress less informative ones to refine the spectral–spatial features adaptively. Moreover, in the network training phase, the weighted contrastive loss function is used for more reliable separation of changed and unchanged pixels and to accelerate the convergence of the network. SSA-SiamNet was validated using four groups of bitemporal HSIs. The accuracy of CD using the SSA-SiamNet was found to be consistently greater than for ten benchmark methods.

Deep learning-based methods (e.g., convolutional neural network (CNN)-based methods), have shown increasing potential in hyperspectral image (HSI) change detection (CD). However, the recent advances in CNN-based methods in HSI CD tasks are mostly devoted to designing more complex architectures or adding additional hand-designed blocks. This increases the number of parameters making model training difficult. In this paper, we propose an end-to-end residual self-calibrated network (RSCNet) to increase the accuracy of HSI CD. To fully exploit the spatial information, the proposed RSCNet method adaptively builds inter-spatial and inter-spectral dependencies around each spatial location with fewer extra parameters and reduced complexity. Moreover, the introduced self-calibrated convolution (SCConv) helps to generate more discriminative representations by heterogeneously exploiting convolutional filters nested in the convolutional layer. The designed RSC module can explicitly incorporate richer information by introducing response calibration operation. The experiments on four bi-temporal HSI datasets demonstrated that the proposed RSCNet method is more accurate than ten widely used benchmark methods.

The very high spectral resolution in hyperspectral images (HSIs) offers an opportunity to detect subtle land-cover changes. However, the availability of HSIs acquired from different platforms requires the development of change detection (CD) methods capable of processing HSIs with different spatial resolutions. In this paper, we propose a general end-to-end subpixel convolution-based residual network (SPCNet) to accomplish the CD task between high spatial resolution (HR) and low spatial resolution (LR) HSIs. To effectively tackle the resolution matching issue, a super resolution (SR) block with an efficient subpixel convolution layer is introduced to upscale the LR feature maps into HR maps. The subpixel convolution layer can fully explore the subpixel context information by learning an array of upscaling filters. Moreover, the designed SPC module is embedded into the LR branch to generate more discriminative representations. More importantly, the SPC module as a plug-and-play unit has the potential to be embedded into other baseline networks to enhance the feature learning capability. Experimental results on four HSI datasets demonstrate the effectiveness of the proposed SPCNet.

Researches on the classification of hyperspectral images (HSIs) based on deep learning are in full swing, especially the spectral-spatial dependent global learning (SSDGL) framework, which is both efficient and robust. However, the global convolutional long short-term memory (GCL) module under this framework fails to take full consideration of the spectral characteristics contained in HSIs, and the hierarchically balanced (H-B) sampling strategy introduced in this framework prevents the training process from converging smoothly. In this article, we develop a novel regularized spectral-spatial global learning (RSSGL) framework. Compared with SSDGL, the proposed framework mainly makes three improvements. Above all, aiming at the problem that the GCL module used in SSDGL cannot fully tap the local spectral dependence, we apply 3D convolution to the gated units of long short-term memory (LSTM) as an alternative to the GCL module for adjacent and non-adjacent spectral dependencies learning. Furthermore, to extract the most discriminative features, an improved statistical loss regularization term is developed, in which we introduce a simple but effective diversity-promoting condition to make it more reasonable and suitable for deep metric learning in HSI classification. Finally, to effectively address the performance oscillation caused by the H-B sampling strategy, the proposed framework adopts an early stopping strategy to save and restore the optimal model parameters, making it more flexible and stable. Experiments conducted on three representative data sets show that the proposed RSSGL has superior classification performance compared with the existing relatively excellent research methods. The source code is released at https://github.com/swiftest/RSSGL.