With the gradual maturity of deep learning technology and its extensive application in the field of remote sensing, hyperspectral image classification technology has made tremendous progress. The existing methods can achieve excellent classification performance in a closed-set environment (CSE), where the class distribution of the training and test set is consistent. However, in the open environment of the real world, many samples in the test set belong to the classes that never appear in the training set. Therefore, when unknown samples are present, existing closed-set classification methods will predict them as one of the known classes and cannot identify the unknown samples. Open Set Recognition (OSR) aims to solve this problem by simultaneously identifying unknown classes and distinguishing known classes. In this paper, we propose a reconstruction-enhanced prototype network (RePro) for HSI open-set classification. Specifically, the prototypes are constructed for each class in the embedding space, and the classification loss is also introduced to make the samples closer to the prototype of each class. Then the reconstruction loss based onL1is adopted to fully exploit the spectral-spatial information of training samples to provide the auxiliary information to identify the samples of unknown classes. Finally,the reconstruction loss and prototype distance are combined to identify samples of unknown classes. Experiments on multiple datasets demonstrate the effectiveness of the proposed method.

Radar intrapulse signal modulation classification is an important work for the electronic countermeasure and there are mainly two categories of algorithms. The deep learning-based algorithms usually outperform the traditional feature extraction-based ones, but they may rely on massive labeled samples for training, which limits their practical applications. To solve this problem, the SS-LWCNN model which combines the semisupervised learning (SI-SL) with virtual adversarial training (VAT) and the light weight technology is proposed. VAT provides the proposed model with robustness to the local perturbation of samples, which improves the classification accuracy with limited labeled samples provided. The light weight technology greatly reduces the complexity of the model, which increases the speed of classification. As demonstrated by the simulation results, in the condition of limited labeled samples are available, the SS-LWCNN model obtains greater classification accuracy compared to the other models. As tested by both the white Gaussian noise and the impulsive noise affected signals data sets, the SS-LWCNN model shows stronger robustness than the comparable models. Furthermore, the SS-LWCNN model contains much fewer training parameters and less floating point operations (FLOPs) than the other models.

Recently, self-supervised learning (SSL) has gained great prominence in hyperspectral image classification (HSIC) due to its powerful capability to alleviate data-hunger problem. The generative-based method and the contrastive-based method have become two main streams in the field of SSL. To fully integrate the merits of both of them, we propose an efficient hybrid SSL method, that is, mask-enhanced contrastive learning (MECL). Essentially, MECL remains a prototypical contrastive-learning (CL) method, but incorporates the masking-and-predicting idea of the generative-based method. When the sample gradually matches the prototypes, feature reconstruction is implicitly performed in MECL step by step. Furthermore, we design a spatial-spectral multimasking mechanism for hyperspectral data and also propose two complementary strategies to prevent MECL from collapsing. To demonstrate the performance of MECL, a wide range of experiments are carried out in our work. On the one hand, the results of ablation experiments prove that HSIC does benefit from the fusion of SSL methods. On the other hand, the classification results on three hyperspectral datasets confirm that our MECL is an effective SSL method with high stability and strong reliability.

Recent years, in order to solve the problem of lacking accurately labeled hyperspectral image data, self-supervised learning has become an effective method for hyperspectral image classification. The core idea of self-supervised learning is to define a pretext task which helps to train the model without the labels. By exploiting both the information of the labeled and unlabeled samples, self-supervised learning shows enormous potential to handle many different tasks in the field of hyperspectral image processing. Among the vast amount of self-supervised methods, contrastive learning and masked autoencoder are well known because of their impressive performance. This article proposes a Transformer based masked autoencoder using contrastive learning (TMAC), which tries to combine these two methods and improve the performance further. TMAC has two branches, the first branch has an encoder-decoders structure, it has an encoder to capture the latent image representation of the masked hyperspectral image and two decoders where the pixel decoder aims to reconstruct the hyperspectral image at pixel-level and the feature decoder is built to extract the high-level feature of the reconstructed image. The second branch consists of a momentum encoder and a standard projection head to embed the image into the feature space. Then, by combining the output of feature decoder and the embedding vectors via contrastive learning to enhance the model’s classification performance. According to the experiments, our model shows powerful feature extraction capability and gets outstanding results on hyperspectral image datasets.

Hyperspectral image classification is a classic topic aiming to assign the category label of each pixel in hyperspectral images. Some deep learning methods have been introduced and achieved good results, such as the CNN-based architecture, which focuses on local and hierarchical feature extraction to obtain visual information from shallow to deep. Recently, Transformer has been applied to the visual field and also used in the hyperspectral image classification task. Some work applied Transformer to process the spectral information but cannot achieve good results. To optimize the results, a new strategy called co-learning is proposed through a dual-architecture ensemble. The samples selected by the dual-architecture network are iteratively added to increase more reliable training samples. CNN and Transformer use completely different methods to extract features from different views and have great diversity. Experimental results show that this method is better than the algorithm using only a single network.

Hyperspectral imagery (HSI) classification is a widely used method in remote sensing, which can provide accurate label information for each pixel. Though the classification accuracy is very high for many publicly available data sets in many research articles, they often exhibit much worse performance in practical applications. Because most of the articles adopt a random sampling strategy to select training and test samples from the same image, the high correlation between training and test samples will bring optimistic results. However, this strategy is not suitable for practical application. Because the training and test samples are collected from different locations in most situations, in this letter, the nonoverlapped sampling is adopted to reduce the correlation between training and test samples. Four key factors are presented to analyze the HSI classification; then, a new deep mutual-teaching method is proposed to classify the HSI. The experimental results show that the performance of the proposed method outperforms comparison methods.

For hyperspectral imagery (HSI) classification, most of the studies focus on how to improve the classification accuracy, while the influence of sampling strategy for classification performance attracts little attention. For now, random sampling (RS) is the most adopted strategy. That is, for a hyperspectral image, a certain number of labeled samples are randomly selected as the training set, and the remaining labeled samples are taken as the test set. However, the RS strategy will produce over optimistic results when used for performance evaluation because of the overlap between training set and test set. Though spectral-spatial classification methods benefit most from the RS strategy, the pixel-wise classification methods can also benefit from it because of the high spectral correlation between training and test samples. However, in practical applications, the RS strategy is not feasible. Because the training and test samples are often collected from different locations. In this situation, the correlation between training and test samples will decrease dramatically and the performance of HSI classification methods will be affected. In this article, a nonoverlapped sampling method is adopted to reduce the correlation between training and test samples and different classic classification methods are evaluated. Experimental results show that the classification performance of all methods drops a lot when nonoverlapped sampling strategy is adopted. After the analysis of some important factors for HSI classification, we also propose a cotraining-based classification method to relief the influence of sampling strategy and obtains much better performance compared with those classic spectral-spatial classification methods.

Band selection is an important step in efficient processing of hyperspectral images (HSIs), which can be seen as the combination of powerful band search technique and effective evaluation criterion. The existing deep-learning-based methods make the network parameters sparse to search the spectral bands using threshold-based functions or regularization terms. These methods may lead to an intractable optimization problem. Furthermore, these methods need to repeatedly train deep networks for evaluating candidate band subsets. In this article, we formalize hyperspectral band selection as a reinforcement learning (RL) problem. Band search is regarded as a sequential decision-making process, where each state in the search space is a feasible band subset. To evaluate each state, a semisupervised convolutional neural network (CNN), called EvaluateNet, is constructed by adding the intraclass compactness constraint of both limited labeled and sufficient unlabeled samples. A simple stochastic band sampling method is designed to train EvaluateNet, making it possible to efficiently evaluate without any fine-tuning. In RL, new reward functions are defined by taking the EvaluateNet and the penalty of repeated selection into account. Finally, advantage actor–critic algorithms are designed to explore in the state space and select the band subset according to the expected accumulated reward. The experimental results on HSI data sets demonstrate the effectiveness and efficiency of the proposed algorithms for hyperspectral band selection.

Supervised machine learning and deep learning methods perform well in hyperspectral image classification. However, hyperspectral images have few labeled samples, which make them difficult to be trained because supervised classification methods rely heavily on sample quantity and quality. Inspired by the idea of self-supervised learning, this article proposes a hyperspectral imagery classification algorithm based on contrast learning, which uses the information of abundant unlabeled samples to alleviate the problem of insufficient label information in hyperspectral data. The algorithm uses a two-stage training strategy. In the first stage, the model is pretrained in the way of self-supervised learning, using a large number of unlabeled samples combined with data enhancement to construct positive and negative sample pairs, and contrastive learning (CL) is carried out. The purpose is to enable the model to make judgments on positive and negative samples. In the second stage, based on the pretrained model, the features of the hyperspectral image are extracted for classification, and a small amount of labeled samples are used to fine-tune the features. Experiments show that the features extracted by self-supervised learning achieved improved results on downstream classification task.

Band selection has been widely utilized in hyperspectral image (HSI) classification to reduce the dimensionality of HSIs. Recently, deep-learning-based band selection has become of great interest. However, existing deep-learning-based methods usually implement band selection and classification in isolation, or evaluate selected spectral bands by training the deep network repeatedly, which may lead to the loss of discriminative bands and increased computational cost. In this article, a novel convolutional neural network (CNN) based on bandwise-independent convolution and hard thresholding (BHCNN) is proposed, which combines band selection, feature extraction, and classification into an end-to-end trainable network. In BHCNN, a band selection layer is constructed by designing bandwise 1×1 convolutions, which perform for each spectral band of input HSIs independently. Then, hard thresholding is utilized to constrain the weights of convolution kernels with unselected spectral bands to zero. In this case, these weights are difficult to update. To optimize these weights, the straight-through estimator (STE) is devised by approximating the gradient. Furthermore, a novel coarse-to-fine loss calculated by full and selected spectral bands is defined to improve the interpretability of STE. In the subsequent layers of BHCNN, multiscale 3-D dilated convolutions are constructed to extract joint spatial-spectral features from HSIs with selected spectral bands. The experimental results on several HSI datasets demonstrate that the proposed method uses selected spectral bands to achieve more encouraging classification performance than current state-of-the-art band selection methods.

Deep neural networks have achieved excellent performance in computer vision and many other fields. However, this good result is at the cost of high model complexity and expensive hardware requirement. Hence, it is an important topic to keep the performance of the deep model with as few parameters as possible. Inspired by the knowledge distillation (KD) method, a hyperspectral imagery (HSI) classification method based on compressed neural network is proposed in this letter. Different from the original KD method, virtual samples are adopted to describe the classification boundary of the teacher network more accurately, which effectively improves the classification accuracy of the student network. Experimental results of three real HSI data sets show that the superiority of the proposed method to the state-of-the-art algorithms in terms of classification accuracy and model complexity.

The intrinsic noise of synthetic aperture radar (SAR) images has a big influence to the image processing performance, especially in change detection (CD). Image denoising is an important branch of image restoration which aims at enhancing the quality of images. The detection accuracy of CD depends greatly on the quality of red difference image (DI), therefore image denoising can be regarded as a vital step in SAR CD. However, few researches focused on this problem. In this study, an end‐to‐end deep denoising model is first designed to remove the noise of SAR images. With the help of abundant simulated SAR images, deep denoising model is trained effectively to estimate the noise component. Then clean image can be achieved by removing this noise component from the original SAR image. After denoising, the new image pair will generate a clean DI. At last, DI is classified into changed and unchanged areas by a three‐layer Convolutional Neural Network (CNN). Three real SAR image pairs demonstrate the effectiveness of the proposed method.

In recent years, deep learning methods have been widely used for the classification of hyperspectral images (HSIs). However, the training of deep models is very time-consuming. In addition, the rare labeled samples of remote sensing images also limit the classification performance of deep models. In this letter, a simple deep learning model, a rotation-based deep forest (RBDF), is proposed for the classification of HSIs. Specifically, the output probability of each layer is used as the supplement feature of the next layer. The rotation forest is used to increase the discriminative power of spectral features and neighboring pixels are used to introduce spatial information. The RBDF consumes much less training time than traditional deep models. Experimental results based on three HSIs demonstrate that the proposed method achieves the state-of-the-art classification performance. In addition, the RBDF obtains satisfied classification results with very few training samples.

Generative adversarial networks (GANs) are famous for generating samples by training a generator and a discriminator via an adversarial procedure. For hyperspectral image classification, the collection of samples is always difficult. However, directly applying GAN to hyperspectral image classification exists two problems. One is that the generated samples lack discriminative information. Meanwhile, the discriminator has no discriminative ability for multiclassification. Another is that spatial and spectral information requires to be considered in hyperspectral image classification simultaneously. To address these problems, a novel multiclass spatial-spectral GAN (MSGAN) method is proposed. In MSGAN, two generators are devised to generate the samples containing spatial and spectral information, respectively, and the discriminator is devised to extract joint spatial-spectral features and output multiclass probabilities. Moreover, novel adversarial objectives for multiclass are defined. The discriminator is devised to predict training samples belonging to true classes and generated samples belonging to all the classes with the same probability. The generators are devised to make the discriminator mistake. By adversarial learning between the discriminator and generators, the classification performance of the discriminator is promoted with the assistance of discriminative generated samples. Experimental results on hyperspectral images demonstrate that the proposed method achieves encouraging classification performance compared with several state-of-the-art methods, especially with the limited training samples.

Although it is a powerful feature selection algorithm, the wrapper method is rarely used for hyperspectral band selection. Its accuracy is restricted by the number of labeled training samples and collecting such label information for hyperspectral image is time consuming and expensive. Benefited from the local smoothness of hyperspectral images, a simple yet effective semisupervised wrapper method is proposed, where the edge preserved filtering is exploited to improve the pixel-wised classification map and this in turn can be used to assess the quality of band set. The property of the proposed method lies in using the information of abundant unlabeled samples and valued labeled samples simultaneously. The effectiveness of the proposed method is illustrated with five real hyperspectral data sets. Compared with other wrapper methods, the proposed method shows consistently better performance.

Most of the existing spatial-spectral-based hyperspectral image classification (HSIC) methods mainly extract the spatial-spectral information by combining the pixels in a small neighborhood or aggregating the statistical and morphological characteristics. However, those strategies can only generate shallow appearance features with limited representative ability for classes with high interclass similarity and spatial diversity and therefore reduce the classification accuracy. To this end, we present a novel HSIC framework, named deep multiscale spatial-spectral feature extraction algorithm, which focuses on learning effective discriminant features for HSIC. First, the well pretrained deep fully convolutional network based on VGG-verydeep-16 is introduced to excavate the potential deep multiscale spatial structural information in the proposed hyperspectral imaging framework. Then, the spectral feature and the deep multiscale spatial feature are fused by adopting the weighted fusion method. Finally, the fusion feature is put into a generic classifier to obtain the pixelwise classification. Compared with the existing spectral-spatial-based classification techniques, the proposed method provides the state-of-the-art performance and is much more effective, especially for images with high nonlinear distribution and spatial diversity.

Recently, very high resolution (VHR) panchromatic and multispectral (MS) remote-sensing images can be acquired easily. However, it is still a challenging task to fuse and classify these VHR images. Generally, there are two ways for the fusion and classification of panchromatic and MS images. One way is to use a panchromatic image to sharpen an MS image, and then classify a pan-sharpened MS image. Another way is to extract features from panchromatic and MS images, respectively, and then combine these features for classification. In this paper, we propose a superpixel-based multiple local convolution neural network (SML-CNN) model for panchromatic and MS images classification. In order to reduce the amount of input data for the CNN, we extend simple linear iterative clustering algorithm for segmenting MS images and generating superpixels. Superpixels are taken as the basic analysis unit instead of pixels. To make full advantage of the spatial-spectral and environment information of superpixels, a superpixel-based multiple local regions joint representation method is proposed. Then, an SML-CNN model is established to extract an efficient joint feature representation. A softmax layer is used to classify these features learned by multiple local CNN into different categories. Finally, in order to eliminate the adverse effects on the classification results within and between superpixels, we propose a multi-information modification strategy that combines the detailed information and semantic information to improve the classification performance. Experiments on the classification of Vancouver and Xi’an panchromatic and MS image data sets have demonstrated the effectiveness of the proposed approach.

In order to alleviate the subsequent computation burden and storage requirement, band selection has been widely adopted to reduce the dimensionality of hyperspectral images, and the current methods mainly consist of the supervised and the unsupervised. Although these supervised methods have better performance, those unsupervised methods dominate the band selection field. In this letter, based on the unique properties of hyperspectral images, we propose a very simple but effective supervised band selection algorithm based on the local spatial information of the hyperspectral image and wrapper method. By using both the information of labeled and unlabeled pixels of the hyperspectral image, our proposed algorithm consistently outperforms the classical wrapper method. We use five widely used real hyperspectral data to demonstrate the effectiveness of our proposed algorithms. We also analyze the relationship between our band selection algorithm and the well-known Markov random field classifier.