Hyperspectral image super-resolution aims to reconstruct high-quality spatial-spectral cubes from RGB images. However, the limited spectral coverage of RGB inputs hinders the simultaneous modeling of spatial structures and spectral relationships. To address this limitation, we propose a differential low-rank adaptive spatial-spectral transformer (DiLAST). Initially, a differential operator is employed to enhance RGB features in a bottom-up manner, explicitly amplifying subtle inter-channel differences. The enhanced features are then fed into a U-shaped backbone, which integrates three complementary modules for joint spatial-spectral modeling. Specifically, a center spatial-spectral attention (CSSA) module employs cross-attention mechanisms to capture local-to-global dependencies across both spatial and spectral domains; an adaptive cross-scale fusion (ACF) module utilizes learnable gating weights to establish dynamic interaction pathways between shallow high-frequency details and deep semantic representations; and a low-rank spectral calibration (LRSC) module exploits low-rank matrix priors to reveal low-dimensional manifold structures among spectral bands, thereby enhancing spectral consistency. By leveraging the synergistic effects of spatial non-locality, global spectral correlation, and low-rank properties, the proposed DiLAST achieves PSNR improvements of 33.82 dB, 36.03 dB, and 37.01 dB on benchmark datasets. Moreover, the accuracy and practical applicability of the reconstructed spectra have been effectively validated in remote sensing scenarios and object tracking tasks. The code is accessible at https://github.com/renqi1998/DiLAST.

Multispectral and hyperspectral image fusion (MHIF) seeks to combine high-resolution multispectral images (HR-MSIs) with low-resolution hyperspectral images (LR-HSIs) to create high-resolution hyperspectral images (HR-HSIs). Transformer-based architectures have recently become prominent in MHIF tasks due to their effective global self-attention mechanisms. However, the quadratic computational complexity of the global self-attention in Transformers presents significant challenges for practical applications. In this paper, we propose an enhanced spatial-frequency synergistic (ESFS) approach that leverages both spatial and frequency domain features to enhance fusion quality. Our ESFS framework introduces the condensed spatial augmentation module (CSAM), which condenses window features and employs cross-attention to balance extensive contextual understanding and detailed local feature extraction while reducing computational overhead. Additionally, we develop the selective frequency decomposition module (SFDM), which utilizes global filters composed of phase and amplitude information in the frequency domain to retain features, effectively capturing deep frequency domain characteristics and their interdependencies. Comprehensive experiments on three benchmark MHIF datasets demonstrate that our method achieves superior performance, establishing a new state-of-the-art (SOTA) in both quantitative metrics and visual quality assessments. The code is available at http://szu-hsilab.com/.

Hyperspectral images (HSIs) are vital for scene analysis, as they capture detailed spatial and spectral information to characterize surface materials. However, accurate HSI classification is challenged by significant intra-class spectral variability and spatial complexity. To address this, we leverage the fact that pixels of the same class typically form irregular local regions. We propose a structure-adaptive graph transformer (SAGT) that dynamically captures irregular spatial topologies and homogeneous spectral information to achieve adaptive HSI representation and precise classification. Specifically, a structure-aware self-attention (SASA) module is developed to embed graph structures into the self-attention mechanism as a robust positional indicator, which can be extended easily and effectively. SASA comprehensively accounts for the spatial structures and spectral autocorrelation of ground objects, facilitating the aggregation of homogeneous spectral information for noise-robust spectral representations. Additionally, a structure-adaptive pooling (SAP) module is designed to dynamically adjust graph structures by discarding irrelevant edges, thus better indicating spatial relationships. By coupling the SASA and SAP modules, our proposed SAGT model significantly alleviates spectral variability and tolerates prior noise. Furthermore, data augmentation techniques of random discard and random offset are built, which randomly drop and shift graph nodes to generate more diverse samples during preprocessing. In postprocessing, multiview decision-making integrates results from multiple contextual views to provide more robust predictions. Experimental results on three benchmark datasets consistently demonstrate that SAGT is more effective and reliable than other state-of-the-art methods. To facilitate reproduction, we will release the source code for SAGT at https://github.com/ShuGuoJ/SAGT.git.

Super-resolution is vital for the quality improvement of hyperspectral images (HSIs) under the spatial and spectral resolution trade-off. However, deep learning HSI super-resolution approaches typically adopt the “one model and one scale” scheme that is inefficient in training and storing. This is difficult in maximizing orbit equipment performance and aligning multiple spatial resolution data in remote sensing. Therefore, this article intends to address HSI arbitrary-scale super-resolution, enabling the scaling of HSIs to arbitrary sizes using a single model. To do this end, we treat HSI arbitrary-scale super-resolution as a retrieval problem. It conceptualizes the HSI as a dictionary of pixelwise tokens with spatial-spectral features, position information, and scale information. Its objective is to employ a set of initialized tokens related to the high-resolution (HR) HSI as queries to retrieve matched spectral features from low-resolution (LR) one, which is so-called token-based query-to-spectrum. Since these query tokens can be constructed flexibly (e.g., through random initialization), we can generate a desired number of them to reconstruct our HR HSI, thus achieving arbitrary-scale super-resolution. This process considers not only position information but also spectral features so that it can decrease spectral distortion. With the above idea, we developed an HSI arbitrary-scale super-resolution method, dubbed as spectral-query transformer (SQformer). Specifically, it begins by converting the LR HSI into a dictionary of LR tokens and then constructs a desired number of HR tokens. To enable flexible token construction, we design an implicit spectral token (particularly a learnable vector) and replicate it$\alpha H \times \alpha W$times to form the HR tokens. Next, the HR and LR tokens are passed into a transformer decoder to find the most matched spectral response for the former by soft-weighting the LR tokens. Finally, the HR tokens are spatially rearranged in order, forming an HR HSI. Extensive experiments have demonstrated its effectiveness on remote sensing data. The code will be released at:https://github.com/ShuGuoJ/SQformer.git.

With the development of hyperspectral sensors, accessible hyperspectral images (HSIs) are increasing, and pixel-oriented classification has attracted much attention. Recently, graph convolutional networks (GCNs) have been proposed to process graph-structured data in non-Euclidean domains and have been employed in HSI classification. But most methods based on GCN are hard to sufficiently exploit information of ground objects due to feature aggregation. To solve this issue, in this article, we proposed a graph-in-graph (GiG) model and a related GiG convolutional network (GiGCN) for HSI classification from a superpixel viewpoint. The GiG representation covers information inside and outside superpixels, respectively, corresponding to the local and global characteristics of ground objects. Concretely, after segmenting HSI into disjoint superpixels, each one is converted to an internal graph. Meanwhile, an external graph is constructed according to the spatial adjacent relationships among superpixels. Significantly, each node in the external graph embeds a corresponding internal graph, forming the so-called GiG structure. Then, GiGCN composed of internal and External graph convolution (EGC) is designed to extract hierarchical features and integrate them into multiple scales, improving the discriminability of GiGCN. Ensemble learning is incorporated to further boost the robustness of GiGCN. It is worth noting that we are the first to propose the GiG framework from the superpixel point and the GiGCN scheme for HSI classification. Experiment results on four benchmark datasets demonstrate that our proposed method is effective and feasible for HSI classification with limited labeled samples. For study replication, the code developed for this study is available at https://github.com/ShuGuoJ/GiGCN.git.

Hyperspectral cameras capture electromagnetic information within hundreds of narrow spectral bands, producing hyperspectral images (HSIs) with the capability to accurately characterize the attribute information of objects. However, mixed noise induced by instrument and atmospheric effects hinders the interpretations and applications of the HSIs. In this paper, we propose a novel subspace representation based mixed noise removal method for hyperspectral images via Robust Subspace Estimation and weighted Group Sparsity constraint (RoSEGS). An outlier detection method is proposed to effectively detect sparse noise and replace the sparse noise with new estimates. A subspace estimation strategy, which is robust to mixed noise, is proposed. The subspace is first estimated after sparse noise detection and then optimized iteratively. In addition to the introduction of a state-of-the-art denoiser based on the plug-and-play technique to exploit self-similarity characteristics of the eigen-images, we impose a weighted group sparse regularization on the eigen-images to better promote the group sparsity of the spatial differences between the eigen-images, which further improves the denoising performance. We performed extensive experiments on two simulated and two real HSIs to fully demonstrate the effectiveness of the proposed method in comparison with seven state-of-the-art competitors.

The necessary prerequisite for effective data fusion is the strict registration of low-resolution hyperspectral images (LR-HSI) and high-resolution multispectral images (HR-MSI). However, registration requires a complex process that takes into account the effects of light, imaging angle, and geometric distortion of the image during acquisition. Therefore, to avoid complex registration, we focused on developing an unregistered HSI and MSI fusion method for pixel shifting, obtaining fused images with high resolution, high signal-to-noise ratio, and feature identifiability. We identified that the unregistered LR-HSI and HR-MSI in the case of pixel shift are very similar to the disparity maps in stereo vision. Inspired by this, we simulate the structure of stereo cameras to propose a stereo cross-attention network (SCANet) to achieve an accurate fusion of unregistered LR-HSI and HR-MSI. Considering the model complexity and computing efficiency, we design a simple and stackable stereo cross-fusion block (SCFBlock) based on a Transformer to simulate the process of light entering the left and right cameras by extracting the abstract features of the images. Moreover, the purpose of cross-convergence fusion self-attention (CCFSA) is to learn cross-complementary attention and collect contextual information in horizontal and vertical directions to fuse unregistered images using multi-directional cross-view information. We have conducted extensive experiments on Pavia University (PaviaU), Chikusei, and PYLake datasets. The results show that SCANet achieves superior or competitive performance in fusing unregistered LR-HSI and HR-MSI in comparison with the other competitors.

Hyperspectral image (HSI) classification based on deep learning is a hot research topic. The convolutional model employs a single rectangular window to interpret the sample neighborhood features, whereas effective characterization of the complex spatial structure of HSI is still an unsolved problem. In this article, we propose a structure-adaptive convolutional neural network (SACNN) for HSI classification, which efficiently exploits the intrinsic spatial geometry information. Four novel strategies are designed to construct the proposed SACNN network. First, superpixel homogeneous region (SHR) sample generation is introduced to achieve neighborhood features within the intercepted rectangular window of the superpixel. Second, online batch-wise standardization uses zero padding to unify the size of inputs in the same batch, thereby realizing parallel processing of irregular inputs. Third, structure-adaptive convolution (SConv) and structure-adaptive average pooling (SAP) are correspondingly constructed to extract deep spectral, spatial, and geometric features from the effective mapping area of superpixels, and further aggregate the information within irregular boundaries. Finally, a sample-adaptive loss weight (SLW) scheme is designed to adjust the influence of different labels on the same input. Experimental results show that the overall classification accuracy of SACNN reaches 93.11%, 90.96%, and 85.04% for 15 randomly selected training samples per class on three HSI datasets, respectively, obtaining an improvement of 0.97%–2.97% with respect to the best-compared method.

With the rapid development of deep convolutional neural networks (CNNs), super-resolution (SR) in hyperspectral image (HSI) has achieved good results. Current methods generally use 2-D convolution for feature extraction, but they cannot effectively extract spectral information. Although 3-D convolution can better characterize feature structure of HSI, it will lead to parameter redundancy, model complexity, and severe memory shortage. To address the above problems, we propose a new HSI SR method, named diffused CNN (DCNN). Specifically, spectral convolutions have been added into the enhanced convolutional neural (ECN) block, and a series of spectral convolutions are introduced in the residual network to learn features in the channel direction of different depths. Furthermore, histogram of oriented gradient (HOG) and local binary pattern (LBP) are used to retain the shape and texture information of the image, respectively, which can well represent the spatial structure of the object. To effectively make use of the extracted shallow and deep features, a feature fusion strategy is used to reinforce the reconstruction efficiency. Besides, an image enhancement module has been developed to diffuse the SR image into the image space. Extensive evaluations and comparisons show that our DCNN approach can not only recover the HSI data with richer details but also achieve superiority over several state-of-the-art methods.

Spatial resolution is a crucial indicator for measuring the quality of hyperspectral imaging (HSI) and obtaining high-resolution (HR) hyperspectral images without any auxiliary information has become increasingly challenging. One promising approach is to use deep-learning (DL) techniques to reconstruct HR hyperspectral images from low-resolution (LR) images, namely super-resolution (SR). While convolutional neural networks are commonly used for hyperspectral image SR (HSI-SR), they often lead to unavoidable performance degradation due to the lack of long-range dependence learning ability. In this article, we propose a dual self-attention Swin transformer SR (DSSTSR) network that utilizes the ability of the shifted windows (Swin) transformer in the spatial representation of both global and local features and learns spectral sequence information from adjacent bands of HSI. Additionally, DSSTSR incorporates an image denoising module using the wavelet transformation method to mitigate the impact of stripe noise on HSI-SR. Our extensive experiments using publicly close-range datasets demonstrate that DSSTSR outperforms other state-of-art HSI-SR methods in terms of three image quality metrics. Furthermore, we applied DSSTSR to the SR of satellite hyperspectral images and achieved improved classification results. Compared to its competitors, DSSTSR exhibits superior performance in enhancing spatial resolution while preserving spectral information. These results suggest that the DSSTSR network has great potential for standardization in remote-sensing image processing and practical applications.

Haze in hyperspectral images (HSIs) can lead to crosstalk between multiple bands, resulting in errors that can be amplified and transmitted during data processing. As a consequence, this may cause a reduction in the accuracy and precision of remote sensing data. The purpose of haze removal is to restore high-quality HSIs from degraded ones. The high spectral resolution and typically dozens to hundreds of spectral bands in HSIs pose significant challenges for haze removal. Thus, many methods designed for natural and multispectral images are not effective in removing haze in HSIs. To address this challenge, we develop a model called asymmetric attention convolution network (AACNet) designed for haze removal in HSIs. Specifically, the basic architecture of AACNet is mainly composed of several residual asymmetric attention groups (RAAGs), where the core components are residual asymmetric attention blocks (RAABs). This design enables the full utilization of deep spatial-spectral features while skipping low-frequency regions and focusing more on the haze-affected areas. To more accurately restore the spectral information in areas polluted by haze, a pooling channel self-attention (PCSA) module has been proposed. This module can effectively reconstruct the spectral response curve that is affected by the haze. Our experiments on both simulated and real datasets demonstrate that the proposed AACNet outperforms several leading haze removal methods in both precision and visual quality. The source code and data of this article will be made publicly available at https://github.com/SZU710/AACNet for reproducible research.

Fusing a Hyperspectral image (HSI) and a multispectral image (MSI) from different sensors is an economic and effective approach to get an image with both high spatial and spectral resolution, but localized changes between the multiplatform images can have negative impacts on the fusion. In this letter, we propose a novel sparsity constrained fusion method (SCFus) to fuse multiplatform HSIs and MSIs based on matrix factorization. Specifically, we imposed$\ell _{1}$norm on the residual term of the MSI to account for the localized changes between the hyperspectral and MSIs. Furthermore, we plugged a state-of-the-art denoiser, namely block-matching and 3-D filtering (BM3D), as the prior of the subspace coefficients by exploiting the plug-and-play framework. We refer to the proposed method as SCFus for hyperspectral and MSIs. Experimental results suggest that the proposed fusion method is more effective in fusing hyperspectral and MSIs than the competitors.

The high spectral resolution of hyperspectral images (HSIs) generally comes at the expense of low spatial resolution, which hinders the application of HSIs. Fusing an HSI and a multispectral image (MSI) from different sensors to get an image with the high spatial and spectral resolution is an economic and effective approach, but localized spatial and spectral changes between images acquired at different time instants can have negative impacts on the fusion results, which has rarely been considered in many fusion methods. In this article, we propose a novel group sparsity constrained fusion (GSFus) method to fuse hyperspectral and MSIs based on matrix factorization. Specifically, we imposed$\ell _{2,1}$norm on the residual term of the MSI to account for the localized interimage changes occurring during the acquisition of the hyperspectral and MSIs. Furthermore, by exploiting the plug-and-play framework, we plugged a state-of-the-art denoiser, namely block-matching and 3-D filtering (BM3D), as the prior of the subspace coefficients. We refer to the proposed fusion method as GSFus method. We performed fusion experiments on two kinds of datasets, i.e., with and without obvious localized changes between the HSIs and MSIs, and a full resolution dataset. Extensive experiments in comparison with seven state-of-the-art fusion methods suggest that the proposed fusion method is more effective on fusing hyperspectral and MSIs than the competitors.

With the development of hyperspectral imaging technology, hyperspectral images (HSIs) have become important when analyzing the class of ground objects. In recent years, benefiting from the massive labeled data, deep learning has achieved a series of breakthroughs in many fields of research. However, labeling HSIs requires sufficient domain knowledge and is time-consuming and laborious. Thus, how to apply deep learning effectively to small labeled samples is an important topic of research in HSI classification. To solve this problem, we propose a semisupervised Siamese network that embeds Siamese network into a semisupervised learning scheme. It integrates an autoencoder module and a Siamese network to, respectively, investigate information in a large amount of unlabeled data and rectify it with a limited labeled sample set, which is called 3DAES. First, the autoencoder method is trained on the massive unlabeled data to learn the refinement representation, creating an unsupervised feature. Second, based on this unsupervised feature, limited labeled samples are used to train a Siamese network to rectify the unsupervised feature to improve feature separability among various classes. Furthermore, by training the Siamese network, a random sampling scheme is used to accelerate training and avoid imbalance among various sample classes. Experiments on three benchmark HSI datasets consistently demonstrate the effectiveness and robustness of the proposed 3DAES approach with limited labeled samples. For study replication, the code developed for this study is available athttps://github.com/ShuGuoJ/3DAES.git.

Due to the detailed spectral information through hundreds of narrow spectral bands provided by hyperspectral image (HSI) data, it can be employed to accurately classify diverse materials of interest, which is one of the core applications of hyperspectral remote sensing technology. In recent years, with the rapid development of deep learning, convolutional neural networks (CNNs) have been successfully applied in many fields, including HSI classification. However, the random gradient descent-based parameter updating scheme is too general and leading to the inefficiency of CNN models. Moreover, the high dimensionality and limited training samples of HSI data also exacerbate the overfitting problem. To tackle these issues, in this article, a novel deep network with multilayer and multibranch architecture, named 3-D Gabor CNN (3DG-CNN), is proposed for HSI classification. More precisely, since the predefined 3-D Gabor filters in multiple scales and orientations could well characterize the internal spatial–spectral structure of HSI data from various perspectives, the 3-D Gabor-modulated kernels (3-D GMKs) are employed to replace the random initialization kernels. Moreover, the specially designed multibranch architecture enables the network to better integrating the scalable property of 3-D Gabor filters; thus, the representative ability and robustness of the extracted features can be greatly improved. Alternatively, the number of network parameters is substantially reduced due to the incorporation of 3-D Gabor modulation, relieving the training complexity and also alleviating the training process from overfitting. Experimental results on four real HSI datasets (including two newly released ones in the literature) have demonstrated that the proposed 3DG-CNN model can achieve better performance than several widely used machine-learning-based and deep-learning-based approaches. For the sake of reproducibility, the codes of the proposed 3DG-CNN model are available athttp://jiasen.tech/papers/.

Domain adaptation, which cleverly applies the classifier learned from the source domain with sufficient labeled samples to the target domain with limited labeled samples, provides a feasible alternative to handle the small training sample problem of hyperspectral image (HSI) classification and has attracted much attention in the research field recently. Apparently, feature discriminative ability is vital for domain adaptation, which plays a crucial role during the migration process of transfer learning. In this article, a gradient feature-oriented 3-D domain adaptation (GF-3DDA) approach is proposed for HSI classification. First, 3-D Gabor is employed to remove noise from the original data, and two 2-D gradient-based features, 2-D Sobel gradient (SG) and 2-D derivative-of-Gaussian (DtG), are extended to the 3-D domain to coincide with the integrated spatial–spectral organization of HSI. Thus, the 3-D Sobel–Gabor gradient (3DSGG) and 3-D derivative-of-Gaussian-Gabor (3DDGG) features are achieved. Second, a 3-D domain adaptation method is implemented to jointly exploit the second- and fourth-order statistical descriptors in the spatial–spectral dimensions, which could effectively reduce domain shifts and thus achieve improved domain adaptation. Third, all the extracted domain-adapted feature modules are collaboratively classified by extreme learning machine (ELM), and the probability-like outputs of every ELM classifier are combined together to accomplish the classification task. Four hyperspectral data sets that each contains two scenes, i.e., Pavia, Shanghai–Hangzhou, Indiana, and Houston, are tested in the experiments. When only ten labeled samples per class are used in the target domain, the classification accuracies on four hyperspectral data sets achieved by our GF-3DDA approach are 93.31%, 84.35%, 69.32%, and 80.06%, respectively.

Image super-resolution (SR) methods can generate remote sensing images with high spatial resolution without increasing the cost of acquisition equipment, thereby providing a feasible way to improve the quality of remote sensing images. Clearly, image SR is a severe ill-posed problem. With the development of deep learning, the powerful fitting ability of deep neural networks has solved this problem to some extent. Since the texture information of various remote sensing images are totally different from each other, in this paper, we proposed a network based on generative adversarial network (GAN) to achieve high resolution remote sensing images, named multi-attention generative adversarial network (MA-GAN). The main body of the generator in MA-GAN contains three blocks: pyramid-convolutional residualdense (PCRD) block, attention-based upsampling (AUP) block and attention-based fusion (AF) block. Specifically, the developed attention-pyramid convolutional (AttPConv) operator in PCRD block combines multi-scale convolution and channel attention (CA) to automatically learn and adjust the scale of residuals for better representation. The established AUP block utilizes pixel attention (PA) to perform arbitrary scales of upsampling. And the AF block employs branch attention (BA) to integrate upsampled low-resolution images with high-level features. Besides, the loss function takes both adversarial loss and feature loss into consideration to guide the learning procedure of generator. We have compared our MA-GAN approach with several state-of-the-art methods on a number of remote sensing scenes, and experimental results consistently demonstrate the effectiveness of the proposed MA-GAN. For study replication, the source code will be released at: https://github.com/ZhihaoWang1997/MA-GAN.

Hyperspectral imagery (HSI) contains hundreds of bands, which provide a wealth of spectral information and enable better characterization of features. However, the excessive dimensionality also poses a dimensional disaster for subsequent processing. Fortunately, band selection (BS) gives a straightforward and effective way to pick out a subset of bands with rich information and low correlation. Although many hyperspectral BS methods, especially clustering-based ones, have been proposed by researchers in recent years, the contextual information of adjacent bands and the spatial structural information of materials are not well investigated. Therefore, in this article, a multiscale superpixel-level group-clustering framework (MSGCF) has been proposed for hyperspectral BS. Different from previous, a new superpixel-level distance measure is elaborately utilized to group and cluster the spectral bands, which jointly considers the spectral context and spatial structure information. Concretely, to preserve the spatial structural information of HSI, multiple superpixel segmentation is first performed to generate superpixel maps in multiscales, which enables complementarity of multiple superpixel segmentation algorithms and adaptation to diverse scales of land cover types. Second, the grouping and clustering paradigm is introduced to conduct the contextual information among bands. Here the maximum points of superpixel-level KL-$\ell _{1}$distance of adjacent bands are adopted as partition points to separate bands into groups, which encourages adjacent bands with strong correlation to be divided into the same group. Third, a superpixel-level fast density-based clustering method (SuFDPC) with superpixel-level$\ell _{2, 1}$distance is developed to select representative bands in every group. Finally, BS results are achieved with a ranking-based voting strategy by concerning information entropy and frequency of occurrence in a unified scheme. A series of ablation analyses and experimental comparisons on four real HSI datasets have been conducted, as well as similarity comparisons for the selected bands. The experimental results consistently demonstrated the effectiveness of our MSGCF approach. The codes of this work will be available athttp://jiasen.tech/papers/for the sake of reproducibility.

Supervised classification algorithms on the intricate ground object information of hyperspectral images (HSIs) require a large number of training samples that are annotated manually for model learning. To reduce the labeling cost and improve training sample effectiveness, a multiview spatial–spectral active learning (MVSS-AL) model is proposed in this study. First, a committee model composed of collaborative representation classification is introduced to form a leave-one-class-out (LOCO) multiview strategy, which explores more effective information in the limited training data. Second, the sample query strategy is designed from the perspective of classification confidence (CC) and training contribution (TC). The most inconsistent high-quality samples are screened by making full use of iterative prediction information and spatial–spectral features contained in hyperspectral imagery. Finally, the spatial–spectral LOCO active learning (AL) model obtains target samples through two-layer screening in each iteration and utilizes a support vector machine to obtain the final classification results. The proposed method is tested on three real-world hyperspectral datasets, and the comparison with several novel methods shows that the proposed method is better in the classification performance of restricted sample training.

The rapid development of remote sensing sensors makes the acquisition, analysis, and application of hyperspectral images (HSIs) more and more extensive. However, the limited sample sets, high-dimensional features, highly correlated bands, and mixing spectral information make the classification of HSIs a great challenge. In this article, an unsupervised multiscale and diverse feature learning (UMsDFL) method is proposed for HSI classification, which deeply considers the spatial–spectral features via convolutional neural networks (CNNs). Specifically, after employing the simple noniterative clustering (SNIC) algorithm with the heuristic calculation of superpixel size, the HSIs are segmented into superpixels for feature learning. The unsupervised network is designed with the convolutional encoder and decoder, the additional clustering branch, and the multilayer feature fusion to enhance the distinguishability of feature learning and the reusability of feature maps. Then, the spatial relationships and object attributes in large- and small-scale contexts are learned collaboratively through the unsupervised network to utilize the complementary multiscale characteristics. Moreover, the diverse features of hyperspectral information and nonsubsampled contourlet transform (NSCT) textures are learned simultaneously via the unsupervised network to alleviate the insufficiency of geometric representation. Finally, the random forest (RF) is adopted as the comprehensive classifier for land cover mapping based on the UMsDFL, and superpixel regularization is adopted to optimize the classification results. A series of experiments are performed on three real-world HSI datasets to demonstrate the effectiveness of our UMsDFL approach. The experimental results show that the proposed UMsDFL can achieve the overall accuracy of 79.23%, 96.49%, and 77.26% for Houston, Pavia, and Dioni datasets, respectively, when there are only five samples per class for training.

With the rapid development of deep learning technology and improvement in computing capability, deep learning has been widely used in the field of hyperspectral image (HSI) classification. In general, deep learning models often contain many trainable parameters and require a massive number of labeled samples to achieve optimal performance. However, in regard to HSI classification, a large number of labeled samples is generally difficult to acquire due to the difficulty and time-consuming nature of manual labeling. Therefore, many research works focus on building a deep learning model for HSI classification with few labeled samples. In this article, we concentrate on this topic and provide a systematic review of the relevant literature. Specifically, the contributions of this paper are twofold. First, the research progress of related methods is categorized according to the learning paradigm, including transfer learning, active learning and few-shot learning. Second, a number of experiments with various state-of-the-art approaches has been carried out, and the results are summarized to reveal the potential research directions. More importantly, it is notable that although there is a vast gap between deep learning models (that usually need sufficient labeled samples) and the HSI scenario with few labeled samples, the issues of small-sample sets can be well characterized by fusion of deep learning methods and related techniques, such as transfer learning and a lightweight model. For reproducibility, the source codes of the methods assessed in the paper can be found at https://github.com/ShuGuoJ/HSI-Classification.git.

Due to the importance in many military and civilian applications, hyperspectral anomaly detection has attracted remarkable interest. Low-rank representation (LRR)-based anomaly detectors use the low-rank property to represent background pixels, and pixels that cannot be well represented are detected as anomalies. The ability of an LRR-based detector to separate background pixels and anomalous pixels depends on the dictionary representation ability, which usually can be enhanced by designing a proper prior for dictionary representation coefficients and constructing a better dictionary. However, it is not easy to handcraft effective and meaningful regularizers for dictionary coefficients. In this article, we propose a novel anomaly detection algorithm that uses a plug-and-play prior for representation coefficients and constructs a new dictionary based on clustering. Instead of cumbersomely handcrafting a regularizer for representation coefficients, we propose solving the anomaly detection problem using the plug-and-play framework, which enables us to plug state-of-the-art priors for representation coefficients. An effective convolutional neural network (CNN) denoiser is plugged into our framework to fully exploit the spatial correlation of representation coefficients. We also propose a modified background dictionary construction method, which carefully includes background pixels and excludes anomalous pixels from clustering results. We refer to the proposed anomaly detection method as plug-and-play denoising CNN regularized anomaly detection (DeCNN-AD) method. Extensive experiments were performed on five data sets in a comparison with eight state-of-the-art anomaly detection methods. The experimental results suggest that the proposed method is effective in anomaly detection and can produce better anomaly detection results than that of the comparison methods. The codes of this work will be available athttps://github.com/FxyPdfor the sake of reproducibility.

In the hyperspectral image, each pixel corresponds to a small area on the Earth's surface and represents the intrinsic characteristic of objects, which can be applied for recognition of land covers. Nevertheless, hyperspectral image processing should face some critical issues, and a small sample set problem may be the most challenging one in the research. Deep learning (DL), which has successfully been applied in many fields, has also been introduced for hyperspectral image classification. However, the large gap between the massive parameters to be tuned and limited labeled samples can lead to overfitting scenario, inevitably deteriorating the generalization ability of the DL model. In this article, a lightweight convolutional neural network (LWCNN) is proposed for hyperspectral image classification to mainly tackle the small sample set problem. Especially, spatial-spectral Schroedinger eigenmaps (SSSE) feature extraction is first adopted to obtain the joint spatial-spectral information, and the compressed dimensionality could significantly reduce the number of parameters in the following DL model. Second, a dual-scale convolution (DSC) module is carefully designed to address the SSSE features from a 1-D vector viewpoint (the number of parameters is further decreased), and the DSC procedure is successively employed to obtain the hierarchical structure description that could represent data distribution from different aspects. Subsequently, the feature vectors from all DSC layers are separately filtered by a new bichannel fusion (BCF) module, which could well encode both the intrinsic and contextual information inside DSC features. Finally, the filtered features are concatenated together and imported into a global average pooling classifier to achieve the predicted probability of each category. Experimental results on three famous hyperspectral image data sets illustrate that the developed LWCNN approach is advantageous in both the efficiency and robustness sides for hyperspectral image classification tasks and outperforms other state-of-the-art methods (both traditional-based and DL-based) with very limited labeled samples.

Hyperspectral images encompass abundant information and provide unique characteristics for material classification. However, the labeling of training samples can be challenging in hyperspectral image classification. To address this problem, this study proposes a framework named flexible Gabor-based superpixel-level unsupervised linear discriminant analysis (FG-SuULDA) to extract the most informative and discriminating features for classification. First, a number of 3-D flexible Gabor filters are rigorously designed using an asymmetric sinusoidal wave to sufficiently characterize the spatial–spectral structure in hyperspectral images. Then, an unsupervised linear discriminant analysis strategy guided by the entropy rate superpixel (ERS) segmentation algorithm, calledSuULDA, is skillfully introduced to reduce the extracted large amount of FG features. TheSuULDA method not only boosts the classification capability but also increases the peculiarity of features, with the aid of superpixel information. Finally, the achieved features are imported to the popular support vector machine classifier. The proposed FG-SuULDA framework is applied to four real hyperspectral image data sets, and the experiments constantly prove that our FG-SuULDA is superior to several state-of-the-art methods in both classification performance and computational efficiency, especially with scarce training samples. The codes of this work are available athttp://jiasen.tech/papers/for the sake of reproducibility.

The rapid increase in the number of remote sensing sensors makes it possible to develop multisource feature extraction and fusion techniques to improve the classification accuracy of surface materials. It has been reported that light detection and ranging (LiDAR) data can contribute complementary information to hyperspectral images (HSIs). In this article, a multiple feature-based superpixel-level decision fusion (MFSuDF) method is proposed for HSIs and LiDAR data classification. Specifically, superpixel-guided kernel principal component analysis (KPCA) is first designed and applied to HSIs to both reduce the dimensions and compress the noise impact. Next, 2-D and 3-D Gabor filters are, respectively, employed on the KPCA-reduced HSIs and LiDAR data to obtain discriminative Gabor features, and the magnitude and phase information are both taken into account. Three different modules, including the raw data-based feature cube (concatenated KPCA-reduced HSIs and LiDAR data), the Gabor magnitude feature cube, and the Gabor phase feature cube (concatenation of the corresponding Gabor features extracted from the KPCA-reduced HSIs and LiDAR data), can be, thus, achieved. After that, random forest (RF) classifier and quadrant bit coding (QBC) are introduced to separately accomplish the classification task on the aforementioned three extracted feature cubes. Alternatively, two superpixel maps are generated by utilizing the multichannel simple noniterative clustering (SNIC) and entropy rate superpixel segmentation (ERS) algorithms on the combined HSIs and LiDAR data, which are then used to regularize the three classification maps. Finally, a weighted majority voting-based decision fusion strategy is incorporated to effectively enhance the joint use of the multisource data. The proposed approach is, thus, named MFSuDF. A series of experiments are conducted on three real-world data sets to demonstrate the effectiveness of the proposed MFSuDF approach. The experimental results show that our MFSuDF can achieve the overall accuracy of 73.64%, 93.88%, and 74.11% for Houston, Trento, and Missouri University and University of Florida (MUUFL) Gulport data sets, respectively, when there are only three samples per class for training.

As a typical graph-based semisupervised learning technique, the label propagation (LP) approach has gained much attention in recent years. The key to LP algorithms is the propagation capability and efficiency of the similarity matrix, which describes the similarity between two data points. Concerning hyperspectral image which often contains hundreds of thousands of pixels, the corresponding similarity matrix is particularly huge and thus the LP procedure is intractable. Fortunately, superpixel, which can effectively characterize the spatial semantic information of surface objects, provides a reasonable way to solve this problem. In this article, we propose an elaborate superpixel-based weighted LP approach, abbreviated as SuWLP, for hyperspectral image classification. First, the hyperspectral image is oversegmented by the entropy rate segmentation (ERS) method, and the internal consistency of each superpixel can be achieved. Second, a new similarity measure is designed to estimate the similarity between two superpixels, and a superpixel-based similarity matrix can be thus established. Third, after the training samples have been expanded based on the superpixel distribution, a weighted LP technique is designed to propagate the sample label at the superpixel level without any parameter tuning. Finally, the label of each superpixel maps back to the contained pixels. We compared our proposed SuWLP method with several state-of-the-art ones, and experimental results on three real hyperspectral data sets certify the effectiveness and efficiency of the superpixel-level LP strategy.

Hyperspectral images (HSIs) has practical applications in many fields. In practical scenarios, machine learning often fails to handle changes between training (source) and testing (target) input distributions due to domain shifts. A big challenge in hyperspectral image classification is the small size of labeled data for classifier learning. We always face the situation that an HSI scene is not labeled all or with very limited number of labeled pixels, but we have sufficient labeled pixels in another HSI scene with similar land cover classes. In this paper, we propose a simple and effective method for domain adaptation called statistical fusion to minimize domain shifts by aligning the second-order and fourth-order statistics of source and target distributions. After two hyperspectral scenes are transformed into the similar property-space, any traditional HSI classification approaches can be used, and experimental results have validated the generalization of the proposed method.

In virtue of the spatial structural characteristic of surface materials, the performance of the hyperspectral image classification can be boosted by incorporating texture information. Normally, the spatial structure can be extracted by predefined operators, including the popular extended multiattribute profiles (EMAPs) and the Gabor filters. Recently, superpixel segmentation, which reflects the homogeneous regularity of objects, has drawn much attention in the field. In this paper, a collaborative representation-based multiscale superpixel fusion (CRMSF) approach has been proposed for the hyperspectral image classification. First, after obtaining the EMAPs from the raw hyperspectral image, a group of predesigned 3-D Gabor wavelet filters is convolved with the EMAP features, and the EMAP-Gabor features can, thus, be achieved. Second, the collaborative representation-based classification (CRC) is employed to fully and efficiently make use of the huge amount of extracted EMAP-Gabor features. Third, multiscale superpixel maps are generated from the EMAP features that are utilized to regularize the classification map obtained by CRC. A heuristic strategy has been specially devised to automatically decide the number of extracted superpixels in multiple scales, which can be perfectly compatible with hyperspectral images having various spatial sizes and spatial resolutions. This is the most important contribution of the developed CRMSF approach. Finally, the classification task is accomplished by fusing the multiple regularized classification maps. The CRMSF approach has been evaluated on four popular hyperspectral image data sets, and the experimental results show the advantages of CRMSF, particularly for a hyperspectral image with high spatial resolution.

Hyperspectral remote sensing imagery provides valuable and rich information to distinguish the characteristics of materials. However, this advantage of hyperspectral imagery often encounters the problem of a limited amount of training samples, which is caused by the difficulty of manually labeling. Fortunately, the spatial distribution of surface objects can be integrated with the spectral signature to improve the discriminative ability. In this paper, a 3-D Gaussian-Gabor feature extraction and selection framework has been proposed for hyperspectral image classification. First, a bank of 3-D Gaussian-Gabor filters are convolved with the concatenated data of both extended multi-attribute profile (EMAP) features and raw hyperspectral data. Second, an improved fast density peak clustering (IFDPC) method is introduced to select the most representative features from each extracted 3-D Gaussian-Gabor feature cube. Finally, the retained features are combined together to accomplish the classification task. The proposed method is thus named as GG-IFDPC. Three real hyperspectral imagery data sets have been utilized, and the experiments demonstrate the advantages of the proposed GG-IFDPC approach over the compared ones.

Optical remote sensing images are often contaminated by smoke from forest fires or biomass burning. In this paper, a smoke removal method is proposed based on a spectral unmixing technique. Smoke components are detected first by generating subpixel smoke fraction masks with spectral mixture analysis. The smoke component is then subtracted from each smoky pixel. Finally, the attenuated signal is restored by rescaling the abundances of the endmember classes present in each pixel. The proposed method has high feasibility and is not dependent on other auxiliary data. The experiments have been conducted on an AVIRIS data set and the results show that the proposed method is effective for smoke detection and data correction.

Cloud covers, which generally appear in optical remote sensing images, limit the use of collected images in many applications. It is known that removing these cloud effects is a necessary preprocessing step in remote sensing image analysis. In general, auxiliary images need to be used as the reference images to determine the true ground cover underneath cloud-contaminated areas. In this paper, a new cloud removal approach, which is called multitemporal dictionary learning (MDL), is proposed. Dictionaries of the cloudy areas (target data) and the cloud-free areas (reference data) are learned separately in the spectral domain. The removal process is conducted by combining coefficients from the reference image and the dictionary learned from the target image. This method could well recover the data contaminated by thin and thick clouds or cloud shadows. Our experimental results show that the MDL method is effective in removing clouds from both quantitative and qualitative viewpoints.

Cloud removal is an important goal for enhancing the utilization of optical remote sensing satellite images. Clouds dynamically affect the signal transmission due to their different shapes, heights, and distribution. In the case of thick opaque clouds, pixel replacement has been commonly adopted. For thin clouds, pixel correction techniques allow the effects of thin clouds to be removed while retaining the remaining information in the contaminated pixels. In this paper, we develop a new method based on signal transmission and spectral mixture analysis for pixel correction which makes use of a cloud removal model that considers not only the additive reflectance from the clouds but also the energy absorption when solar radiation passes through them. Data correction is achieved by subtracting the product of the cloud endmember signature and the cloud abundance and rescaling according to the cloud thickness. The proposed method has no requirement for meteorological data and does not rely on reference images. Our experimental results indicate that the proposed approach is able to perform effective removal of thin clouds in different scenarios.

Optical remote sensing images are often contaminated by the presence of clouds. The development of cloud effect removal techniques can maximize the usefulness of multispectral or hyperspectral images collected in the spectral range from visible to mid infrared. This paper presents a new data reconstruction technique, via dictionary learning and sparse representation, to remove the cloud effects. Dictionaries of the cloudy data (target data) and the cloud free data (reference data) are learned separately in the spectral domain, where each atom represents a fine ground cover component under the two imaging conditions. In this study, it is found that the sparse coefficients of the reference data are the true weightings of each atom, which can be used to replace the cloud affected coefficients to achieve data correction. Experiments were conducted using Landsat 8 OLI data sets downloaded from the USGS website. The testing results show that clouds of various thickness and cloud shadows can be removed effectively using the proposed method.

The detection of cirrus cloud has historically been difficult due to the lack of a 1.375μm wavelength band in earlier Landsat ETM+ imagery. Landsat 8 OLI has addressed this problem by adding a new cirrus band at this wavelength. This paper presents a study of the effectiveness of utilizing the newly available data for this purpose. An image-based method is developed for cirrus cloud contamination correction. The relationship between a visible or infrared band and the cirrus band is estimated via a linear regression using the data in a homogenous land cover area. The key issue of how to automatic identify homogenous background from the cirrus contaminated data is addressed. The images corrected by our method show satisfactory quality.