Collaboratively utilizing the complementary information provided by hyperspectral imagery and light detection and ranging (LiDAR) data will extend the applications associated with land cover recognition and mapping. Existing joint classification algorithms mainly focus on learning complementary patterns in the pure spatial domain, while paying little attention to complementary cues in the spatial-frequency domain. The model’s expressive capability of these methods may be limited by an upper bound subject to the spatial domain. To fill this gap, a Dynamic Multiple Fractional Fourier Domains Fusion (DMFraF) is proposed for joint classification of hyperspectral and LiDAR data. Firstly, to comprehensively learn the complementary patterns between HSI and LiDAR data, we transform the features of two modalities into multiple fractional domains containing different spatial-frequency components for multimodal fusion. Secondly, to obtain the optimal representation from the multimodal features of multiple fractional domains, we propose a dynamic fusion scheme guided by the optimal transport (OT) technique, which can dynamically adjust the contributions from different fractional domains. Finally, to extract purer modality-specific features, we propose a channel aggregation Transformer encoder with central cross-attention (C2AT encoder), to aggregate channel-wise features of central pixels into the spatial branch and compress interference from noisy surroundings. Extensive experiments and analysis on three hyperspectral and LiDAR datasets suggest the superiority of the proposed method.

In recent years, language-supervised vision models have demonstrated impressive potential in learning open-world concepts. Some research has introduced this learning paradigm to the hyperspectral image (HSI) processing domain; however, there has been limited work integrating textual information into the hyperspectral open-set recognition task. To fill this gap, we leverage textual supervision information in open-set HSI classification (HSIC) and propose a language-enhanced dual-level contrastive learning network (LDCLNet). Specifically, we introduce a linguistic mode with prior knowledge as a supervised signal to enhance the metric distances between closed-set samples and provide supplementary semantic information for open-set samples. Second, a dual-level visual-language (V-L) contrastive learning (CL) approach, which can align visual and language embeddings separately at the instance level and manifold level, is proposed to establish a more accurate link between visual and language representations. Finally, a distance-refined open-set recognition method is proposed, which aims to effectively discover unknown class samples during testing by refining predictions of known and unknown classes. Extensive experiments and analysis on three public HSI datasets validate the effectiveness of LDCLNet.

Cross-scene hyperspectral image classification (HSIC) poses a significant challenge in recognizing hyperspectral images (HSIs) from different domains. The current mainstream approaches based on domain adaptation (DA) methods need to access target data when aligning distributions between domains, limiting the applicability of the model. In contrast, recent domain generalization (DG) methods aim to directly generalize to unseen domains, eliminating the requirements for target data during training. Nonetheless, most DG-based methods overly focus on randomizing sample styles, leading to semantically compromised samples. In addition, broadening the source distribution without ensuring reasonable support may result in undesired extended distributions. To address these issues, we propose a novel DG network with frequency disentanglement and data geometry (FDGNet) for cross-scene HSIC. Specifically, we first develop a spectral-spatial encoder based on frequency disentanglement (FDSS encoder), which facilitates synthesized domains to preserve their semantic consistency while simulating interdomain gaps with the source domain. Second, to avoid the generation of unrealistic samples, we incorporate data geometry into adversarial training. This helps diversify new domains while keeping the data geometry of extended domains in an explainable support. To improve the learning of domain-invariant representation, we propose an intermediate domain sampling strategy based on the class-wise perceptual manifold. This strategy synthesizes reliable intermediate domains by sampling from class-wise manifold flows estimated over the source and extended domains. Extensive experiments and analysis on three public HSI datasets yield the superiority of our proposed FDGNet. The codes will be available from the website: https://github.com/Qba-heu/FDGNet.

Existing hyperspectral cross-domain few-shot learning (FSL) methods focus mainly on elaborating on training strategies or domain alignment algorithms, while paying less attention to the biased meta-knowledge introduced by a large amount of source data and the implicit encouragement of learning target domain-specific attributes. In this paper, from the perspective of disentangled representation learning, a novel cross-domain FSL method based on feature disentanglement (FDFSL) is proposed for hyperspectral image classification (HSIC). Specifically, to suppress the representation biased towards the source data and enable the model to implicitly focus on the inherent knowledge of the target domain, an orthogonal low-rank feature disentanglement method is employed to acquire desired features of source and target pipelines. Furthermore, to preserve more shared and discriminative information from the heterogeneous data space (i.e., the spectral dimensions of the source and target scenes are typically different), a multi-order spectral interaction block based on central position encoding (MICD) is proposed to fully integrate the respective features into the spectral domain, which allows the model to emphasize informative spectral dimensions in a data-driven manner. Finally, to diversify the feature representation space while preventing the model overfitting domain alignment task, a self-distillation scheme is developed to facilitate the acquisition of task-relevant feature components. Extensive experiments and analysis on three public HSI datasets suggest the superiority of the proposed method. The code will be available on the website at https://github.com/Qba-heu/FDFSL.

The existing methods for hyperspectral image classification (HSIC) mainly focus on the extraction of spectral and spatial features while paying less attention to the interaction of each other. Besides, most of them directly use a parameterized classifier as the final layer of the network. While this design is convenient for end-to-end optimization with the backbone, it overlooks the utilization of the metric space. In this article, a novel hyperspherical structural-aware distillation enhanced spatial–spectral bidirectional interaction network (HSDBIN) is proposed for HSIC. HSDBIN uses a dual-branch design combining the 1-D CNN and transformer to separately learn the detailed spectral correlations and global spatial relationships in parallel. Then, by interacting and aggregating the independent information between two parallel branches, a bidirectional interaction block across branches is designed to explore complementary clues between spectral and spatial pipelines. Finally, to enhance the utilization of metric space and keep compact intraclass relationship, we propose a hyperspherical structural-aware distillation (HSD) to transfer the geometric relationship of hyperspherical space into the metric space of output logits. Extensive experiments and analysis on three public HSI datasets suggest the superiority of the proposed method and verify the effectiveness of the proposed modules.

Hyperspectral remote sensing images exhibit fine spectral curves, but they are also susceptible to spectral variations caused by factors like cloud and haze. It is evident that these issues become more pronounced when there is a limited number of labeled samples available. Thus, a full range feature extraction network (FRFENet) based on quality-quantity-balance sample enhancement is proposed for hyperspectral image classification. First, the full-range feature extraction method combines local-range, short-range, and long-range spatial-spectral features to address spectral variability and ensure accurate feature extraction, particularly in scenarios with limited labeled samples. Furthermore, the approach of balancing quality and quantity for pseudo-labeled samples allows for an increased number of pseudo-labels while maintaining their quality, effectively leveraging unlabeled samples. Additionally, the utilization of superpixel region homogeneity directly contributes to an expanded training sample set, resulting in improved classification performance of the algorithm. Experiments on three HSI datasets indicate that the FRFENet can obtain better classification performance when compared with the other ten state-of-the-art methods.

Hyperspectral image (HSI) classification is a fundamental research in the field of HSI processing, which has made great development, especially after deep learning-based method is widely used in this field. These methods are commonly starved for labeled samples. However, it is more challenging to obtain labeled samples than HSI in practice. Fortunately, self-supervised leaning (SSL) can take advantage of unlabeled data. In this paper, an HSI classification method based on masked self-supervised network (MSSL) is proposed. To obtain a representative and label-independent representation of HSI, a novel masking and reconstruction based proxy task is designed to accomplish SSL. Furthermore, a spatial masking strategy for HSIs is employed due to the high similarity of adjacent objects in remote sensing images. Experimental results on two benchmark HSI datasets indicate that the proposed MSSL can achieve better classification results with small number of labeled samples.

At present, hyperspectral image classification (HSIC) technology based on deep learning has been widely explored. However, the time and labor cost of obtaining enough labeled samples are expensive. To obtain higher classification performance with a few number of labeled samples, a coarse-to-fine semi-supervised classification learning (CFSSL) method is proposed in this letter. First of all, the CFSSL performs coarse-grained classification with a few number of labeled samples, and the breaking-ties (BT) criterion is introduced to sample the coarse-grained classification results to ensure that the samples with high confidence are selected to generate pseudo-labels. Then, the pseudo-labels and their corresponding unlabeled samples are sent to the feature extraction network for fine-grained classification, so as to obtain more advanced classification results. Finally, in the fine-grained classification stage, a multi-scale convolution kernel attention aggregation network (A2-MCKN) is designed to simultaneously extract the spatial-spectral features of the image and ensure clear texture boundaries of ground objects. Experimental results on two public datasets show that the CFSSL can obtain better accuracy than other methods with a few number of labeled samples.

Deep learning (DL) based methods represented by convolutional neural networks (CNNs) are widely used in hyperspectral image classification (HSIC). Some of these methods have strong ability to extract local information, but the extraction of long-range features is slightly inefficient, while others are just the opposite. For example, limited by the receptive fields, CNN is difficult to capture the contextual spectral-spatial features from a long-range spectral-spatial relationship. Besides, the success of DL-based methods is greatly attributed to numerous labeled samples, whose acquisition are time-consuming and cost-consuming. To resolve these problems, a hyperspectral classification framework based on multi-attention Transformer (MAT) and adaptive superpixel segmentation-based active learning (MAT-ASSAL) is proposed, which successfully achieves excellent classification performance, especially under the condition of small-size samples. Firstly, a multi-attention Transformer network is built for HSIC. Specifically, the self-attention module of Transformer is applied to model long-range contextual dependency between spectral-spatial embedding. Moreover, in order to capture local features, an outlook-attention module which can efficiently encode fine-level features and contexts into tokens is utilized to improve the correlation between the center spectral-spatial embedding and its surroundings. Secondly, aiming to train a excellent MAT model through limited labeled samples, a novel active learning (AL) based on superpixel segmentation is proposed to select important samples for MAT. Finally, to better integrate local spatial similarity into active learning, an adaptive superpixel (SP) segmentation algorithm, which can save SPs in uninformative regions and preserve edge details in complex regions, is employed to generate better local spatial constraints for AL. Quantitative and qualitative results indicate that the MAT-ASSAL outperforms seven state-of-the-art methods on three HSI datasets.

Nowadays, graph convolution networks are getting more and more attention in the field of hyperspectral image classification. The graph convolution can be divided into long-range and short-range graph convolution (GConv). However, the two graph convolutions cannot acquire global and local features at the same time, making the node features may not be accurate enough. Therefore, we propose a novel graph convolution approach, called short and long range graph convolution (SLGConv), which combines the advantages of long-range and short-range GConv. SLGConv can extract long-range (global) and short-range (local) spatial-spectral features, eliminating the disadvantages of each of long-range and short-range graph convolution. Furthermore, SLGConv can ensure that the features of nodes are not smoothed in the convolution process. Then, three layers of SLGConv are used to form the short and long range graph convolution network (SLGCN) for hyperspectral image classification. Experiments on three HSI datasets indicate that the SLGCN can obtain better classification performance when compared with seven state-of-the-art methods.

Nowadays, domain adaptation (DA) is getting more attention in cross-scene hyperspectral image (HSI) classification, and various DA algorithms have been proposed. However, regular convolution indiscriminately extracting features around the center pixel will result in the inaccurate extraction of spatial-spectral features, which significantly affect the subsequent feature alignment. Meanwhile, the method of aligning the category features of source and target domains from a single-level may not cope well with complex HSIs. Therefore, we propose a multilevel feature alignment algorithm based on spatial attention deformable convolution (MFA-SADC), which achieves multilevel feature alignment from feature to feature, feature to cluster-center, and cluster-center to cluster-center. In addition, spatial attention deformable convolution is proposed to compose the feature extraction network of MFA-SADC, which guarantees the purity of spatial-spectral features. Experiments on three HSI datasets indicate MFA-SADC can obtain better classification performance when compared with the seven state-of-the-art methods.

Despite success in the same-scene hyperspectral image classification (HSIC), for the cross-scene classification, samples between source and target scenes are not drawn from the independent and identical distribution, resulting in significant performance degradation. To tackle this issue, a novel unsupervised domain adaptation (UDA) framework toward multilevel features and decision boundaries (ToMF-B) is proposed for the cross-scene HSIC, which can align task-related features and learn task-specific decision boundaries in parallel. Based on the maximum classifier discrepancy, a two-stage alignment scheme is proposed to bridge the interdomain gap and generate discriminative decision boundaries. In addition, to fully learn task-related and domain-confusing features, a convolutional neural network (CNN) and Transformer-based multilevel features extractor (generator) is developed to enrich the feature representation of two domains. Furthermore, to alleviate the harm even the negative transfer to UDA caused by task-irrelevant features, a task-oriented feature decomposition method is leveraged to enhance the task-related features while suppressing task-irrelevant features, and enabling the aligned domain-invariant features can be contributed to the classification task explicitly. Extensive experiments on three cross-scene HSI benchmarks have validated the effectiveness of the proposed framework.

Nowadays, graph convolution networks (GCNs) are getting more attention in hyperspectral image classification, and various algorithms based on GCNs have been proposed. However, because of hyperspectral images’ complex spatial texture information, the long-range graph convolution (GConv) and short-range GConv may cause inaccurate or over-smoothed feature extraction of some nodes. Thus, a multiscale short and long range graph convolution network (MSLGCN) is proposed for hyperspectral image classification. First, MSLGCN not only extracts spatial information of ground objects at different scales but also simultaneously captures global and local spectral features, which preserves objects’ fine boundaries. Then, the rich multiscale information is complementary, enabling the MSLGCN to take full advantage of texture structures of varying sizes. In addition, a method to determine the superpixel scale by the intrinsic properties of hyperspectral images is proposed to ensure that the segmentation boundary depicts the texture structure of the object accurately. Finally, the short-long graph convolution (SLGConv) is designed to fuse the advantages of global and local features, enabling the MSLGCN to extract accurate spatial-spectral features of nodes at any location. Experiments on three HSI datasets indicate that the MSLGCN can obtain better classification performance when compared with the other eleven state-of-the-art methods.

In the research of hyperspectral image (HSI) classification based on deep learning, the small sample problem and the lack of classification accuracy caused by not considering global information have not been well solved. In this paper, an HSI classification method based on dense convolution and conditional random field (DCRF) is proposed. First, the 1D-2D convolution kernel is used to extract the spectral-spatial features and the layers are densely connected to obtain a dense convolutional network to reduce parameters. Second, the Max Pooling layer is used as the output layer of the dense convolutional network to improve the accuracy of feature extraction, and the Softmax layer is used to calculate the probability of the category of the sample and preliminary classification. Finally, the conditional random field is used to fully integrate spatial global information to achieve HSI final classification. Extensive experimental results on two HSI data sets have demonstrated the effectiveness of the proposed DCRF when compared with other state-of-the-art methods.