For the classification of hyperspectral images (HSIs), most deep learning networks are data-driven and lack the usage of prior knowledge. In this letter, we propose a knowledge graph-guided classification network (KGNet), attempting to utilize the prior knowledge of land cover categories to enhance the classification performance. We first construct a knowledge graph on several hyperspectral scenes, which can characterize not only the attributes of land cover categories but also the rich connections between categories. Semantic features are then derived to represent the knowledge in the graph. Knowledge-guided learning is achieved by performing feature alignment between semantic and visual features. Finally, classification is performed on visual features that have contained the knowledge from semantic features. Experiments on three datasets demonstrate the effectiveness of applying the knowledge graph for the classification of hyperspectral remote sensing images.

Recent studies have shown that the deep domain adaptation (DA) technique has achieved remarkable results in cross-domain hyperspectral image (HSI) classification task. However, these DA methods assume that the source and target domains share the same classes, which may not hold true in real-world applications. Under open-set conditions, since the target domain may contain classes unseen in the source domain, direct domain alignment can lead to negative transfer phenomena. Moreover, the presence of multiple unknown classes in the target domain makes it difficult to learn more discriminative classification boundaries between known and unknown classes. To address these issues, we propose an open-set DA (OSDA) method for HSI classification based on weighted generative adversarial networks and dynamic thresholding (WGDT). First, we introduce a class anchor (CA) strategy to learn the metric space of known classes in the source domain. By calculating the similarity between the target-domain samples and the CA, we compute the reliability weights of the samples belonging to known classes. Then, based on these weights, we design an instance-level weighted-domain adversarial learning strategy to better align samples that are more likely to belong to known classes, avoiding negative transfer phenomena. Finally, we propose a dynamic thresholding method to learn the classification boundaries between known and unknown classes in the feature space and reject unknown class samples, thereby separating known class samples in the target domain. The experimental results on four cross-scene HSI classification tasks demonstrate that our proposed method outperforms some existing methods. The code is available athttps://github.com/Li-ZK/WGDT.

This paper introduces a scale-generalization Transformer for pansharpening (DPP), ensuring high performance across resolutions. First, we employ joint learning of the pansharpening model across resolutions, establishing a collaborative iteration mechanism between dynamic full-resolution spectral projection model estimation and the pansharpening process. This effectively reduces errors introduced by directly using the reduced-resolution spectral projection model, guiding the pansharpening model to generalize from the well-defined reduced-resolution scale to the ill-posed full-resolution scale. Second, we customize a novel network for pansharpening, employing inverse spatial injection to enhance local and global awareness. Initially, CNN captures detailed spatial features from the panchromatic image, which are then inversely downsampled for aligned injection. Subsequently, CNN and Transformer are combined to integrate global spectral features with local spatial features, ensuring high-quality pansharpening. Extensive experiments demonstrate the advantages of our DPP in terms of information fidelity at both reduced and full resolutions.

We propose a novel two-stage cross-domain self-training (CDST) framework for unsupervised domain adaptive object detection in remote sensing. The first stage introduces the generative adversarial network (GAN)-based domain transfer strategy to preliminarily mitigate the domain shift for higher quality initial pseudo-labeled images, which utilizes the CycleGAN to transfer source-domain images to match the target domain. Moreover, the key issue in tailoring the self-training (ST) to unsupervised domain adaptive detection lies in the quality of pseudo-labeled images. To select high-quality pseudo-labeled images under the domain-shift circumstance, we propose hard example selection-based self-training (HES-ST) with the three key steps: 1) detector-based example division (DED), which divides the detected examples into easy examples and hard ones according to their confidence level; 2) confidence and relation joint score (CRJS)-based hard example selection, which combines two reliability levels calculated, respectively, by the detector and relation network (RN) module to mine reliable examples; and 3) union example (UE)-based training image selection, which combines both easy and reliable hard examples to choose target-domain images that may contain fewer detection errors. The experimental results on several remote sensing datasets demonstrate the effectiveness of our proposed framework. Compared with the baseline detector trained on the source dataset, our approach consistently improves the detection performance on the target dataset by 15.7%–16.8% mean average precision (mAP) and achieves the state-of-the-art (SOTA) results under various domain adaptation scenarios.

A sparse label oriented semantic segmentation network (SL-SSNet) is proposed for classification of hyperspectral images in this paper. Since semantic segmentation network performs pixel-level classification and can extract long-range contextual information, we apply it to the task of hyperspectral image classification. To mitigate the small size sample problem, we not only design a lightweight fully convolutional network, but also explore the usefulness of unlabeled data by introducing two constraints. Firstly, an adversarial learning based multi-classifier consistency strategy is employed to improve the classification of unlabeled data. It constrains two different classifiers to have consistent prediction results on unlabeled data. As a result, the extracted features of unlabeled data can be more discriminative and the predictions are more reliable. Secondly, a manifold regularizer is applied to constrain the classification results of unlabeled data to be smooth with respect to the data manifold, which can further exploit the unlabeled data and alleviate the small size sample problem. The experimental results using multiple hyperspectral data demonstrate the efficiency of the proposed method.

Deep domain adaptation has achieved promising results in cross-domain hyperspectral image (HSI) classification. However, existing methods often focus on aligning data distributions without sufficient consideration of separability of source and target domain data themselves. In addition, current adversarial domain adaptation methods aim to achieve similar distributions between domains by confusing the discriminator, rather than obtaining a more compact distribution. In particular, existing methods are not discriminative enough for the target domain due to the difficulty of obtaining high-confidence labeled samples of the target domain. To address the above challenges, we propose a supervised contrastive learning-based unsupervised domain adaptation for HSI classification. A supervised contrastive learning strategy is then performed in both the source and target domains, which allows samples from the same category to be pulled closer together and samples from different categories to be pushed further apart, thus enhancing the separability of the data within the domain. The domain adaptation task is treated as a one-class classification (OCC) task, and a novel domain similarity loss based on OCC is introduced to reduce the discrepancy between domains. Finally, a confidence learning-based sample selection strategy is designed to select high-confidence labeled samples from the target domain to fine-tune the domain adaptation model, which can enhance the discrimination of the model to the target domain. Experimental results on three cross-domain datasets demonstrate that our proposed method outperforms existing domain adaptation methods. Our source code is available at https://github.com/Li-ZK/SCLUDA-2023.

Oriented object detection in remote sensing images has drawn great attention since it can provide more accurate bounding boxes. We propose a one-stage anchor-free network based on searching four corner points of an object, which can yield an arbitrary quadrilateral to fit objects with different shapes and orientations. We detect the corners by combining two strategies, where one regresses to the relative corner positions with respect to their corresponding center and the other directly detects the absolute corner positions from the corner heatmaps. By defining a candidate corner region based on the regressed results, we check whether corner points from the corner heatmaps are included in the region. If so, the closest one relative to the regressed corner is selected as the final position; otherwise, the regressed corner position is utilized. Experiments were conducted on two aerial remote sensing datasets, and the results demonstrated that the proposed method achieves superior performance to both the anchor-based and anchor-free methods.

Cross-domain hyperspectral image classification is one of the major challenges in remote sensing, especially for target domain data without labels. Recently, deep learning approaches have demonstrated effectiveness in domain adaptation. However, most of them leverage unlabeled target data only from a statistical perspective but neglect the analysis at the instance level. For better statistical alignment, existing approaches employ the entire unevaluated target data in an unsupervised manner, which may introduce noise and limit the discriminability of the neural networks. In this article, we propose confident learning-based domain adaptation (CLDA) to address the problem from a new perspective of data manipulation. To this end, a novel framework is presented to combine domain adaptation with confident learning (CL), where the former reduces the interdomain discrepancy and generates pseudo-labels for the target instances, from which the latter selects high-confidence target samples. Specifically, the confident learning part evaluates the confidence of each pseudo-labeled target sample based on the assigned labels and the predicted probabilities. Then, high-confidence target samples are selected as training data to increase the discriminative capacity of the neural networks. In addition, the domain adaptation part and the confident learning part are trained alternately to progressively increase the proportion of high-confidence labels in the target domain, thus further improving the accuracy of classification. Experimental results on four datasets demonstrate that the proposed CLDA method outperforms the state-of-the-art domain adaptation approaches. Our source code is available athttps://github.com/Li-ZK/CLDA-2022.

Class-wise adversarial adaptation networks are investigated for the classification of hyperspectral remote sensing images in this article. By adversarial learning between the feature extractor and the multiple domain discriminators, domain-invariant features are generated. Moreover, a probability-prediction-based maximum mean discrepancy (MMD) method is introduced to the adversarial adaptation network to achieve a superior feature-alignment performance. The class-wise adversarial adaptation in conjunction with the class-wise probability MMD is denoted as the class-wise distribution adaptation (CDA) network. The proposed CDA does not require labeled information in the target domain and can achieve an unsupervised classification of the target image. The experimental results using the Hyperion and Airborne Visible/Infrared Imaging Spectrometer (AVIRIS) hyperspectral data demonstrated its efficiency.

The original manifold alignment (MA) approach is for semisupervised domain adaptation. Since the target prior information is difficult to obtain, we conduct it in an unsupervised manner, resulting in an unsupervised MA (UMA) method. This approach utilizes the probabilistic prediction results of target data to construct the cross-domain similarity matrix, which characterizes the relationships between domains and is used for alignment. Due to the spectral drift, the prediction results may not be accurate, and thus affect the alignment. We employed spatial filtering and overall centroid alignment method as two preprocessing strategies to improve the prediction results. Furthermore, per-class maximum mean discrepancy (MMD) constraint is introduced to the UMA to further improve the alignment performance. The proposed UMA_MMD algorithm is applied for the classification of remote sensing images, and the experimental results using hyperion multitemporal remote sensing images demonstrated the effectiveness of the proposed approach.

Domain adaptation, which aims at learning an accurate classifier for a new domain (target domain) using labeled information from an old domain (source domain), has shown promising value in remote sensing fields yet still been a challenging problem. In this letter, we focus on knowledge transfer between hyperspectral remotely sensed images in the context of land-cover classification under unsupervised setting where labeled samples are available only for the source image. Specifically, a discriminative transfer joint matching (DTJM) method is proposed, which matches source and target features in the kernel principal component analysis space by minimizing the empirical maximum mean discrepancy, performs instance reweighting by imposing an ℓ2,1-norm on the embedding matrix, and preserves the local manifold structure of data from different domains and meanwhile maximizes the dependence between the embedding and labels. The proposed approach is compared with some state-of-the-art feature extraction techniques with and without using label information of source data. Experimental results on two benchmark hypersepctral data sets show the effectiveness of the proposed DTJM.

An extreme learning machine (ELM)-based heterogeneous domain adaptation (HDA) algorithm is proposed for the classification of remote sensing images. In the adaptive ELM network, one hidden layer is used for the source data to provide the random features, whereas two hidden layers are set for target data to produce the random features as well as a transformation matrix. DA is achieved by constraining both the source data and the transformed target data to share the same output weights. Moreover, manifold regularization is adopted to preserve the local geometry of unlabeled target data. The proposed ELM-based HDA (EHDA) method is applied to cross-domain classification of remote sensing images, and the experimental results using multisensor remote sensing images demonstrate the effectiveness of the proposed approach.

A new domain adaptation algorithm based on the class centroid and covariance alignment (CCCA) is proposed for classification of remote sensing images. This approach exploits both the first- and second-order statistics to describe the data distribution and aligns the data distribution between domains on a per-class basis. Since the predicted labels of target data are used to estimate the two statistics, we applied overall centroid alignment (OCA) as a coarse domain adaptation strategy to improve the estimation accuracy. In addition, the OCA coarse adaptation in conjunction with CCCA refined adaptation can also benefit by incorporation of spatial information, resulting in a Spa_OCA_CCCA approach. The proposed approach is easy to implement, and only one parameter is required in the spatial filtering step. It does not require labeled information in the target domain and can achieve labor-free classification. The experimental results using Hyperion, National Center for Airborne Laser Mapping, and Worldview-2 remote sensing images demonstrated the effectiveness of the proposed approach.

Graph construction, which is at the heart of graph-based semisupervised learning (SSL), is investigated by using manifold learning (ML) approaches. Since each ML method can be demonstrated to correspond to a specific graph, we build the relation between ML and SSL via the graph, where ML methods are employed for graph construction. Moreover, sparsity is important for the efficiency of SSL algorithms, and therefore, local ML (LML)-method-based sparse graphs are utilized. The LML-based graphs are able to capture the local geometric properties of hyperspectral data and, thus, are beneficial for classification of data with complex geometry and multiple submanifolds. In experiments with Hyperion and AVIRIS hyperspectral data, graphs constructed by two LML methods, namely, locally linear embedding and local tangent space alignment (LTSA), performed better than several popular graph construction methods, and the highest accuracies were obtained by using graphs provided by LTSA.

For hyperspectral image classification, manifold learning based graph Laplacian is proposed in the Laplacian support vector machine (LapSVM) classifier. The manifold regularization term in LapSVM constrains the smoothness of classification function on the data manifold. Since manifold learning approach is capable of exploring the manifold geometry of data, it is suitable for calculating the graph Laplacian in the regularization term. Two manifold learning methods, local tangent space alignment (LTSA) and locally linear embedding (LLE) are utilized to obtain graph Laplacian. Experimental results indicate that the LTSA and LLE based graph Laplacian produce superior classification results than heat kernel weights and binary weights based graph Laplacian in LapSVM.

Anomaly detection in hyperspectral images is investigated using local tangent space alignment (LTSA) for dimensionality reduction (DR) in conjunction with a minimum distance detector. The LTSA is implemented for large images by constructing a manifold with training data and employing the out-of-sample extension for testing data. The training data that should represent all the background types are generated by the recursive hierarchical segmentation (RHSEG) algorithm and the elimination of the very small segments that may represent anomalies. Experimental results indicate that the LTSA is able to distinguish anomalies from background using a small number of features in the embedded space, and the LTSA-based detector has superior anomaly detection performance to the well-known RX and kernel RX detectors.

Approaches to combine local manifold learning (LML) and thek-nearest-neighbor (kNN) classifier are investigated for hyperspectral image classification. Based on supervised LML (SLML) andkNN, a new SLML-weightedkNN (SLML-WkNN) classifier is proposed. This method is appealing as it does not require dimensionality reduction and only depends on the weights provided by the kernel function of the specific ML method. Performance of the proposed classifier is compared to that of unsupervised LML (ULML) and SLML for dimensionality reduction in conjunction with thekNN (ULML-kNN and SLML-kNN). Three LML methods, locally linear embedding (LLE), local tangent space alignment (LTSA), and Laplacian eigenmaps, are investigated with these classifiers. In experiments with Hyperion and AVIRIS hyperspectral data, the proposed SLML-WkNN performed better than ULML-kNN and SLML-kNN, and the highest accuracies were obtained using weights provided by supervised LTSA and LLE.