Multimodal remote sensing image joint classification has achieved significant progress. However, existing methods primarily focus on designing modality-specific networks, lacking adaptive generalization capabilities in diverse and dynamic modality combinations encountered in real-world scenarios. Inspired by the generalization capabilities of visual foundation model in downstream tasks, we propose a unified Text-guided Arbitrary Modalitiy Prompting (T-APT) framework, which leverages complementary fused features to drive the foundation model and employs text-guided modality-specific prior knowledge as cross-modal prompts to fine-tune a pretrained Vision Transformer (ViT) model. Specifically, a Mamba-Based Arbitrary Modal-Focused Feature Capture (MAMF-FC) module is designed to extract complementary joint features and modality-specific prior knowledge from arbitrary modalities through a shared-specific scanning encoder-decoder architecture. Subsequently, a Text-Guided Modality-Aware Prompt Tuning (TMPT) module is proposed to support the adaptation of fused features to the foundation model, enabling our arbitrary remote sensing image classification task. Extensive experiments on public datasets spanning multispectral (MS), hyperspectral (HS), light detection and ranging (LiDAR), and synthetic aperture radar (SAR) modalities demonstrate that our T-APT achieves classification performance comparable to specialized networks across arbitrary modal combinations.

Hyperspectral target detection effectively identifies fixed targets using specific spectral signatures but suffers from catastrophic forgetting when detecting multiple targets of interest within the same scene. Traditional data replay strategies may further exacerbate training instability due to mislabeled samples. To address these limitations, we propose an Adaptive Coarse-to-Fine Parameter Optimization framework (ACFPO) for incremental hyperspectral target detection, which enables stable continual learning via structural adaptation and parameter sensitivity–aware refinement. ACFPO formulates the task as a dual-stage process: coarse-grained matching and fine-grained detection. Specifically, an Adaptive Spectral Prior-Guided Coarse Matching (AS-PCM) module is designed to hierarchically organize detection tasks into semantic domains and construct intra- and inter-class spectral pairs for coarse-level alignment to adaptively select optimal submodels. Subsequently, a Distance-Aware Localized Fine-Grained Parameter Optimization (DA-LFPO) module is proposed to identify layer-wise sensitive parameters of the selected submodels by measuring spectral–spatial discrepancy, enabling selective retraining to preserve model stability on previously learned classes. By dynamically freezing non-sensitive parameters and optimizing critical modules, our approach mitigates inherent model drift and gradient conflicts in replay-based methods. Extensive experiments on three benchmark datasets demonstrate the superior performance of ACFPO, achieving a balanced trade-off between stability of the existing target and the adaptability of incremental targets. The code is available at https://github.com/Jiahuiqu/ACFPO.

Hyperspectral image (HSI) change detection is a technique that can identify the changes occurring between the bitemporal HSIs covering the same geographic area. The field of change detection has witnessed the proposal and successful implementation of numerous methods. However, a majority of these approaches adhere to the centralized learning paradigm, which requires data transmission to a central server for training. The sensitivity of remote sensing data generally prohibit their sharing across different clients. Furthermore, manual labeling is a costly effort in practically. In this paper, we propose a spatial-spectral-temporal collaborative Mamba-based active federated hyperspectral change detection (MambaFedCD) framework, which utilizes the limited labeled samples from multiple clients to achieve change detection while ensuring the data privacy of each client. Specifically, there are three key characteristics: 1) a spatial-spectral-temporal collaborative Mamba-based change detection ( ${{\text {S}}^{2}}{\text {TMamba}}$ ) model is proposed to efficiently synergize the temporal and global spatial-spectral information of the bitemporal HSIs for change detection; 2) a difference feature diversity correction-based model aggregation (DFDCMA) strategy is devised to incorporate the diversity of difference features for rational allocation of weight factors among clients and to facilitate effective aggregation of the global model; 3) we propose a multi-decision federated active learning (MDFAL) strategy that selects both error-prone and valuable samples for model training to alleviate the burden of sample labeling. Comprehensive experiments conducted on commonly utilized datasets demonstrate that the proposed method outperforms other state-of-the-art methods. The code is available at https://github.com/Jiahuiqu/MambaFedCD.

Recent research on the joint classification of multimodal remote sensing data has achieved outstanding performance in tasks within predefined label spaces. However, surface conditions are dynamic and change over time, resulting in variations in land cover classes collected from the same region at different time points. As a result, when new classes are discovered, the previous works must use a combination of old and new class data to retrain the model, which incurs high computational costs and raises concerns about data privacy. In this work, we propose the prototype-based meta-prompt tuning (PMPT) framework, which fine-tunes only a few session-relevant visual prompts to adapt to incremental classes, while simultaneously learning prototype embeddings for each class to preserve historical knowledge. Specifically, the PMPT consists of a meta-learning-based feature representation backbone and an incrementally updated nearest-class-mean (NCM) classifier. The backbone is trained on base class data to learn shared and stable global knowledge, then frozen, with only the prompts fine-tuned to extract sessions-specific local knowledge from incremental sessions. The NCM classifier is a globally shared classifier that measures the similarity between test samples and prototypes, effectively alleviating the issues of knowledge forgetting and overfitting. Additionally, we propose an incremental prototype contrastive loss to reduce semantic drift and prototype overlap in the embedding space. During the testing phase, the PMPT reproduces the complete embedding function by matching samples, class prototypes, and visual prompts, thereby enabling accurate classification of unknown samples. The method has been tested on widely used multimodal remote sensing datasets, demonstrating the effectiveness of the proposed PMPT in addressing the dilemma of stability-plasticity with limited incremental samples. The code is available at https://github.com/Jiahuiqu/PMPT.

Class-incremental learning aims to continuously expand the category space while mitigating catastrophic forgetting of previously learned classes, and has recently attracted increasing attention in remote sensing image classification for long-term adaptive applications. However, most existing class-incremental learning methods are developed under single-source classification settings, where the complementary information across multiple data sources is not fully exploited, limiting their applicability in multi-source remote sensing scenarios classification. To address this issue, we propose a novel prompt-based framework for class-incremental learning on multi-source remote sensing images. Specifically, we introduce a Bidirectional Cross-Modal Prompt Tuning (BiCM-PT) module that dynamically selects modality-specific prompts while preserving historical cross-modal relationships by freezing modality relation projectors from previous tasks, thereby enhancing model stability. Furthermore, to improve plasticity for new-class learning, we design a Prompt-Guided Knowledge Aggregator (PGKA) that leverages learned prompts to guide decision-level feature aggregation and extract discriminative multi-modal representations. Together, these components enable effective and stable class-incremental learning in multi-source remote sensing environments. Extensive experiments on three real-world remote sensing benchmarks demonstrate the effectiveness of our approach in balancing stability and plasticity under multi-source incremental learning settings. The code is available at https://github.com/Jiahuiqu/BiCMPT.

In recent years, deep learning has revolutionized hyperspectral image (HSI) classification. However, it remains a significant challenge to achieve high-precision classification with limited image quality and labeled samples. Most existing methods fail to effectively leverage unlabeled samples and neglect the impact of image quality degradation on classification performance. To address these issues, this paper proposes a Fusion-Driven Task Mutual-Guidance Network (FTMNet), which enhances image quality and improves classification performance through mutual guidance between fusion and classification tasks. Specifically, we propose an image fusion subnet integrating contrastive learning to jointly optimize input quality enhancement and discriminative feature representation through multi-objective constraints. To mitigate sample scarcity, a multi-task interactive multimodal contrastive architecture is developed, leveraging cross-modal complementarity and cross-task feature sharing mechanisms to strengthen discriminative power. Furthermore, we introduce a cross-task collaborative mutual-guidance strategy that synchronizes inter-task information exchange via learnable parametric constraints, forming unified optimization directions for coordinated performance enhancement. The experimental results demonstrate that the proposed method outperforms the existing state-of-the-art methods in both quantitative and qualitative aspects. Code is available at https://github.com/Jiahuiqu/FTMNet.

Change detection aims to identify land cover changes by analyzing multitemporal images that cover the same area. However, It may be difficult to effectively obtain high-quality multitemporal images with the same modality in real dynamic scenarios. The rapid development of remote sensing technology enables collaborative observation of multimodal images, but it is challenging for uni-modal image-specific methods to overcome modal discrepancy and achieve complementary advantage detection. To this end, we propose a bidirectional diffusion guided collaborative change detection model (Bi-DiffCD) for arbitrary-modal images, which eliminates the modal discrepancy between arbitrary-modal images through the bidirectional diffusion and makes full use of the multilevel complementary advantage features to improve the detection accuracy. Specifically, a conditional diffusion-based bidirectional modal alignment module (CDBMA) is designed to step-wise align the modal attribute bidirectionally while preserving the multimodal complementary features. Furthermore, a multilevel complementary feature collaborative change detection module (MLCCD) is proposed to collaborate the multilevel enhanced complementary change information from transformed images and potential features for change detection. Experiments have been conducted on three widely used and one self-made multimodal datasets to demonstrate the effectiveness of the proposed method with different combinations of modalities. Code is available at https://github.com/Jiahuiqu/Bi-DiffCD.

Multimodal remote sensing image classification (RSIC) has emerged as a key focus in Earth observation, driven by its capacity to extract complementary information from diverse sources. Existing methods struggle with modality absence caused by weather or equipment failures, leading to performance degradation. As a solution, knowledge distillation-based methods train student networks (SN) using a full-modality teacher, but they usually require training separate SN for each modality absence scenario, increasing complexity. To this end, we propose a unified Distillation Prompt Mamba (DPMamba) framework for multimodal RSIC with missing modalities. DPMamba leverages knowledge distillation in a shared text semantic space to optimize learnable prompts, transforming them from ``placeholder" to ``adaptation" states by enriching missing modality information with full-modality knowledge. To achieve this, we focus on two main aspects: first, we propose a new modality-aware Mamba for dynamically and hierarchically extracting cross-modality interactive features, providing richer, contextually relevant representations for backpropagation-based optimization of prompts; and second, we introduce a novel text-bridging distillation method to efficiently transfer full-modality knowledge, guiding the inclusion of missing modality information into prompts. Extensive evaluations demonstrate the effectiveness and robustness of the proposed DPMamba.

Due to the high cost of data collection and training, the well-performed hyperspectral image (HSI) classification models are of great value and vulnerable to piracy threat during transmission and use. Model watermarking is a promising technology for intellectual property (IP) protection of models. However, the existing model watermarking methods for RGB image classification models ignore the complexity of ground objects and high dimension of HSIs, which makes trigger samples easy to be detected and forged. To address this problem, we propose a signature diffusion embedded dual-verification watermarking method, which generates imperceptible trigger samples with explicit owner information to achieve dual verification of both model ownership and legality of trigger set. Specifically, the subpixel-space owner signature diffusion incorporated imperceptible trigger set generation method is proposed to manipulate owner signature incorporated to the abundance matrix of seeds via diffusion model in subpixel space, thus balancing the perceptual quality of trigger samples and signature extraction capability. To resist ownership confusion, dual-stamp ownership verification is proposed to query the suspicious model with trigger samples for ownership verification, and further extracts signature from trigger samples to guarantee their legality. Extensive experiments demonstrate the proposed method can effectively protect IP of HSI classification models.

Hyperspectral image super-resolution (HSI-SR) has attracted significant attention in high-resolution HSI reconstruction. Most existing fusion-based HSI-SR methods assume that multi-source images are perfectly registered, which is impractical due to varying imaging conditions. Furthermore, methods that consider the registration issue typically treat registration and fusion as two separate steps, resulting in the accumulation of registration errors during the fusion process. To address these issues, we propose a Cycle-Consistent Mamba-Based Registration-Fusion Joint Network (CCM-RFJN), which step-wise optimizes the Registration-Fusion Unified Module (RFUM) through multiple cyclic iterative SR processes. Specifically, in each SR iteration, we map the super-resolved HR-HSI obtained through the RFUM back to the unregistered LR-HSI for the next SR, with cycle-consistency constraints imposed on both LR-HSI and HR-HSI to adaptively optimize the RFUM based on the reciprocal training strategy. In RFUM, we integrate the proposed Interactive Mamba Registration Module (IMR) and Dual-attention Mamba Fusion Module (DAMF), thereby achieving registration-fusion joint optimization. Specifically, IMR is developed to incorporate the interactive Mamba encoder into a pyramid architecture to facilitate multi-level information interactions, generating the deformation field to correct non-rigid misalignments. DAMF is designed to utilize the dual-attention Mamba mechanism to highlight and aggregate key features, thereby enhancing fusion performance. Experiments on three public datasets demonstrate that CCM-RFJN achieves the state-of-the-art performance. The code is available at https://github.com/Jiahuiqu/CCM-RFJN.

Currently, fusion-based hyperspectral image super-resolution (fusion-based HSI-SR) has become an efficient technology to improve the spatial resolution of hyperspectral images. However, in real scenarios, it may not be possible to obtain high-resolution multispectral images (HR-MSI) of the same temporal and region corresponding to low-resolution hyperspectral images (LR-HSI) due to the limitations of imaging conditions and environmental changes. In view of this spatial-temporal constraint, it becomes a feasible solution to regard HR-MSI, which has similar spatial structure and semantics to LR-HSI, as a reference to assist in reconstruction. Therefore, this paper proposes a Cross-Correlation & Self-Similarity Guided Texture Transfer Network (C2S2TNet), which utilizes the texture details of HR-MSI and the self-similarity information of LR-HSI to achieve reference-based hyperspectral image super-resolution. Specifically, we design a Cross-Correlation & Self-Similarity Guided Cluster-Aware Matching (C2S2CAM) strategy, which realizes multi-correspondence texture matching and feature aggregation in non-local regions based on dynamic clustering and cluster-aware graph structure, effectively alleviating the misuse and underuse of information. In addition, we also propose a Spectral-Spatial State-Space Fusion Module (S2-SSFM) based on the state-space model to perform feature fusion and enhancement in both spatial and spectral domains to ensure that the target HR-HSI maintains the spatial-spectral structural consistency with the LR-HSI. Experimental verification shows that C2S2TNet can achieve excellent performance in cross-temporal and cross-regional scenarios, confirming the effectiveness of this method. Code can be accessed at https://github.com/Jiahuiqu/C2S2TNet.

Multi-task learning (MTL) serves as a effective technology to improving both the performance of hyperspectral image (HSI) pansharpening and that of downstream classification tasks. However, most of the existing MTL frameworks overlook the close relationship between the two tasks, resulting in suboptimal performance. To solve the problem, we propose a progressive mutual-guided Mamba MTL framework (MambaMTL), which achieves mutual enhancement between the two tasks by interacting the refined spatial-spectral features and the rich class semantic information generated by each task. Specifically, we design a dual-branch Mamba-Unet subnetwork with class-aware refinement (MTL-CRMNet) for HSI pansharpening. It incorporates parallel multi-head class-aware refinement layers (MCRLs) to enhance object details in the reconstructed HSI by enriching class-specific features with semantic information from the classification network, thereby improving pansharpening performance. Besides, we introduce a Mamba-based pyramid subnetwork with hierarchical guided fusion (MTL-MGFNet) for HSI classification. It utilizes the pansharpened HSI to guide multi-scale feature fusion in a feature pyramid manner, enhancing classification accuracy. Notably, both MTL-MGFNet and MTL-CRMNet share the same multi-scale feature encoder, enabling the classification subnetwork to generate spatial-spectral features rich in texture and semantics. The interaction between tasks significantly enhances their respective performance. The evaluation results conducted on relevant datasets illustrate the superiority of the proposed MambaMTL in simultaneously achieving better HSI classification accuracy and higher reconstructed image quality.

The existing methods of hyperspectral image (HSI) and multispectral image (MSI) fusion usually overlook the fact that multi-source images acquired under different imaging conditions are generally not perfectly registered. Despite the many such methods that have begun to address registration issues, it is still a challenge that most works perform registration and fusion as two separate steps, resulting in a cumulative error. To address this challenge, we propose a progressive registration-fusion co-optimization A-Mamba network (PRFCoAM), which iteratively optimizes the modal-aligned progressively registration-fusion (MAPRF) module to adaptively corrects the deformation from an extensive to a detailed level and refines the fusion results at each level to achieve progressive registration-fusion co-optimization. The proposed MAPRF module integrates the modal unified local aware registration (MULAR) block and interactive attention Mamba fusion (IAMF) block, which facilitates the network comprehensively and efficiently capture features of different levels. Specifically, MULAR adaptively learns spectral and spatial degradation functions to transform the input images into a unified modality and progressively repairs non-rigid pixel offsets by capturing the correlations and differences between corresponding regions of images. IAMF multi-directionally scans the spatial and spectral global dependent features of the well-registered images, which can stimulate the potential of Mamba in fusion and achieve a win-win situation of computational efficiency and selectivity advantages in the global acceptance domain. Extensive experiments demonstrate PRFCoAM can flexibly deal with different degrees and kinds of non-rigid deformation and achieves state-of-the-art performance. The code will be available at https://github.com/Jiahuiqu/PRFCoAM-for-HSI-MSI-Registration-Fusion.

Change detection of heterogeneous remote sensing images is a crucial topic for Earth observation, which has various applications in many fields. Most of the existing heterogeneous change detection methods obtain modal-consistent feature representation without fully considering the characteristic of specific data modality, such as hyperspectral image (HSI). Moreover, the acquirement of labeled samples requires high costs of manual operation and extensive domain knowledge. To solve these problems, we propose a cyclic consistency constrained multi-view graph matching network (C3MGM-Net) for unsupervised change detection, which fully considers the spatial-spectral similarity of heterogeneous multi-temporal images from multiple views while preventing the information loss of HSI and PAN/RGB image. The C3MGM-Net transforms the heterogeneous images into three common domains for modal alignment, which not only enhances the spatial-spectral information, but also well preserves the original high-resolution spatial and spectral information in the multi-temporal images. The modal-consistent spatial and spectral information is interacted between multiple domains, so as to make the difference features more distinguishable in terms of both structural and node similarity. With the guidance of change detection results in all domains, the most informative samples are intelligently selected to enlarge the training set, and then fed back to further constrain the consistency of unchanged areas of the multi-temporal images in each domain. The experimental results on heterogeneous datasets demonstrate the effectiveness of the proposed method compared with the state-of-the-art methods. Code is available at https://github.com/Jiahuiqu/C3MGM-for-Heterogeneous-Change-Detection.

Fusion of hyperspectral and multispectral images (HSI and MSI) has become a mainstream technique for obtaining high spatial resolution HSI. However, most existing fusion methods are based on the ideal assumption that the low-resolution HSI (LR-HSI) and high-resolution MSI (HR-MSI) are perfectly registered, which is difficult to achieve in the practical scenario. Some works attempt to fuse unregistered images, treating registration and fusion as independent tasks, however this will lead to the accumulation of registration errors in the fusion result. In this work, we propose a Progressive Synergistic Registration and Fusion Diffusion Network, called PSRF-DiffNet, to iteratively co-optimize image registration and fusion throughout the reverse process in a progressive bridging diffusion model. Specifically, a novel Fine Registration and Fusion Bridging model (FRFB) is designed, which models the image registration and fusion problem in a single model to greatly reduce the sensitivity of fusion performance to registration accuracy. In FRFB, Attention-enhanced-matching Fine Registration model (AFR) is proposed to generate registered images, on which, the Spatial-Spectral-Coupled Fusion model (S2CF) performs fusion operation to generate high-quality hyperspectral images. FRFB further allows direct connections from the state of preceding FRFB to the current FRFB, leading to a contiguous refinement mechanism. Extensive experimental results indicate that our proposed PSRF-DiffNet could effectively deal with different types and degrees of nonrigid deformations and yield high-quality HR-HSI, demonstrating a better performance than other state-of-the-art baselines. The code is available at https://github.com/Jiahuiqu/PSRF-DiffNet.

In the field of hyperspectral image super-resolution (HSI-SR), fusion-based image super-resolution (SR) is a research hotspot different from single image super-resolution (SISR). However, it is common for hyperspectral images (HSI) and multi-spectral images (MSI) to cover different areas and be acquired at different times. Such image pairs pose two challenges to the SR task, which are how to match similar regions of two images and to achieve efficient texture transfer in the complex texture details of remote sensing images. We define this task as unpaired HSI-SR and propose a cross-modal texture matching and enhancement mamba diffusion model (CTMEM-Diff) to solve the above problems. In the framework of the diffusion model, CTMEM-Diff gradually transfers texture details in MSI in an iterative manner to generate high-resolution HSI. Specifically, at each step of the iterative process, we propose a degradation matching block (DMB), which employs different degradation functions for different modal images to convert HSI and MSI to a uniform data dimension to improve the performence of texture matching. In addition, in order to fully utilize the spectral information in HSI, as well as the spatial information in MSI, we design a texture consistent mamba denoising module (TCMM) that utilizes the global context modeling capability and a gating mechanism of mamba to achieve efficient texture transfer. We constructed three datasets based on five publicly available datasets to validate the effectiveness of our model, and the experimental results show that our method achieves optimal results. The source code is available at https://github.com/Jiahuiqu/CTMEM-Diff.

Hyperspectral image change detection (HSI-CD) benefits from HSIs with continuous spectral bands, which uniquely enables the analysis of more subtle changes. Existing methods have achieved desirable performance relying on multi-temporal homogenous HSIs over the same region, which is generally difficult to obtain in real scenes. HSI-RGB multimodal CD overcomes the constraint of limited HSI availability by incorporating another temporal RGB data, and the combination of advantages within different modalities enhances the robustness of detection results. Nevertheless, due to the different imaging mechanisms between two modalities, existing HSI CD methods cannot be directly applied. In this paper, we propose a cycle translation-based collaborative training (co-training) for HSI-RGB multimodal CD, which achieves cross-modal mutual guidance to collaboratively learn complementary difference information from diverse modalities for identifying changes. Specifically, a cross-modal guided CycleGAN-based image translation module is designed to implement bi-directional image translation, which mitigates modal difference and enables the extraction of information related to land cover changes. Then, a spatial-spectral interactive co-training CD module is proposed to achieve iterative interaction between cross-modal information, which jointly extracts the multimodal difference features to generate the final results. The proposed method outperforms several leading CD methods in extensive experiments carried out on both real and synthetic datasets. In addition, a new public HSI-RGB multimodal dataset along with our code are available at https://github.com/Jiahuiqu/CT2Net.

Hyperspectral images (HSI) change detection (CD) has become a powerful tool to analyze the sublte surface changes. However, the application of HSI CD is constrained by the limited availability of homogeneous HSIs. HSI-RGB multimodal CD address these limitations by collaboratively utilizing multi-source data. Although multimodal CD methods have achieved encouraging results, their performance often relies on the assumption that the training and test samples have similar distributions. Recently, some domain adaptive CD methods have been introduced. However, the additional modality differences in cross-domain multimodal CD pose challenges to existing domain adaptation techniques. To address these challenges, we propose a cycle-based frequency disentanglement diffusion model with self-training for cross-domain HSI-RGB multimodal CD, which explores a frequency-domain diffusion-driven self-training mechanism to enhance consistency in change representations across different modalities and domains. Specifically, a cyclic frequency domain disentanglement-based modality-domain alignment diffusion network is proposed to achieve modality and domain alignment within a unified diffusion framework. Subsequently, a curriculum-learning based self-training dual-domain CD network is designed to process the aligned images, which leverages pseudo-label reliability to ensure stable transfer of prior knowledge while exploits complementary features across modalities for collaborative CD. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art approaches in cross-domain multimodal CD tasks.

The fusion of hyperspectral image (HSI) and multispectral image (MSI) is an effective mean to improve the inherent defect of low spatial resolution of HSI. However, existing fusion methods usually rigidly upgrade the spatial resolution of HSI to that of matching MSI under the ideal assumption that multi-source images are accurately registered. In real scenes where multi-source images are difficult to be perfectly registered and the spatial resolution requirements are dynamically different, these fusion algorithms is difficult to be effectively deployed. To this end, we construct the spatial-spectral consistent arbitrary scale observation model (S2cAsOM) to model the dependence between the unregistered HSI and MSI and the ideal arbitrary resolution HSI. Furthermore, an optimization algorithm is designed to solve S2cAsOM, and a deep interpretable arbitrary resolution fusion network (IR&ArF) is proposed to simulate the optimization process, which achieves the model-data dual-driven arbitrary resolution fusion of unregistered HSI and MSI. IR&ArF breaks the dependence of traditional fusion methods on the accuracy of image registration in a robust way, and can flexibly cope with the dynamic requirements of diverse applications for the spatial resolution of HSI, which improves the application ability of HSI fusion in real scenes. Extensive systematic experiments demonstrate the superiority and generalization of the proposed method. Source code of the proposed method is available on https://github.com/Jiahuiqu/IR-ArF.

Hyperspectral image (HSI) super-resolution through the fusion of low-resolution HSI (LrHSI) and high-resolution multispectral image (HrMSI) has emerged as a critical technique for enhancing the quality of HSIs. The recent progress in this field predominantly assume a known mapping relationships between high-resolution HSI (HrHSI) and low-resolution version, relying on networks to learn this mapping to generate HrHSI. However, this assumption is often unrealistic in practical applications. To address this limitation, we propose the Spatial-Spectral-Integrated Transparent Diffusion Model (S2TD) for blind HSI-SR, which is more adaptive to scene-variant degradations with a universal framework for both spatial and spectral reconstruction. Specifically, we design a multi-order degradation pool to generate diverse samples, thereby reducing the distribution gap between low-resolution images in real scenes. Additionally, we develop a spatial-spectral consistent degradation model, which is iteratively solved using an optimization algorithm and unrolled into neural networks for separate restoration in spatial and spectral aspects. Furthermore, the capabilty of progressive reconstruction in the diffusion model is involved to fit various degradations in different dimensions using similar network architectures, thereby enhancing the overall robustness of the network to various and complex scenarios. Comprehensive experiments conducted on three publicly synthetic datasets and one real-world dataset validate the superior performance of the proposed method under the condition that the degradation remains unknown.

Hyperspectral (HS) pansharpening aims at fusing an observed HS image with a panchromatic (PAN) image, to produce an image with the high spectral resolution of the former and the high spatial resolution of the latter. Most of the existing convolutional neural networks (CNNs)-based pansharpening methods reconstruct the desired high-resolution image from the encoded low-resolution (LR) representation. However, the encoded LR representation captures semantic information of the image and is inadequate in reconstructing fine details. How to effectively extract high-resolution and LR representations for high-resolution image reconstruction is the main objective of this article. In this article, we propose a feature pyramid fusion network (FPFNet) for pansharpening, which permits the network to extract multiresolution representations from PAN and HS images in two branches. The PAN branch starts from the high-resolution stream that maintains the spatial resolution of the PAN image and gradually adds LR streams in parallel. The structure of the HS branch remains highly consistent with that of the PAN branch, but starts with the LR stream and gradually adds high-resolution streams. The representations with corresponding resolutions of PAN and HS branches are fused and gradually upsampled in a coarse to fine manner to reconstruct the high-resolution HS image. Experimental results on three datasets demonstrate the significant superiority of the proposed FPFNet over the state-of-the-art methods in terms of both qualitative and quantitative comparisons.

For hyperspectral image (HSI) and multispectral image (MSI) fusion, it is often overlooked that multisource images acquired under different imaging conditions are difficult to be perfectly registered. Although some works attempt to fuse unregistered images, two thorny challenges remain. One is that registration and fusion are usually modeled as two independent tasks, and there is no yet a unified physical model to tightly couple them. Another is that deep learning (DL)-based methods may lack sufficient interpretability and generalization. In response to the above challenges, we propose an unregistered HSI fusion framework energized by a unified model of registration and fusion. First, a novel registration-fusion consistency physical perception model (RFCM) is designed, which uniformly models the image registration and fusion problem to greatly reduce the sensitivity of fusion performance to registration accuracy. Then, an HSI fusion framework (MoE-PNP) is proposed to learn the knowledge reasoning process for solving RFCM. Each basic module of MoE-PNP one-to-one corresponds to the operation in the optimization algorithm of RFCM, which can ensure clear interpretability of the network. Moreover, MoE-PNP captures the general fusion principle for different unregistered images and therefore has good generalization. Extensive experiments demonstrate that MoE-PNP achieves state-of-the-art performance for unregistered HSI and MSI fusion. The code is available at https://github.com/Jiahuiqu/MoE-PNP.

Hyperspectral change detection, which provides abundant information on land cover changes in the Earth's surface, has become one of the most crucial tasks in remote sensing. Recently, deep-learning-based change detection methods have shown remarkable performance, but the acquirement of labeled data is extremely expensive and time-consuming. It is intuitive to learn changes from the scene with sufficient labeled data and adapting them into an unlabeled new scene. However, the nonnegligible domain shift between different scenes leads to inevitable performance degradation. In this article, a cycle-refined multidecision joint alignment network (CMJAN) is proposed for unsupervised domain adaptive hyperspectral change detection, which realizes progressive alignment of the data distributions between the source and target domains with cycle-refined high-confidence labeled samples. There are two key characteristics: 1) progressively mitigate the distribution discrepancy to learn domain-invariant difference feature representation and 2) update the high-confidence training samples of the target domain in a cycle manner. The benefit is that the domain shift between the source and target domains is progressively alleviated to promote change detection performance on the target domain in an unsupervised manner. Experimental results on different datasets demonstrate that the proposed method can achieve better performance than the state-of-the-art change detection methods.

Hyperspectral image super-resolution (HSI-SR) is a technology to improve the spatial resolution of HSI. Existing fusion-based SR methods have shown great performance, but still have some problems as follows: 1) existing methods assume that the auxiliary image providing spatial information is strictly registered with the HSI, but images are difficult to be registered finely due to the shooting platforms, shooting viewpoints and the influence of atmospheric turbulence; 2) most of the methods are based on convolutional neural networks (CNNs), which is effective for local features but cannot utilize the global features. To this end, we propose a multi-modal cross-scale deformable transformer network (M2DTN) to achieve unregistered HSI-SR. Specifically, we formulate a spectrum-preserving based spatial-guided registration-SR unified model (SSRU) from the view of the realistic degradation scenarios. According to SSRU, we propose multi-modal registration deformable module (MMRD) to align features between different modalities by deformation field. In order to efficiently utilize the unique information between different modals, we design multi-scale feature transformer (MSFT) to emphasize the spatial-spectral features at different scales. In addition, we propose the cross-scale feature aggregation module (CSFA) to accurately reconstruct the HSI by aggregating feature information at different scales. Experiments show that M2DTN outperforms the-state-of-the-art HSI-SR methods. Code is obtainable at https://github.com/Jiahuiqu/M2DTN.

Hyperspectral image super-resolution (HISR) is a technique that can break through the limitation of imaging mechanism to obtain the hyperspectral image (HSI) with high spatial resolution. Although some progress has been achieved by existing methods, most of them directly learn the spatial-spectral joint mapping between the observed images and the target high-resolution HSI (HrHSI), failing to fully reserve the spectral distribution of low-resolution HSI (LrHSI) and the spatial distribution of high-resolution multispectral imagery (HrMSI). To this end, we propose a spatial-spectral-bilateral cycle-diffusion framework (S2CycleDiff) for HISR, which can step-wise generate the HrHSI with high spatial-spectral fidelity by learning the conditional distribution of spatial and spectral super-resolution processes bilaterally. Specifically, a customized conditional cycle-diffusion framework is designed as the backbone to achieve the spatial-spectral-bilateral super-resolution by repeated refinement, wherein the spatial/spectral guided pyramid denoising (SGPD) module seperately takes HrMSI and LrHSI as the guiding factors to achieve the spatial details injection and spectral correction. The outputs of the conditional cycle-diffusion framework are fed into a complementary fusion block to integrate the spatial and spectral details to generate the desired HrHSI. Experiments have been conducted on three widely used datasets to demonstrate the superiority of the proposed method over state-of-the-art HISR methods. The code is available at https://github.com/Jiahuiqu/S2CycleDiff.

Recent research on the joint classification of multimodal remote sensing data has achieved great success. However, due to the limitations imposed by imaging conditions, the case of missing modalities often occurs in practice. Most previous researchers regard the classification in case of different missing modalities as independent tasks. They train a specific classification model for each fixed missing modality by extracting multimodal joint representation, which cannot handle the classification of arbitrary (including multiple and random) missing modalities. In this work, we propose a local diffusion shared-specific autoencoder (LDS2AE), which solves the classification of arbitrary missing modalities with a single model. The LDS2AE captures the data distribution of different modalities to learn multimodal shared feature for classification by designing a novel local diffusion autoencoder which consists of a modality-shared encoder and several modality-specific decoders. The modality-shared encoder is designed to extract multimodal shared feature by employing the same parameters to map multimodal data into a shared subspace. The modality-specific decoders put the multimodal shared feature to reconstruct the image of each modality, which facilitates the shared feature to learn unique information of different modalities. In addition, we incorporate masked training to the diffusion autoencoder to achieve local diffusion, which significantly reduces the training cost of model. The approach is tested on widely-used multimodal remote sensing datasets, demonstrating the effectiveness of the proposed LDS2AE in addressing the classification of arbitrary missing modalities. The code is available at https://github.com/Jiahuiqu/LDS2AE.

Considering the existing issues in unregistered hyper-spectral images (HSI) and multispectral images (MSI) fusion methods: i) the designed registration modules introduce a significant computational burden, and registration errors accumulate in fusion errors; ii) the methods lack model guidance, resulting in poor interpretability of the network. In this paper, we propose a matrix factorization informed interpretable deep network to address the challenges of unregistered HSI and MSI fusion (IUFNet). In particular, we derive an extended matrix factorization model for unregistered fusion (EUMF), which substitutes the abundance matrix of HSI containing low-resolution and distorted spatial information by the high-resolution abundance matrix of MSI. This substitution ingeniously eliminates the dependence of fusion performance on registration accuracy. Subsequently, IUFNet is designed to unfold the iterative results obtained by proximal gradient descent into the deep learning network, where each operation has a clear physical meaning. Overall, this network achieves the fusion of unregistered HSI and MSI and exhibits inter-pretability. Experimental results on the widely used Paiva Center dataset demonstrate the effectiveness and superiority of the proposed method.

Hyperspectral image super-resolution (HSISR) has shown very promising potential for earth observation and deep space exploration tasks. However, most existing HSISR methods formulate HSISR tasks with different scale factors as independent tasks, and train a specific model for each scale factor. In this letter, we propose a coarse-to-fine meta diffusion HSISR method, termed as CFMDM, which is capable of solving the problem of HSISR with scale-arbitrary factors in a unified model. The proposed CFMDM is composed of a coarse-to-fine upsampling module. The module encompasses two pivotal units: a coarse meta upsampling unit that utilizes meta-learning to map features of arbitrary scales to the corresponding scales, and a gradual refinement diffusion unit, which is designed to refine the details of the reconstructed HSI. In addition, we develop an imaging model-driven downsampling algorithm for generating training samples tailored to practical applications. The proposed method performs well in both quantitative and qualitative evaluation on benchmark datasets, achieving the average PSNR of 41.45dB at 1.5x super-resolution for the CAVE dataset.

Due to its capability to monitor subtle changes occurring on the Earth’s surface, hyperspectral images change detection (HSI-CD) has emerged as a focal research area in the field of remote sensing. Recently, diffusion models have demonstrated remarkable performance in the field of HSI-CD. However, vanilla diffusion models are mostly constructed by CNN, which struggles to model global context relationships in complex scenes to result in limited change detection accuracy. In order to overcome the shortcomings about vanilla diffusion models, we innovatively design graph representation learning-guided diffusion model (GDM) and propose the GDM-based HSI-CD network (GDMCD). Specially, we utilize graph convolutional to construct the GDM as the feature extractor, which can adequately extract global difference features of HSIs. Then, we design the difference perception amplification module (DPAM) to increase the distinction between difference features extracted by GDM. Finally, we obtain the change map by classifying difference features which are processed by DPAM. Experiments conducted on three publicly available datasets with 1% sample size demonstrate that the proposed method outperforms the other state-of-the-art methods in terms of Overall Accuracy (OA), Kappa Coefficient (KC) achieving improvements of approximately 0.006%, 1.61%, and 0.34%, respectively.

Hyperspectral target detection aims to locate targets of interest in the scene, and deep learning-based detection methods have achieved the best results. However, black box network architectures are usually designed to directly learn the mapping between the original image and the discriminative features in a single data-driven manner, a choice that lacks sufficient interpretability. On the contrary, this article proposes a novel deep spatial-spectral joint-sparse prior encoding network (JSPEN), which reasonably embeds the domain knowledge of hyperspectral target detection into the neural network, and has explicit interpretability. In JSPEN, the sparse encoded prior information with spatial-spectral constraints is learned end-to-end from hyperspectral images (HSIs). Specifically, an adaptive joint spatial-spectral sparse model (AS2JSM) is developed to mine the spatial-spectral correlation of HSIs and improves the accuracy of data representation. An optimization algorithm is designed for iteratively solving AS2JSM, and JSPEN is proposed to simulate the iterative optimization process in the algorithm. Each basic module of JSPEN one-to-one corresponds to the operation in the optimization algorithm so that each intermediate result in the network has a clear explanation, which is convenient for intuitive analysis of the operation of the network. With end-to-end training, JSPEN can automatically capture the general sparse properties of HSIs and faithfully characterize the features of background and target. Experimental results verify the effectiveness and accuracy of the proposed method. Code is available at https://github.com/Jiahuiqu/JSPEN.

Hyperspectral target detection is a binary classification problem of detecting targets by utilizing the spectral characteristics of hyperspectral images (HSIs). However, in real-world applications, there is more than one class of interest in the same scene. The accuracy of detection for the previously learned target classes may be decreased when the model is retrained for detecting new target classes in scenes, which is called catastrophic forgetting. Consequently, how to ensure that the model has high detection performance for previously learned targets while learning new ones has become a key challenge for hyperspectral multitarget detection tasks. In this article, we propose an incremental detection of hyperspectral targets (IDHTs) method based on continual learning. IDHT decomposes the multitarget detection task into a series of independent subtasks and learns them sequentially. Within each subtask, our proposed incremental spectral detector (ISD) enables the training and learning of new class targets. Simultaneously, we introduce the previous label replay strategy (PLRS), which synthesizes fused labels for the current task training by combining detection outcomes from the previous model with pseudo-labels of the current target. PLRS effectively bridges the knowledge gap across various subtasks in hyperspectral multitarget detection tasks. The proposed IDHT can flexibly and dynamically adapt to new categories and overcome the limitations of fixed-category feature learning. In addition, two hyperspectral datasets are disclosed to evaluate the proposed method. Our method demonstrates significant effectiveness and superiority on both public datasets and two self-collected datasets. Our code and dataset are available athttps://github.com/Jiahuiqu/IDHT.

Hyperspectral image (HSI) super-resolution (SR) employing the denoising diffusion probabilistic model (DDPM) holds significant promise with its remarkable performance. However, existing relevant works exhibit two limitations: i) Directly applying DDPM to fusion-based HSI SR (HSI-SR) ignores the physical mechanism of HSI-SR and unique characteristics of HSI, resulting in less interpretability; ii) Scale-invariant DDPM suffers from a time-consuming inference. To tackle these issues, we propose an interpretable scale-propelled diffusion model (ISPDiff) for HSI-SR, which combines the underlying principles of HSI-SR with DDPM for progressively unrolling reconstruction by learning its distribution at various scales, enhancing the transparency significantly and reducing the inference time prominently. Concretely, we destroy and downsample HSI into Gaussian noise in the forward process of ISPDiff. Then we design a unified scale-flexible model in the backward process to iteratively refine HSI in a coarse-to-fine manner through scale-matched reconstruction and cross-scale upsampling, which can be unfolded with optimization algorithms. These solved equations are one-to-one corresponding unrolled into two deep neural networks, called progressive perceptual model-driven scale-matched restoration network (P2MSRN) and cross-scale model-driven upsampling network (CMUN). Through end-to-end training, the proposed ISPDiff implements HSI-SR with a scale-propelled unrolling diffusion characterized by enhanced interpretability, stronger task orientation, and reduced time consumption. Systematic experiments have been conducted on three public datasets, demonstrating that ISPDiff outperforms state-of-the-art methods. Code is available at https://github.com/Jiahuiqu/ISPDiff.

Domain adaptation (DA) aims to transfer knowledge from a labeled source domain (SD) to an unlabeled target domain (TD), and its effectiveness has been demonstrated in unsupervised multisource remote sensing image classification. Existing DA frameworks simultaneously learn the mapping between data in SD and category labels, as well as minimize the distribution discrepancy between different domains. However, significant computational resources are needed to optimize the dual objectives in a data-driven DA network. In addition, the lack of interpretability of deep learning (DL)-based methods results in unpredictable feature distributions, thereby impeding the smooth update of the network in the desired direction. To address these issues, we propose a contrastive constrained cross-scene model-informed interpretable classification strategy (C3MI-C) for hyperspectral image (HSI) and light detection and ranging (LiDAR), which achieves a model-interpretable decoupling of domain adaptive task and classification task. The proposed C3MI-C optimizes the classification network interpretably in the same subspace and further aligns deep-adapted features extracted from two domains to accomplish high-precision unsupervised cross-scene classification. Comparative experiment results and ablation studies show that C3MI-C performs better than other advanced methods.

Existing fusion-based hyperspectral image super-resolution (fusion-based HSI-SR) methods usually reconstruct high-resolution hyperspectral image (HR-HSI) by integrating the complementary information of low-resolution hyperspectral image (LR-HSI) and high-resolution multispectral image (HR-MSI). However, most of such methods rely on accurately registered images or consider registration and fusion as a two-stage task, which means that fusion must tolerate the accumulation of errors due to misregistration. In this paper, we propose a progressive multi-iteration registration-fusion co-optimization network (PMI-RFCoNet) for unregistered hyperspectral image super-resolution, which progressively refines the registration and fusion result over multiple levels to reconstruct registered HR-HSI. To achieve registration-fusion co-optimization, the registration-fusion cooptimization block (Co-RFB) is designed to iterate continuously over multiple levels. We embed the interactive registration module (IRM) and the spectral recalibration and fusion module (SRFU) in Co-RFB, which can facilitate the network utilizing spatial and spectral features at different levels to generate more accurate HR-HSI. Specifically, IRM generates deformation field based on spatial correlations captured at long distances to repair non-rigid pixel offsets, and SRFU further performs adaptive high-fidelity spectral correction and spatial information fusion on the registration results. We conduct experimental verification on four widely used datasets, and the results show that PMI-RFCoNet can flexibly cope with different types and degrees of non-rigid deformation and achieve superior performance. Code is available at https://github.com/Jiahuiqu/PMI-RFCoNet.

Deep learning (DL) has increasingly become the mainstream technology for hyperspectral image change detection (HSI-CD). However, these methods lack transparency and often overlook priors in hyperspectral images (HSIs), making it difficult to extract more generalizable features directly from them. Moreover, the performance of DL-based methods typically depends heavily on a large corpus of high-quality labeled data, which is often impractical and expensive in real-world scenarios, particularly for complex HSIs. To address these issues, we propose a semi-supervised deep interpretable network for HSI-CD. Specifically, by applying structured sparse prior constraints, we propose a multiscale convolutional sparse coding (MSCSC) model to capture shared and private sparse coefficients (SSC and PSCs) across different scales, extracting multiscale features while reducing redundancy. We then unfold the proposed MSCSC model to establish an MSCSC-guided deep interpretable network (MSCSCNet) that serves as the encoder, namely, MSCSCNet, in which each network module is model-driven, enhancing the transparency of internal mechanisms and extracting more fundamental features. In addition, we devise a two-stage semi-supervised training strategy for MSCSCNet using limited labeled data, combining a change-sensitive teacher-student self-distillation (CS-TSSD) paradigm with a novel loss function to reduce the annotation dependency and remain sensitive to change components. The proposed method not only enhances the transparency of multiscale feature extraction but also accomplishes semi-supervised learning to extract meaningful representations, effectively integrating the advantages of model-driven and data-driven approaches. Comparative experiments on benchmark datasets demonstrate the effectiveness of our method over the existing approaches. Code is available athttps://github.com/Jiahuiqu/MSCSCNet

The joint classification methods of hyperspectral image (HSI) and light detection and ranging (LiDAR) data based on deep learning have demonstrated exceptional classification performance with sufficient labeled samples. However, it is expensive and time-consuming to acquire labeled data. To address this limitation, we propose a shared-private decoupling-based multilevel feature alignment semisupervised (SASS) learning method for HSI and LiDAR classification, which introduces the idea of domain adaptation (DA) to capture the shared features of labeled and unlabeled data for classification and circularly selects reliable pseudolabels based on these features to retrain the model. Specifically, we treat labeled data as the source domain (SD) and unlabeled data as the target domain (TD) and propose a shared-private feature decoupling (SPFD) module to acquire shared representations of SD and TD by separating domain private features. The multilevel shared feature alignment (MSFA) strategy is designed to synthetically consider both spatial details and semantic information by minimizing the maximum mean discrepancy (MMD) between these shared features. In addition, we design a graph transformer-based class-balanced pseudolabel generation (GBPG) strategy for iterative model training with reliable pseudolabels, which exploits the graph transformer network-based sample acquisition (GTSA) strategy to select valuable samples and generate corresponding pseudolabels using the adaptive class-specific threshold-based sample annotation (ATSA) strategy. Experimental results on three public datasets validate the effectiveness of the proposed method. The code is available athttps://github.com/Jiahuiqu/SASS.

Recently, deep neural networks (DNNs) have been widely used in hyperspectral image change detection (HSI-CD). Generally, training such a DNN-based HSI-CD network often requires a large number of labeled training samples. However, it is time-consuming, labor-intensive, or even infeasible to label training samples in practice. In this article, we propose a feature mutual representation-based graph domain adaptive network (FGDANet) for unsupervised HSI-CD. This method constructs a pseudosiamese backbone consisting of two customized unsupervised learning domains, which can make full use of the information from different domains through the graph domain adaptation strategy to improve the feature expression capability and generalization. There are three key characteristics: first, in each customized unsupervised learning domain, a graph convolutional network (GCN)-based difference feature extraction architecture is designed to model the local and global dependence among the features of multitemporal HSIs; second, a progressive graph-to-pixel joint constraint strategy (PJCS) is proposed to provide the high-confidence training sample labels for the unsupervised learning of the network in each domain; and third, the homogeneous mutual representation joint graph feature alignment (HJGFA) module of the graph domain adaptation strategy can make full use of the difference features from the two domains through the information interaction to facilitate the model to capture the changed and unchanged essential characteristics. The experimental results on four HSI datasets demonstrate the superiority of the proposed FGDANet. Code is available athttps://github.com/Jiahuiqu/FGDANet.

Convolutional neural networks (CNNs) with good feature learning capabilities are widely used in hyperspectral image change detection (HSI-CD) tasks. However, most existing CNN-based HSI-CD methods face two inherent challenges: 1) the lack of available labeled datasets and 2) the limited receptive field that cannot capture the long-distance dependence between the spectral sequences of HSIs. In this article, we propose a parallel compared-and-stacked pyramid transformer network (PCPTNet) for unsupervised HSI-CD, which can model the context information of spectral sequences in the input multi-temporal HSI patch without real labeled data. Specifically, a superpixel-level joint decision-based training samples selection strategy is presented that fully considers the correlation between pixels to improve the reliability of training samples. Then, taking advantage of transformer in context information modeling, PCPTNet is proposed to capture sufficient difference features and stacked features with different scales for CD, which can effectively reduce missed and false detection. The multiscale features containing sufficient low-level detail information and high-level semantic features are fused hierarchically to classify changed and unchanged pixels. Extensive experiments on three real HSI datasets demonstrate that the PCPTNet outperforms other state-of-the-art HSI-CD methods in both visual and quantitative results.

The joint classification of hyperspectral images (HSIs) and LiDAR data plays a crucial role in earth observation missions. Most advanced methods are based on discrete label supervision. However, since discrete labels only convey limited information that a sample belongs to a single definite class and lack of prior information, it is difficult to supervise the model to capture rich inherent semantic information in complex data distributions, hindering the classification performance. To this end, we propose a text-supervised multidimensional contrastive fusion network, termed as TMCFN, which leverages class text information to guide the learning of visual representations while establishing a semantic association of text and visual features for classification by using multidimensionally incorporated contrastive learning (CL) paradigms. Specifically, TMCFN is composed of text information encoding (TIE), visual features representation (VFR) and text-visual features alignment and classification (TVFAC). TIE is employed to extract semantic information from class text extended from class names, intrinsic attributes and inter-class relationships. VFR mainly comprises a new fusion-based contrastive feature learning module (FCFLM) to extract discriminative visual features and a text-guided attention feature fusion module (TAF2M) to fuse visual features under the guidance of text information. TVFAC optimizes the learning of visual features under the supervision of text information while using a CL paradigm to align text and visual features for establishing the semantic association, and achieves the classification by directly computing the similarity between the visual features and each text feature without an additional classifier. Experiments with three standard datasets verify the effectiveness of TMCFN.

The unsupervised domain adaptation (UDA) based cross-scene remote sensing image classification has recently become an appealing research topic, since it is a valid solution to unsupervised scene classification by exploiting well-labeled data from another scene. Despite its good performance in reducing domain shifts, UDA in multisource data scenarios is hindered by several critical challenges. The first one is the heterogeneity inherent in multisource data complicates domain alignment. The second challenge is the incomplete representation of feature distribution caused by the neglect of the contribution from global information. The third challenge is the inaccuracies in alignment due to errors in establishing target domain conditional distributions. Since UDA does not guarantee the complete consistency of the distribution of the two domains, networks using simple classifiers are still affected by domain shifts, resulting in poor performance. In this paper, we propose a graph embedding interclass relation-aware adaptive network (GeIraA-Net) for unsupervised classification of multi-source remote sensing data, which facilitates knowledge transfer at the class level for two domains by leveraging aligned features to perceive inter-class relation. More specifically, a graph-based progressive hierarchical feature extraction network is constructed, capable of capturing both local and global features of multisource data, thereby consolidating comprehensive domain information within a unified feature space. To deal with the imprecise alignment of data distribution, a joint de-scrambling alignment strategy is designed to utilize the features obtained by a three-step pseudo-label generation module for more delicate domain calibration. Moreover, an adaptive inter-class topology based classifier is constructed to further improve the classification accuracy by making the classifier domain adaptive at the category level. The experimental results show that GeIraA-Net has significant advantages over the current state-of-the-art cross-scene classification methods.

Spatio-spectral fusion of panchromatic (PAN) and hyperspectral (HS) images is of great importance in improving spatial resolution of images acquired by many commercial HS sensors. DenseNets have recently achieved great success for image super-resolution because they facilitate gradient flow by concatenating all the feature outputs in a feedforward manner. In this article, we propose a residual hyper-dense network (RHDN) that extends the DenseNet to solve the spatio-spectral fusion problem. The overall structure of the proposed RHDN method is a two-branch network, which allows the network to capture the features of HS images within and outside the visible range separately. At each branch of the network, a two-stream strategy of feature extraction is designed to process PAN and HS images individually. A convolutional neural network (CNN) with cascade residual hyper-dense blocks (RHDBs), which allows direct connections between the pairs of layers within the same stream and those across different streams, is proposed to learn more complex combinations between the HS and PAN images. The residual learning is adopted to make the network efficient. Extensive benchmark evaluations well demonstrate that the proposed RHDN fusion method yields significant improvements over many widely accepted state-of-the-art approaches.

Hyperspectral (HS) spatial-spectral fusion technology is an important means to obtain high spatial and spectral resolution data for major strategic missions such as manned spaceflight and earth observation. The existing parameter-tuning black-box fusion model lacks the guidance of mathematical theory and is not explicable, so it is difficult to apply in practice. In this letter, we design an interpretable deep network for spatial-spectral fusion tasks, called S2Fusion, which embedded the mature component substitution (CS) fusion model into a deep neural network. As a result of the careful design of the method, each module in the S2Fusion network corresponds to a specific operation in the CS fusion model, which is easily interpretable. Compared with the traditional fusion models, the S2Fusion makes it easier to ensure the implementation mechanism of spatial-spectral fusion throughout the network flow. Extensive experiments demonstrate the superiority of S2Fusion both quantitatively and visually over state-of-the-art methods.

Hyperspectral image (HSI) change detection is a technique to observe the change information between the multitemporal HSIs, which is currently considered a major focus of research in the filed of remote sensing intelligent interpretation. Most existing deep learning-based methods have created satisfactory performance, but these methods lack transparency and have poor generalization. To tackle the problems outlined above, we propose a dictionary learning-guided deep interpretable network for hyperspectral change detection, which unfolds a dictionary learning-based change detection model into an interpretable deep neural network. Specifically, we first design a dictionary learning-based change detection model, whose solution process can be decomposed into two iterative subproblems. Then, the mathematical model can be unfolded into a dual-branch deep neural network with two modules iterating with each other. Finally, the difference map of the coefficients output from the ultimate stage is classified to obtain the change detection result. Experimental results prove that the proposed method has comparable or even better performance than state-of-the-art methods.

The effective utilization of hyperspectral image (HSI) and light detection and ranging (LiDAR) data is essential for land cover classification. Recently, deep learning-based classification approaches have achieved remarkable success. However, most deep learning classification methods are data-driven and designed in a black-box architecture, lacking sufficient interpretability, and ignoring the potential correlation of heterogeneous complementary information between multisource data. To address these issues, we propose an interpretable deep neural network, namely multisource aligning joint contextual representation model-informed interpretable classification network (MACRMoI-N), which fully exploits correlation of multisource data by aligning complementary spectral-spatial-elevation information during end-to-end training. We first present a multimodal aligning joint contextual representation classification model (MACR-M), which incorporates local spatial-spectral prior information into representation. MACR-M is optimized by an iterative algorithm to solve dictionaries of HSI and LiDAR and their corresponding sparse coefficients, in which the dictionary distribution are aligned to enable the complementary information of multisource data to guide a more accurate classification. We further propose the unfolded MACRMoI-N, where each module corresponds to a specific operation of the optimization algorithm, and the parameters are optimized in an end-to-end manner. Comparative experiment results and ablation studies show that MACRMoI-N performs better than other advanced methods.

Existing deep convolutional neural networks (CNNs) have recently achieved great success in pansharpening. However, most deep CNN-based pansharpening models are based on "black-box" architecture and require supervision, making these methods rely heavily on the ground-truth data and lose their interpretability for specific problems during network training. This study proposes a novel interpretable unsupervised end-to-end pansharpening network, called as IU2PNet, which explicitly encodes the well-studied pansharpening observation model into an unsupervised unrolling iterative adversarial network. Specifically, we first design a pansharpening model, whose iterative process can be computed by the half-quadratic splitting algorithm. Then, the iterative steps are unfolded into a deep interpretable iterative generative dual adversarial network (iGDANet). Generator in iGDANet is interwoven by multiple deep feature pyramid denoising modules and deep interpretable convolutional reconstruction modules. In each iteration, the generator establishes an adversarial game with the spatial and spectral discriminators to update both spectral and spatial information without ground-truth images. Extensive experiments show that, compared with the state-of-the-art methods, our proposed IU2PNet exhibits very competitive performance in terms of quantitative evaluation metrics and qualitative visual effects.

Hyperspectral image (HSI) change detection is a challenging task that focuses on identifying the differences between multi-temporal HSIs. The recent advancement of convolutional neural network (CNN) has made great progress on HSIs change detection. However, due to the limited receptive field, most CNN based change detection models trained with sufficient labeled samples cannot flexibly model the global information that is essential for distinguishing complex objects, thereby achieving relatively-low performance. In this paper, we propose a dual-branch local information enhanced graph-transformer change detection network to fully exploit the local-global spectral-spatial features of the multi-temporal HSIs with limited training samples for change recognition. Specifically, the proposed network is composed of a cascaded of local information enhanced graph-transformer (LIEG) blocks, which jointly extracts local-global features by learning local information representation to enhance the information of graph-transformer. A novel graph-transformer is developed to model global spectral–spatial correlation between graph nodes, enabling the spectral information preservation of HSIs and accurate change detection of areas with various sizes. Extensive experiments have proved that our method achieves significant performance improvement than other state-of-the-art methods on four commonly used HSI datasets.

Hyperspectral image (HSI) change detection is a technique for detecting the changes between the multitemporal HSIs of the same scene. Many existing change detection methods have achieved good results, but there still exist problems as follows: 1) mixed pixels exist in HSI due to the low spatial resolution of hyperspectral sensor and other external interference and 2) many existing deep learning-based networks cannot make full use of the correlation difference information between the bitemporal images. These problems are not conducive to further improving the accuracy of change detection. In this article, we propose an abundance matrix correlation analysis network based on hierarchical multihead self-cross-hybrid attention (AMCAN-HMSchA) for HSI change detection, which hierarchically highlights the correlation difference information at the subpixel level to detect the subtle changes. The endmember sharing-based abundance matrix learning module (AMLM) maps the changed information between bitemporal HSIs to the corresponding abundance matrices. The hierarchical MSchA extracts the enhanced difference features by constantly comparing the self-correlation with cross correlation between the abundance matrices of the HSIs. Then, the difference features are concatenated and fed into the fully connected layers to obtain the change map. Experiments on three widely used datasets show that the proposed method has superior performance compared with other state-of-the-art methods.

Well-known deep learning (DL) is widely used in fusion based hyperspectral image super-resolution (HS-SR). However, DL-based HS-SR models have been designed mostly using off-the-shelf components from current deep learning toolkits, which lead to two inherent challenges: i) they have largely ignored the prior information contained in the observed images, which may cause the output of the network to deviate from the general prior configuration; ii) they are not specifically designed for HS-SR, making it hard to intuitively understand its implementation mechanism and therefore uninterpretable. In this paper, we propose a noise prior knowledge informed Bayesian inference network for HS-SR. Instead of designing a "black-box" deep model, our proposed network, termed as BayeSR, reasonably embeds the Bayesian inference with the Gaussian noise prior assumption to the deep neural network. In particular, we first construct a Bayesian inference model with the Gaussian noise prior assumption that can be solved iteratively by the proximal gradient algorithm, and then convert each operator involved in the iterative algorithm into a specific form of network connection to construct an unfolding network. In the process of network unfolding, based on the characteristics of the noise matrix, we ingeniously convert the diagonal noise matrix operation which represents the noise variance of each band into the channel attention. As a result, the proposed BayeSR explicitly encodes the prior knowledge possessed by the observed images and considers the intrinsic generation mechanism of HS-SR through the whole network flow. Qualitative and quantitative experimental results demonstrate the superiority of the proposed BayeSR against some state-of-the-art methods.

Due to the side effects of asbestos on human health and environments, Italy has banned the use of asbestos-containing materials since 1992, but there are still illegal products with asbestos in daily life. In order to investigate the distributions of asbestos to facilitate its removal, this paper carries out asbestos identification with hyperspectral (HS) and panchromatic (PAN) data captured by the PRISMA satellite over Pavia, Italy. In this work, a pansharpening method with guided filter was used to inject more spatial details from 5m PAN to 30m HS. Then, the possible location of asbestos could be obtained by a bilinear sparse target detector (BSTD). Detection maps using BSTD are compared with that obtained by hierarchical constrained energy minimization (hCEM), ensuring the accuracy and reliability, also compared with the results using matched subspace detector with interaction effects (MSDinter) and adaptive MSD (AMSD) to verify the superiority of the bilinear sparse model.

Hyperspectral (HS) pansharpening is an attractive topic in the field of remote sensing, which has attracted the attention of many researchers. Component substitution (CS)-based HS pansharpening algorithms are of great interest due to their simplicity and high spatial quality, and they mainly consist of two phases: detail extraction and detail injection. Detail extraction is performed by estimating the intensity component, whereas detail injection depends on the definition of injection gain. In the classic CS-based pansharpening methods, the intensity component is estimated through a global synthesis scheme, and injection gains can be obtained by a context-adaptive or a global approach. In this letter, we propose an improved CS-based HS pansharpening method in which the intensity component and the injection gain are estimated locally achieved by the binary partition tree (BPT) image segmentation algorithm. The proposed method is applied to two credible CS-based HS pansharpening algorithms, including the Gram–Schmidt adaptive (GSA) and the Brovey transform (Brovey). The experimental results show that the proposed method improves the performance of GSA and Brovey and creates promising results perceptually and quantitatively.

Due to the side effects of asbestos on human health and environments, many countries have banned the use of asbestos-containing materials, but there are still illegal products with asbestos in daily life. In order to investigate the distributions of asbestos to facilitate its removal, this paper studies the feasibility of asbestos identification with HyperSpectral (HS) and panchromatic (PAN) data, taking images captured by the PRISMA and ZY1E 2D satellites over Pavia, Italy as examples. In this work, a pansharpening method with guided filter was used to improve HS image quality in terms of spectral fidelity and spatial details. Then, the possible location of asbestos could be obtained by a nonlinear target detector named BSTD. Considering high computational cost for large-scale remote sensing data processing, we further develop BSTD to its parallelized version (denoted as PBSTD). Given the groundtruth of asbestos over Pavia by the Regional Environmental Protection Agency-ARPA Lombardia, our PBSTD and several popular methods are evaluated from both qualitative and quantitative perspectives, showing that most algorithms could correctly detect large-size asbestos roofs, and the nonlinear PBSTD and MSDinter perform better in small-size asbestos identification than other linear detectors. However, the detection accuracy on small-size asbestos is insufficient in practical applications, which indicates that there are still issues to achieve accurate small-size asbestos identification using coarse-spatial-resolution spaceborne remote sensing.

Hyperspectral image (HSI) and light detection and ranging (LiDAR) data classification has attracted more and more attention in remote sensing. Convolution neural network (CNN) has been proven to be effective for HSI and LiDAR data classification. In this letter, a novel three-branch CNN is designed to learn spectral, spatial, and elevation features, each of which adopts the multi-level feature fusion (MLF) module to fuse the shallow and deep features. Furthermore, in order to fully fuse the spatial and elevation information, we propose a mutual guidance attention (MGA) module. The MGA module increases the information flow between spatial and elevation branches, highlights the features of interest, and weakens useless features. The proposed method is evaluated on public datasets Houston and Trento. Experimental results demonstrate that our proposed method can provide higher classification accuracy than some existing methods.

Convolutional neural networks (CNNs) have shown impressive performance in computer vision due to their non-linearity. Particularly, DenseNet that facilitates feature re-use in a feedforward manner has achieved state-of-the-art reconstruction accuracy for super-resolution (SR). However, most DenseNet based SR models transfer the features generated from each layer to all the subsequent layers, inevitably introducing redundancy, especially for high-dimensional hyperspectral (HS) images. To tackle this problem, we propose a two-branch cross-feedback dense network with context-aware guided attention (CFDcagaNet) for HS super-resolution (HSSR), which allows the network to learn the attention maps of high-level features and refine the low-level features in a feedback manner across two branches. Context-aware guided attention uses high-level posterior information to provide more faithful spatial-spectral guidance for low-level features, which enables CFDcagaNet to learn more effective spatial-spectral features at low levels and yield more effective spatial-spectral transfer in the network. Extensive experiments on widely-used datasets demonstrate that the proposed method outperforms state-of-the-art methods in terms of both quantitative values and visual qualities.

Hyperspectral (HS) image fusion aims at integrating a panchromatic (PAN) image and an HS image, featuring the fused image with the spatial quality of the former and the spectral diversity of the latter. The classic fusion algorithm generally includes three consecutive procedures that are upsampling, detail extraction, and detail injection. In this article, we propose an HS and PAN image fusion method based on generative adversarial network and local estimation of injection gain. Instead of upsampling the HS image by classical interpolation techniques, a generative adversarial super-resolution network (GASN) is designed to obtain the interpolated HS image in the fusion framework. GASN establishes a spectral-information-based discriminator to conduct adversarial learning with the generator, so as to preserve the spectral information of the low-resolution HS image. An image segmentation-based injection gain estimation (ISGE) algorithm is subsequently proposed for HS and PAN images fusion. The injection gain is estimated over image segments obtained by a binary partition tree approach to improve the fusion performance. The proposed GASN and ISGE are implemented into two credible global estimation pansharpening methods, and experimental results prove the performance improvement of the proposed method. The proposed method is also compared with existing state-of-the-art methods, and experiments on several public databases demonstrate that the proposed method is competitive or superior to the state-of-the-art fusion methods.

Deep learning, especially convolutional neural networks (CNNs), has shown very promising results for multispectral (MS) and hyperspectral (HS) image fusion (MS/HS fusion) task. Most of the existing CNN methods are based on “black-box” models that are not specifically designed for MS/HS fusion, which largely ignore the priors evidently possessed by the observed HS and MS images, and lack clear interpretability, leaving room for further improvement. In this paper, we propose an interpretable network, named as spatial-spectral dual-optimization model driven deep network (S2DMDN), which embeds the intrinsic generation mechanism of the MS/HS fusion to the network. There are two key characteristics: (i) Explicitly encode the spatial prior and spectral prior evidently possessed by the input MS and HS images in the network architecture; (ii) Unfold an iterative spatial-spectral dual-optimization algorithm into a model driven deep network. The benefit is that the network has good interpretability and generalization capability, and the fused image is richer in semantics and more precise in spatial. Extensive experiments are conducted to prove the superiority of our proposed method over other state-of-the-art methods in terms of quantitative evaluation metrics and qualitative visual effects.

Hyperspectral (HS) pansharpening aims to create a pansharpened image that integrates the spatial details of the panchromatic (PAN) image and the spectral content of the HS image. In this article, we present a deep convolutional network within the mature Gaussian–Laplacian pyramid for pansharpening (LPPNet). The overall structure of LPPNet is a cascade of the Laplacian pyramid dense network with a similar structure at each pyramid level. Following the general idea of multiresolution analysis (MRA), the subband residuals of the desired HS images are extracted from the PAN image and injected into the upsampled HS image to reconstruct the high-resolution HS images level by level. Applying the mature Laplace pyramid decomposition technique to the convolution neural network (CNN) can simplify the pansharpening problem into several pyramid-level learning problems so that the pansharpening problem can be solved with a shallow CNN with fewer parameters. Specifically, the Laplacian pyramid technology is used to decompose the image into different levels that can differentiate large- and small-scale details, and each level is handled by a spatial subnetwork in a divide-and-conquer way to make the network more efficient. Experimental results show that the proposed LPPNet method performs favorably against some state-of-the-art pansharpening methods in terms of objective indexes and subjective visual appearance.

The effective fusion of multi-source data helps to improve performance of land cover classification. Most existing convolutional neural network (CNN) based methods adopt an early/late fusion strategy to fuse the low-level/high-level features for classification, which still has two inherent challenges: i) the conventional convolution operation performs a weighted average operation on each pixel in the receptive field, which will reduce the discriminability of the center pixel due to the influence of the interference pixels, and ii) the spatial-spectral features of the hyperspectral image (HSI), the elevation features of light detection and ranging (LiDAR), and the complementary features between the multimodal data are not fully exploited, which results in the reduction of classification accuracy. In this paper, an effective multi-branch feature fusion network with self- and cross-guided attention (MB2FscgaNet) is proposed for joint classification of LiDAR and HSI. The main concern of this paper is how to accurately estimate more effective spectral-spatial-elevation features and yield more effective transfer in network. Specifically, MB2FscgaNet adopts a multi-branch feature fusion architecture to fully exploit the hierarchical features from LiDAR and HSI level by level. At each level of the network, a self- and cross-guided attention (SCGA) is developed to assign higher weight to interesting areas and channels of LiDAR and HSI feature maps to obtain refined spectral-spatial-elevation features and provide complementary information cross guidance between LiDAR and HS. We further designed a spectral supplement module (SeSuM) to improve the discriminative ability of the center pixel. Comparative classification results and ablation studies demonstrate that the proposed MB2FscgaNet achieves competitive performance against state-of-the-art methods.

Convolutional neural networks (CNNs) have attracted much attention in change detection (CD) for their superior feature learning ability. However, most of the existing CNN-based CD methods adopt an early- or late-fusion strategy to fuse low-level spatial details or high-level semantic information. So far, the impact of multilevel fusion strategy across multitemporal hyperspectral (HS) images, and its application to CD, remains unexplored. In this article, we propose a multilevel encoder–decoder attention network (ML-EDAN), which allows the network to make full use of the hierarchical features for CD in HS images. A two-stream encoder–decoder framework is taken as the backbone to exploit and fuse the hierarchical features from all the convolutional layers of multitemporal HS images. Within the encoder–decoder, a contextual-information-guided attention module is developed to yield more effective spatial–spectral feature transfer in the network. After fully obtaining the multilevel hierarchical features, the long short-term memory (LSTM) subnetwork is devised to analyze temporal dependence between multitemporal images. Moreover, the proposed ML-EDAN is trained in an end-to-end manner with a new joint loss function considering both reconstruction error and pixelwise classification error. The experiments are conducted on three datasets, demonstrating the effectiveness of the proposed ML-EDAN in HS CD in comparison with widely accepted state-of-the-art methods.

Hyperspectral (HS) pansharpening aims at creating a high-resolution hyperspectral (HR-HS) image by integrating a high spatial resolution panchromatic (HR-PAN) image with a low-resolution hyperspectral (LR-HS) image. It is an important preprocessing procedure in many remote sensing tasks. Most of the existing pansharpening methods train a specific convolutional neural network (CNN) model for each type of dataset with the same number of spectral bands. The main contribution of this study is to propose a new dual-branch detail extraction pansharpening network (called DBDENet) that can sharpen HS images with any number of spectral bands using a single pre-trained model by fine-tuning the parameters of a small module in the network. Specifically, DBDENet extracts spatial details from LR-HS and HR-PAN images by two bidirectional branches of the dual-branch detail extraction network level by level. For each level, the spatial details captured from the HR-PAN and those of the LR-HS images are fused by a spatial cross attention fusion module (SCAFM). The spatial details fused by the last SCAFM module are injected into the upsampled HS image to obtain an HR-HS image. Experimental results prove to show the proposed DBDENet is superior to other widely accepted state-of-the-art methods in terms of objective indicators and visual appearance.

The fusion of hyperspectral (HS) and panchromatic (PAN) images aims to generate a fused HS image that combines spectral information of the HS image with spatial information of the PAN image. In this article, we propose a multiresolution spatial–spectral feature learning (MSSL) framework for fusing HS and PAN images. The proposed MSSL transforms the existing deep and complex network into several simple and shallow subnetworks to simplify the feature learning process. MSSL upsamples the HS image while downsamples the PAN image and designs multiresolution 3-D convolutional autoencoder (CAEs) networks with a spectral constraint to learn complete spatial–spectral features of the HS image. MSSL designs multiresolution 2-D CAEs with spatial constraint to extract spatial features of the PAN image, with a low computational cost. In order to effectively generate the pansharpened HS image with high spatial and spectral fidelity, a multiresolution residual network is presented to reconstruct the HS image from the extracted spatial–spectral features. Extensive experiments are conducted on three widely used remote sensing data sets in comparison with state-of-the-art HS image fusion methods, demonstrating the superiority of the proposed MSSL method. Code is available athttps://github.com/Jiahuiqu/MSSL.

Hyperspectral image (HSI) change detection aims to identify the differences in multitemporal HSIs. Recently, a graph convolutional network (GCN) has attracted increasing attention in the field of remote sensing due to its advantages in processing irregular data. In comparison with a convolutional neural network (CNN) that can only perform convolution operations on data with the assumption of the Euclidean structure, GCN adopts a graph structure to flexibly capture the characteristics and structure information of non-Euclidean data. In this article, we propose a novel dual-branch difference amplification GCN (D2AGCN) for HSI change detection with limited samples, which allows the network to fully extract and effectively amplify the difference features of multitemporal HSIs for change detection. The dual-branch structure can effectively extract sufficient different features to facilitate the detection of the changed areas. As far as we know, this is the first time that GCN has been introduced into HSI change detection. A difference magnification module is designed to suppress similar regions and highlight the feature differences between the multitemporal HSIs in the dual-branch structure, which increases the distinction between change and nonchange classes. The visual and quantitative experimental results on three real hyperspectral datasets (i.e., China, Bay Area, and Santa Barbara) show that the proposed D2AGCN outperforms most of the state-of-the-art methods in HSI change detection with limited training samples.

Hyperspectral image pansharpening is of great importance in improving spatial resolution for many commercial platforms and remote sensing tasks. Convolutional neural network (CNN) has recently been applied in pansharpening. However, most existing CNN-based pansharpening models followed an early-fusion/late-fusion strategy, which integrates the low-level/high-level features of PAN and HS streams at the input/output of the network. It is difficult to learn more complex combinations between panchromatic (PAN) and hyperspectral (HS) streams. This paper proposes a novel end-to-end residual hyper-dense pansharpening network with a cross-guided pyramid attention (called RHDcgpaNet). The overall architecture of the proposed method is a residual hyper-dense network, which extends the definition of dense connections to two-stream pansharpening problem. The proposed RHDcgpaNet allows guidance from the state of the preceding layers to all the layers in-between PAN and HS streams in a feed-forward manner, significantly increasing the learning representation. A cross-guided pyramid attention is designed and embedded to the proposed residual hyper-dense network to yield more useful spatial-spectral feature transfer in network. Extensive experiments on widely-used datasets demonstrate that the proposed RHDcgpaNet achieves favorable performance in comparison with state-of-the-art methods.

Recently developed deep learning-based hyperspectral anomaly detection (HAD) methods typically include two steps where the deep feature extraction is not designed specifically for the HAD task. In this article, we propose a spectral distribution-aware estimation network (SDEN) that does not conduct feature extraction and anomaly detection in two separate steps but instead learns both jointly to estimate anomalies directly in an end-to-end manner without postprocessing. The unified framework can ensure that the extracted features serve better for anomaly detection. To preserve the distribution of hyperspectral images (HSIs) during dimensionality reduction, the SDEN introduces a spectral distribution (SD)-aware module imposed with a local-invariant constraint. More specifically, we adopt Markov chain Monte Carlo (MCMC) that enables the SD module to better estimate the distribution of the complex HSIs. Considering the powerful representation capability of Gaussian mixture model (GMM), the SDEN leverages it to establish an estimation module in the deep latent space where the anomaly resides in low density while the background not. We demonstrate that the SDEN yields competitive and highly promising results in comparison with the anomaly detection benchmarks.

Change detection plays an important role in Earth surface observation and has been extensively investigated over recent decades. A hyperspectral image (HSI) with high spectral resolution provides abundant ground object information, which is expected by finer change detection. The existing convolutional neural network (CNN)-based methods extract image features with a fixed kernel, which is incompetent to cope with complicated object details at diverse scales in HSI. In this article, we propose a deep multiscale pyramid network enhanced with spatial–spectral residual attention (DMP$\text {s}^{2} $raN) for HSI change detection, which has strong capability to mine multilevel and multiscale spatial–spectral features, improving the performance in complex changed regions. There are two key characteristics: 1) the multiscale spatial–spectral features are extracted by the multiscale pyramid convolution and enhanced by spatial–spectral residual attention module ($\text {S}^{2} $RAM) of each scale and 2) the multilevel features are obtained by aggregating the multiscale features level by level. As a result of this design, the proposed DMP$\text {s}^{2} $raN learns more discriminative features with both strong semantic information and rich spatial–spectral information. Experiments carried out on three datasets demonstrate the competitive performance of the proposed method in both qualitative and quantitative analyses.

Hyperspectral (HS) pansharpening is of great importance in improving the spatial resolution of HS images for remote sensing tasks. HS image comprises abundant spectral contents, whereas panchromatic (PAN) image provides spatial information. HS pansharpening constitutes the possibility for providing the pansharpened image with both high spatial and spectral resolution. This article develops a specific pansharpening framework based on a generative dual-adversarial network (called PS-GDANet). Specifically, the pansharpening problem is formulated as a dual task that can be solved by a generative adversarial network (GAN) with two discriminators. The spatial discriminator forces the intensity component of the pansharpened image to be as consistent as possible with the PAN image, and the spectral discriminator helps to preserve spectral information of the original HS image. Instead of designing a deep network, PS-GDANet extends GANs to two discriminators and provides a high-resolution pansharpened image in a fraction of iterations. The experimental results demonstrate that PS-GDANet outperforms several widely accepted state-of-the-art pansharpening methods in terms of qualitative and quantitative assessment.

Hyperspectral images (HSIs) with rich spectral information have been widely used in many fields. Anomaly detection is one of the most interesting and important applications. In this article, a novel Gaussian mixture model (GMM)-based anomaly detection (GMMD) method for HSI is proposed. The main contributions of this article are a new GMM-based extraction approach for extracting the anomaly pixels and an effective GMM-based weighting approach for fusing the extracted anomaly results. Specifically, based on the fact that the spectral values of anomaly pixels in some bands are different from those of background pixels, we propose a GMM-based anomaly extraction approach in which the HSI is characterized by the GMM and the anomaly pixels are extracted by a range prescribed by the GMM parameters. In order to fuse the extracted anomaly results, the GMM-based weighting method is introduced to adaptively construct the detection map. The detection map is rectified by using a guided filter to obtain the final anomaly detection map. Experimental results conducted on four hyperspectral data sets demonstrate the superior performance of the proposed GMMD method.

In this paper, we propose a deep residual spatial attention network (DRSAN) for hyperspectral (HS) pansharpening. Different from the existing methods, our newly proposed method not only considers the spatial information of both the panchromatic (PAN) and the HS image simultaneously, but also adaptively learns more informative features of spatial locations for details enhancement, which mainly includes four steps. Firstly, the spatial details of the enhanced PAN image are obtained through the structure tensor. Then we extract the spatial information of the upsampled HSI by using the guided filter. The integrated spatial information of both PAN and HS images is subsequently fed into the DRSAN to map the residual HSI between the upsampled HSI and the reference HSI, where several residual spatial attention blocks (RSABs) are cascaded to exploit more useful details information. Finally, the fused HSI is generated by the summation of the upsampled HSI and the reconstructed residual HSI. Extensive visual and quantitative assessments validate the superiority of our proposed DRSAN over the state-of-the-art HS pansharpening methods.

In this paper, a new hyperspectral image (HSI) and panchromatic image (PANI) fusion approach via weighted tensor matrix is proposed. In the proposed method, homomorphic filtering is use for obtaining spatial component of HSI, and a weighted root mean squared error (RMSE)-based algorithm is proposed to extract the total intensity details of HSI. In addition, an optimized weighted tensor matrix-based method is proposed to acquire the integrated intensity details from both HSI and PANI. Comparative analyses show the proposed approach performs better than other excellent approaches in visual inspection and objective assessment.

In this paper, a novel hyperspectral (HS) image fusion method using matting model is presented. Matting model refers to each band of an HS image that can be decomposed into three components, i.e., alpha channel, spectral foreground, and spectral background. First, panchromatic (PAN) image is sharpened to enhance details, and the spatial information of each band of HS image is obtained by weighted least squares filtering. Different from traditional matting model based methods that PAN image is served as the alpha channel, we do the PCA transformation to PAN image and spatial information of each band to obtain the first principal component channel which is selected for the alpha channel. This processing reduces spatial distortion. Finally, HS foreground and HS background are estimated by the alpha channel, and the fused HS image is reconstructed nearly perfectly. Experiments reveal that the proposed method is superior to the state-of-the-art methods.

The component substitution (CS) and multiresolution analysis (MRA) based methods have been well adopted in hyperspectral pansharpening. The major contribution of this paper is a novel MRA and CS hybrid framework based on the intrinsic image decomposition. First, the weighted least squares (WLS) filter is performed on the sharpened panchromatic (P) image to extract the high-frequency component. Then, the intrinsic image decomposition (IID) is adopted to decompose the interpolated hyperspectral (H) image into the illumination and reflectance components. Finally, the detail map is generated by making a proper compromise between the high-frequency component of the P image and the illumination component of the H image. The detail map further refined by the information ratio of different bands of the H image is injected into each band of the interpolated H image. Experimental results indicate that the proposed method achieves a better fusion result than several state-of-the-art hyperspectral pansharpening methods.

In this paper, a new hyperspectral image fusion method with structure tensor is proposed. The proposed method utilizes PCA transformation to obtain the spatial details of HS image. Then, an image enhancement approach is applied to the PAN image to sharpen spatial information. Since structure tensor represents structure and spatial information, structure tensor is introduced to extract spatial details of the enhanced PAN image. Unlike traditional methods which extract details only from PAN image, the proposed method considers spatial details of the HS and PAN images simultaneously, and a weighted fusion method is presented to integrate spatial details of the two images to obtain complete spatial details. Finally, an injection gains matrix is constructed to reduce spectral and spatial distortion, and the fused image is generated by injecting the complete spatial information. Experimental results demonstrate that the proposed method obtains the excellent performance in both objective and subjective evaluations.

Because the guided filter can transfer structures, and avoid ringing artifacts, it can be applied to image fusion. A new hyperspectral pansharpening method based on guided filter (GFP) is proposed in this paper. The proposed method which works on each band of the hyperspectral (HS) image successively is different from the traditional methods. The detail information of each band is extracted at first. Then, we sharpen the panchromatic (PAN) image to enhance the details and obtain the difference between the enhanced PAN image and the detail information of each band in turn using the guided filter without causing spectral and spatial distortion. In order to reduce spectral distortion and add enough spatial information, the injection gains matrix is generated. The fused HS image is finally achieved by injecting the corresponding spatial difference into each band of the interpolated HS image. Experiments demonstrate that the proposed method can obtain superior performance in terms of subjective and objective evaluations.

A new hyperspectral (HS) pansharpening method based on guided filter is proposed in this letter. The proposed method, which obtains the spatial detail difference of each band successively, is different from the traditional component substitution method. The detail information of each band is extracted at first. Then, the panchromatic (PAN) image is sharpened to enhance the details. The spatial information difference between the enhanced PAN image and the detail information of each band is obtained using the guided filter, without causing spectral and spatial distortion. In order to reduce spectral distortion and add enough spatial information, the injection gains matrix is generated. The fused HS image is finally achieved by injecting the corresponding spatial difference into each band of the interpolated HS image. Experiments demonstrate that the proposed method can obtain superior performance in terms of subjective and objective evaluations.