Classifying hyperspectral remote sensing images across different scenes has recently emerged as a significant challenge. When only historical labeled images (source domain, SD) are available, it is crucial to leverage these images effectively to train a model with strong generalization ability that can be directly applied to classify unseen samples (target domain, TD). To address these challenges, this paper proposes a novel single-domain generalization (SDG) network, termed the domain-aware adversarial domain augmentation network (DADAnet) for cross-scene hyperspectral image classification (HSIC). DADAnet involves two stages: adversarial domain augmentation (ADA) and task-specific training. ADA employs a progressive adversarial generation strategy to construct an augmented domain (AD). To enhance variability in both spatial and spectral dimensions, a domain-aware spatial-spectral mask (DSSM) encoder is constructed to increase the diversity of the generated adversarial samples. Furthermore, a two-level contrastive loss (TCC) is designed and incorporated into the ADA to ensure both the diversity and effectiveness of AD samples. Finally, DADAnet performs supervised learning jointly on the SD and AD during the task-specific training stage. Experimental results on two public hyperspectral image datasets and a new Hangzhouwan (HZW) dataset demonstrate that the proposed DADAnet outperforms existing domain adaptation (DA) and domain generalization (DG) methods, achieving overall accuracies of 80.69%, 63.75%, and 87.61% on three datasets, respectively.

Change detection (CD) in hyperspectral images (HSIs) has become an increasingly vital research field in remote sensing. Over the past few years, the adoption of deep learning approaches, particularly convolutional neural network (CNN) and transformer-based architectures have significantly advanced performance in this field. While these models effectively capture spectral-spatial features, they may also introduce redundant or irrelevant spatial information, potentially degrading the accuracy of HSI CD. To address this challenge, a center-pixel and gated mechanism-based attention network (CGMNet) is proposed for HSI CD, leveraging the central pixel's significance to enhance accuracy and robustness. First, a gated-based center spatial attention (GCSA) module is designed to emphasize spatial relationships surrounding the central pixel. By incorporating gating mechanisms, GCSA selectively enhances relevant spatial features while suppressing irrelevant information. Second, a gated-based spectral attention (GSA) module is proposed to dynamically highlight the most significant spectral features, ensuring an effective spectral representation. Finally, a global transform fusion (GTF) module is proposed to capture global contextual information and to fuse it with the extracted spatial and spectral features. Moreover, we introduce a novel benchmark dataset, named the Hangzhou Bay (HZB), specifically designed to advance coastal remote sensing research. Experimental evaluations conducted on three publicly available datasets, as well as the HZB dataset, show that our CGMNet consistently outperforms some state-of-the-art methods in the HSI CD task. The source code of the proposed CGMNet, along with the HZB dataset, will be made publicly available at https://github.com/creativeXin/CGMNet.

Due to the complex acquisition environment and scarcity of labels, domain adaptation (DA) techniques are widely applied to cross-scenario hyperspectral image (HSI) classification to achieve more precise labeling. Many existing approaches mainly rely on convolutional neural networks (CNNs) to capture local spatial contextual relationships, supplemented by graph convolutional networks (GCNs) for long-range modeling. However, GCNs usually require full batch training and fixed initial graph structures, which significantly limits the exploration of topological structures. To address this, a multi-contrastive and dynamic topological matching network (MCDTM) is introduced to accomplish cross-domain HSI classification. Unlike fixed graph construction methods, mini-batches of samples are utilized to construct dynamic subgraphs within the source and target domains, respectively, with locally extracted features from CNNs serving as the basis for graph construction. More importantly, as the model is optimized and the domain gap narrows, the dynamic graph structure is adaptively adjusted according to the evolving samples, thereby boosting the accuracy of the graph and enhancing the discriminative power of the model. Moreover, the integration of weighted multi-positive contrastive learning and graph matching achieves distribution alignment and graph alignment, enhancing the model's capacity to distinguish and align complex patterns in HSI. Experimental results in three tasks show that the MCDTM surpasses several advanced DA methods, achieving impressive accuracies of 80.10%, 70.01%, and 94.17% on the Houston, HyRank, and YC-YC tasks, thereby showcasing its superior performance.

Satellite-based hyperspectral (HS) imagery holds great potential in remote sensing applications due to its fine spectral resolution. However, the low spatial resolution limits its practical utility. Combining ancillary high resolution panchromatic (PAN) or multispectral (MS) images has become a common practice to improve the spatial quality of HS images. Most present approaches, however, are based on dual-sensor fusion (e.g., MS-HS or PAN-HS), which generally falls short of comprehensively integrating their complementary spatial and spectral information of PAN, MS, and HS images. Meanwhile, existing few integrated fusion methods suffer from two key limitations: modality mismatch due to inconsistent spatial-spectral characteristics among PAN, MS, and HS data, and shallow and redundant cross-modal coupling caused by inadequate modeling of inter-modal relationships. In this paper, we propose a progressive spatial-spectral interactive network (PSSNet) for the integrated fusion of panchromatic, multispectral, and hyperspectral images. Specifically, a context-aware fusion block is introduced to extract and enhance contextual spatial and spectral information across different modalities. To ensure an effective integration of spatial and spectral details, the entire network is structured progressively, allowing for a smooth transition and fusion of features from HS, MS, and PAN images. Additionally, a spatial-spectral feature recombination module is designed to dynamically adjust the contribution of spectral features at various levels. This module, in combination with a spatial enhancement component, facilitates the optimal fusion of spatial and spectral information by enhancing their interactions. Extensive experiments on simulated and real datasets, both qualitatively and quantitatively, demonstrate the superiority of PSSNet compared to other state-of-the-art methods.

Recently, deep neural networks have demonstrated outstanding performance in hyperspectral image (HSI) change detection (CD), especially when there is sufficient labeled samples. However, the labels of HSI are difficult to obtain, and acquiring enough labels to train deep network is a great challenge in practice. Therefore, to mitigate the effect of insufficient labels in detection results, this paper proposes a cross-scene few-shot learning (FSL) network based on intra-inter domain contrastive alignment (CAFSL) for HSI-CD, which combines contrastive learning (CL) and FSL into a unified framework, aiming to achieve better detection results using only a few labeled samples. Specifically, we perform cross-scene FSL using a pair of dual-phase images from a very high-resolution image (VHRI) as the source domain and a pair of HSI as the target domain. Then, an intra-domain supervised contrastive learning (INSCL) module is designed to enhance the compactness within classes and widen the discrimination between classes by maximizing the feature similarity of intra-class and minimizing the feature similarity of inter-class. Finally, a cross-domain contrastive alignment (CRCA) module is proposed to align the features of source and target domains, which mitigates the effect of domain migration problems caused by different data types. Experiments on three HSI benchmark datasets reveal that the CAFSL algorithm outperforms current advanced algorithms based on deep learning and FSL while with limited labeled samples.

Achieving land cover change detection (LCCD) through remotely sensed images (RSIs) is important in the observation of the changes on the Earth’s surface. In such detection, spectral-reflectance noise and the uncertainty of the imaging external conditions for the bitemporal RSIs usually cause some salt-and-pepper noisy pixels in the results and reduce the change detection accuracy. In this article, a hierarchical feature-fusion triple network (HFTN) is proposed to improve the performance of LCCD with RSIs. Overall, the proposed HFTN aims to learn representative features to improve change detection performance via two feature learning enhancement strategies and a hierarchical feature-fusion mechanism. First, an image feature difference model is proposed to generate the input feature for the middle branch and guide the learning performance. Second, a progressive denoising module (PDM) is proposed and applied to each temporal image to reduce the noise before feeding the features into the backbone of the proposed HFTN. Finally, a hierarchical feature-fusion module (HFFM) is proposed to fuse the learned deep feature for generating a change-magnitude image. Additionally, multiscale convolution, cross-scale fusion, and a shared weight are adopted in the backbone of the proposed HFTN to further enhance the feature learning performance. Compared with eight state-of-the-art methods, experimental results verified the feasibility and superiority of the proposed HFTN for LCCD with RSIs. For example, the proposed HFTN achieved improvement rates of approximately 0.43%–11.83% for overall accuracy (OA) and 0.11%–4.81% for false alarms (FAs) across six pairs of real RSIs. The code can be available athttps://github.com/ImgSciGroup/HFTN-NET.git.

Sample augmentation is crucial for improving land cover classification performance when the samples are limited. However, the traditional sample augmentation approach concentrates on enlarging the quantity of sample via generation and synthetic technique directly, the sample quality is usually neglected. In this article, we propose a novel sample augmentation approach with threshold estimation (SATE) to improve both the quantity and quality of samples for hyperspectral remotely sensed image (HRSI) classification. Firstly, a threshold estimation algorithm (TEA) is proposed to identify high-confidence potential samples from the initial classification map by utilizing the prediction probabilities of different classes. Second, a semi-variational model is employed to detect and correct pseudo-labels in the spatial domain, further enhancing the quality of selected potential samples. Finally, a farthest point sampling (FPS) algorithm optimizes sample distribution in the spectral domain, improving representation for intra-class heterogeneity. Experimental results based on four real HRSIs and compared with eight state-of-the-art few-shot-based methods verify the feasibility and superiority of the proposed SATE approach. The improvement achieved by our proposed approach is about 0.79% ~ 4.31% in terms of the overall accuracy. Code is available at https://github.com/ImgSciGroup/SATE.

Achieving satisfactory land cover classification performance with high-resolution remote sensing images (HRSIs) usually requires sufficient samples for a supervised classifier. However, labeling sufficient samples is labor-intensive and time-consuming. In this article, a Novel Sample Augmentation Approach (NSAA) is proposed to synthesize new samples and improve classification accuracies for HRSI when initial known samples are very limited. First, a very small sample set of each class is prepared manually for the algorithm’s initialization. Second, a sample generator based on normal cloud model is proposed, and an adaptive region growing algorithm is suggested to explore some potential samples around a known sample for parameter estimation of the sample generator. Third, to further refine the generated samples around an initial known sample, a near-to-far space constraint strategy is proposed based on the K-means clustering algorithm to improve the quality of the generated samples. The proposed sample augmentation approach is incorporated with a classifier iteratively, and a sample balancing strategy is suggested in the iterative progress. Experiment results based on six real HRSIs and compared with eight state-of-the-art methods demonstrate the feasibility and superiorities of the proposed sample augmentation approach. Moreover, the reliability and robustness of the generated samples are verified by popular deep-learning networks and typical traditional classifiers. The improvement achieved by our proposed approach is about 0.12% – 0.95% in terms of the overall accuracy.

Spatial-temporal-spectral fusion is dedicated to integrating the complementary advantages of multisource images to obtain fused image with all high spatial, high temporal and high spectral resolutions, which is promising but more challenging. On the one hand, traditional studies deployed on MODIS and Landsat data cannot be transferred to most spaceborne hyperspectral (HS) data with lower temporal resolution; on the other hand, the rigid time relation modeling in most existing studies exhibits weakness orienting to non-linear land-cover changes. In this paper, we propose a dual-task cascaded network for spatial-temporal-spectral fusion, with collaborative modeling on spatialspectral joint enhancement and temporal variation estimation in a unified framework. The spatial-spectral joint enhancement task was designed with an iterative alternating projection, meticulously crafted to address the scale variance among observation. Additionally, the spatial enhancement unit and error correction unit were coupled modeling to enhance the spatial and spectral fidelity. The temporal variation estimation on spectral fine tuning network was developed, to further enhance the temporal and spectral fidelity. Extensive experiments were implemented on Ziyuan(ZY)-1 02D HS data and Sentinel-2 multispectral (MS) data. Both qualitative and quantitative results demonstrated the competitive performance of the proposed method.

The integrated fusion of the high-spatial-resolution (HR) panchromatic image (PAN), the relative “moderate”-spatial-resolution (MR) multispectral image (MSI), and the low-spatial-resolution (LR) hyperspectral image (HSI), to generate the optimal HR HS fused image, is promising but challenging. On the one hand, existing mainstream fusion models mostly focus on the “pairwise fusion” between HR PAN, MR MSI, and LR HSI, which cannot sufficiently integrate their complementary spatial and spectral advantages. On the other hand, one of the few integrated fusion methods roughly introduced the MR MSI as a simple intermediate medium; however, the role of MSIs as a spatial and spectral “bridge” between HR PAN and LR HSI, generally characterized by significant scale differences, remains largely unexplored. To solve these problems, we proposed an integrated PAN–MSI–HSI fusion method from the perspective of MSIs, by comprehensively considering the scale difference among the multisource observations. In the proposed method, a spatial–spectral feature transfer network was designed by comprehensively exploring the spatial–spectral variations and connections among the HR PAN, MR MSI, and LR HSI. Then, a spatial–spectral joint reconstruction module was constructed to reconstruct the HR HSI with optimal spatial and spectral fidelity. Experiments were conducted on simulated and real datasets from qualitative and quantitative aspects. The experimental results demonstrated the competitive effectiveness over other state-of-the-art methods.

Recently, domain adaptation (DA) methods based on contrastive learning are widely used to solve the cross-scene classification problem. However, existing contrastive learning methods only focus on source domain or target domain features, or do not adequately consider the interaction of domain information, thus the learned domain-invariant features still have large discrepancies. To address this problem, we propose a novel domain fusion contrastive learning (DFCL) framework for cross-scene hyperspectral image (HSI) classification. DFCL uses an interdomain and intradomain dual-domain fusion strategy at the feature level, which introduces domain information as a noise interference term for sample enhancement. With the interference of domain information, same category samples are pulled closer and different categories samples are pushed further apart to learn more discriminative features. In addition, we construct an intermediate domain through the source and target domains and define a feature space loss that measures domain discrepancy by feature similarity and label similarity. Finally, a progressive selection strategy based on prototype learning is proposed to select high-confidence pseudolabels for DFCL. Experiments on three HSI cross-scene datasets show that the proposed method is superior to existing DA methods.

When hyperspectral image classification encounters high-dimensional spectral channels, the feature utilization rate is often low due to significant redundancy between channels and uneven discriminatory power. Furthermore, different land cover types exhibit notable differences in scale and morphological changes across space. This structural diversity poses significant challenges to spatial feature modeling. To address this problem, this paper proposes a novel framework based on the Mamba model-PS2Mamba for hyperspectral image classification. This framework integrates three strategies: spectral fine modeling, global perceptual scale adaptation, and multi-scale spatial structure modeling. Firstly, this paper designs a statistically enhanced normalized band refinement (SENBR) module, which dynamically enhances or suppresses channel features based on channel correlation, variability, and importance, effectively suppressing redundant and noisy bands. Secondly, a global-aware scale adaptation (GASA) module is proposed. By incorporating a scale scoring network and a multi-directional modeling mechanism, this module enables adaptive perception and directional enhancement modeling of spatial structures at different scales. Finally, a lightweight multi-scale spatial structure extraction module, LightPyramid, is constructed. By enriching spatial semantic information through multi-branch parallel convolutions, it enhances the model’s ability to represent complex surface structures, while maintaining spatial resolution. Experimental results on four representative hyperspectral datasets, including Pavia University, Salinas, and two UAV-based datasets (HongHu and HanChuan), demonstrate that PS2Mamba achieves overall accuracies of 98.77%, 99.61%, 96.20%, and 96.01%, respectively. Compared with existing CNN, GCN, Transformer, and Mamba based models, PS2Mamba achieves up to 12.91% improvement in accuracy, showing superior generalization and robustness, particularly under small-sample conditions.

Recently, Mamba has made significant strides in sequence modeling, with its global receptive field, dynamic weighting strategy and linear growth in computational complexity. In remote sensing (RS) change detection (CD), several studies have demonstrated that Mambas leverage a unique scanning mechanism to traverse images from various directions, showcasing excellent long-range modeling capabilities. However, as the network depth increases, Mamba often struggle to retain shallow textures and local features effectively. In particular, modern RS images frequently capture complex surface scenes, including seasonal climate variations and densely built environments, making local contextual details crucial for effective CD. Therefore, a difference enhancement and inter-scale interactive fusion Mamba (DEIF-Mamba) is proposed to alleviate the issue. This entire network framework integrates CNN and Mamba, utilizing CNN to capture local feature information, while Mamba employs a cross-scanning mechanism to integrate global information. To address the interference caused by mixed texture features and the missed detection of subtle changes in complex scenes, a differential feature enhancement module (DFEM) is proposed to enrich local contextual details and improve feature representation. In addition, we propose an inter-scale interactive fusion (ISIF) strategy to fully utilize the cross-scale interactive information and minimize information redundancy. Extensive experiments on four CD datasets demonstrate that the proposed DEIF-Mamba achieves an average F1 of 85.87%, and shows superior performance compared with other state-of-the-art (SOTA) methods. Code will be available online (https://github.com/Jyl199904/DEIF-Mamba).

Hyperspectral image (HSI) change detection (CD) plays a crucial role in remote sensing observation. It leverages the abundant spectral and spatial information in bi-temporal HSIs to identify subtle Earth surface changes. Most current deep-learning-based HSI CD methods primarily utilize convolutional neural networks or transformers to extract features from bi-temporal images. However, these methods lack an effective attention mechanism to enhance differential features. In addition, they do not fully leverage the aggregation relationship between the features of bi-temporal images to extract interaction features. To address these challenges, we propose a novel adaptive information weighting and synchronized enhancement network (AIWSEN) for HSI CD. This network employs the information entropy to capture change features specific to the CD task and enhances bi-temporal interaction features. Specifically, an adaptive information weighting attention module (AIWAM) leverages the maximum discrete entropy theorem to capture the difference information. A dual-time synchronic change enhancing module (DSCEM) is designed to extract features by interactively aggregating features from bi-temporal HSIs to enhance difference features. A bi-temporal image feature selection and fusion module (BFSFM) is constructed to filter out important features using forget and update gates. Experimental results on three HSI CD datasets demonstrate that the proposed AIWSEN method outperforms several state-of-the-art methods. The source code of the proposed AIWSEN will be released athttps://github.com/creativeXin/AIWSEN.

In recent years, graph neural networks (GNNs) have been increasingly applied to hyperspectral image change detection (HSI-CD). However, existing methods propagate features by weighting nodes and edges, which may weaken the original node characteristics and spectral differences. Moreover, current methods typically treat bitemporal features in isolation and lack inter-temporal semantic modeling. To address the above issues, this paper proposes a graph contrastive learning and multi-graph attention fusion network (GCLMA), which leverages contrastive learning to extract discriminative features and enhances semantic relevance through graph attention-based interactions across multiple graphs. The proposed GCLMA method mainly consists of four modules, i.e., spatial-spectral graph construction (SSGC), contrastive feature learning ResGCN (CFLR), differential feature enhancement (DFE), and multi-graph interactive fusion (MGIF). The SSGC module is first designed to construct a well-defined graph structure by adaptively integrating spatial proximity and spectral similarity. Then, the CFLR module utilizes contrastive learning to extract discriminative feature representations that are sensitive to subtle changes. Next, the DFE module amplifies significant spectral variations to emphasize critical change regions. Finally, the MGIF module employs the graph attention mechanism to interactively fuse enhanced difference graph features with bitemporal graph features, thereby facilitating semantic modeling. Experimental results on three HSI-CD datasets show that the proposed GCLMA outperforms most existing state-of-the-art methods. The source code of the proposed method will be released at https://github.com/YYYYYJJJJJJ/GCLMA.

Hypergraph neural networks (HGNNs) have garnered considerable attention in hyperspectral image (HSI) classification for their ability to model higher-order nonlinear relationships. However, most existing HGNN-based HSI classification methods adopt simple hypergraph structures, which fail to fully leverage low-order and high-order information within HSIs. Moreover, these methods are constrained by rigid single-scale superpixel segmentations, which fail to fully capture rich spectral-spatial features of HSIs. To overcome these limitations, this paper proposes a progressive hybrid-order hypergraph framework (PHHF) that establishes a graph-hypergraph collaborative learning paradigm to learn multi-scale hybrid-order features across hierarchical graphs. Specifically, by synergistically modeling multiple structural information, we develop a progressive hypergraph neural network framework to enhance the extraction of hierarchical representations of HSIs. Second, at each level, a novel hybrid-order hyperedge generation strategy is designed to enhance spectral-spatial consistency across scales and overcome the limitations of existing hypergraph methods. Finally, a novel heterogeneous kernel-based convolution (HetConv) is introduced to enhance pixel-level feature extraction for the PHHF. Compared to conventional convolution, HetConv offers richer spatial-spectral details with lower computational cost. Extensive experiments on widely-used HSI datasets demonstrate that our proposed method achieves state-of-the-art HSI classification performance.

In recent years, few-shot learning (FSL) has made significant progress in hyperspectral image classification (HSIC) by transferring meta-knowledge from a source domain with sufficient labeled samples to a target domain with limited labeled samples. However, existing FSL methods face two key challenges in coastal wetland HSIC applications, i.e., prototype instability due to limited labeled samples and domain shift due to the domain’s distribution difference. These challenges are further exacerbated by the unique characteristics of coastal wetland environments, which have complex land cover classes and subtle spectral differences between land cover classes. To address these limitations, we propose a language-guided and similarity-aware network (LGSAnet) for few-shot coastal wetland HSIC in this article. The network mainly consists of two modules: language-guided prototype alignment (LGPA) and similarity-aware prototype calibration (SAPC). The LGPA module uses linguistic features to guide the learning of visual features, enabling the model to obtain visual representations enriched with linguistic prior knowledge. This guided alignment can learn more accurate and stable visual prototype representations, thereby improving the accuracy of the model. To mitigate domain shift, an SAPC module is constructed to refine the target domain prototypes and minimize domain-specific differences using the domain’s semantic similarity. The experimental results on three coastal wetland hyperspectral datasets demonstrate that the proposed LGSAnet outperforms existing state-of-the-art FSL methods.

Deep learning has advanced hyperspectral image classification (HSIC), but label scarcity remains a significant challenge. Traditional unimodal methods usually produce unstable prototypes, while multimodal methods suffer from cross-modal semantic misalignment. Moreover, standard metrics fail to capture high-order feature correlations, further limiting the discriminative ability of the model. To address these issues, we propose a query-oriented dynamic multimodal alignment (QODMA) method, which integrates visual-textual guidance with dual-distance metric learning for few-shot HSIC. Specifically, a query-oriented dynamic attention (QODA) module is designed to bridge the modality gap by aligning visual and textual features through query-driven attention interactions. A bidirectional attention mechanism is constructed to employ contrastive learning to enhance intra-class compactness. Additionally, a dual-distance metric learning (DML) module that combines the Euclidean distance and Brownian distance covariance (BDC) metrics is employed to refine the feature space representation, thereby enhancing the discriminative ability of the model. To mitigate domain shift, an inter-domain structural consistency loss (IDSCL) is constructed. Experimental results on four public hyperspectral data sets demonstrate that the proposed QODMA outperforms state-of-the-art few-shot classification methods.

In recent years, marine oil spills have occurred frequently, causing serious damage to the marine ecological environment. Hyperspectral images (HSIs) can provide rich spectral and spatial information, and have broad development prospects in marine oil spill detection. This article proposes a hyperspectral marine oil spill detection network, HMOSDN, that integrates improved superpixel segmentation and a mixed attention mechanism (MAM). First, to deal with the extensive clutter and diverse morphology of marine oil spill areas in HSIs, we propose a new superpixel segmentation algorithm based on improved simple linear iterative clustering (ISLIC), which achieves preliminary extraction of spatial features and reduces spatial noise via a Gaussian filter and a pixel intensity smoothing technique (PIST). Then, to further fuse spectral and spatial features and strengthen the feature mining and utilization of fused information, we design a spectral-spatial feature extraction network with an MAM, MAM-SSFEN, which adds a spectral attention module and a spatial attention module, further improving the performance of the deep feature extraction network for oil spill detection. Experiments on the hyperspectral oil spill database (HOSD) demonstrate that our proposed method, HMOSDN, outperforms several other detection techniques regarding area under the curve (AUC) and recall evaluation metrics.

Spectral super-resolution (SSR) is the computational process of generating a high-dimensional hyperspectral image from a low-dimensional image through spectral reconstruction techniques. Recently, deep learning has demonstrated remarkable potential in the field of SSR, achieving impressive results. However, existing deep learning-based approaches often fail to deliver high-fidelity SSR outcomes. These methods tend to focus primarily on spectral information while paying insufficient attention to the critical role of spatial features. Furthermore, they lack effective strategies for capturing inter-band relationships, resulting in suboptimal spectral information modeling. To address these limitations, we propose a novel network for SSR, termed Bidirectional Spectral Attention Multi-Scale Aggregation Network (BiSANet). BiSANet features three U-Net-like branches and integrates two advanced attention mechanisms. The bidirectional spectral attention modules dynamically model inter-spectral dependencies through forward and reverse spectral feature extraction, enhanced by a weight-sharing strategy. Specifically, we reverse the spectral order of feature maps to activate complementary global trends and local details, overcoming the limitations of unidirectional modeling in traditional methods. Additionally, an independent spatial reconstruction branch with a dedicated loss function ensures precise spatial detail preservation. Experimental results demonstrate that BiSANet outperforms state-of-the-art methods across three benchmarks. For instance, on the DFC2018 Houston dataset, it achieves a 4.26% PSNR improvement and an 11.52% SAM reduction, highlighting its robustness and accuracy in spectral-spatial reconstruction.

High-resolution satellite imagery with dense temporal series is crucial for long-term surface change monitoring. Spatiotemporal fusion seeks to reconstruct remote sensing image sequences with both high spatial and temporal resolutions by leveraging prior information from multiple satellite platforms. However, significant radiometric discrepancies and large spatial resolution variations between images acquired from different satellite sensors, coupled with the limited availability of prior data, present major challenges to accurately reconstructing missing data using existing methods. To address these challenges, this paper introduces GCM-PDA, a novel generative compensation model with progressive difference attenuation for spatiotemporal fusion of remote sensing images. The proposed model integrates multi-scale image decomposition within a progressive fusion framework, enabling the efficient extraction and integration of information across scales. Additionally, GCM-PDA employs domain adaptation techniques to mitigate radiometric inconsistencies between heterogeneous images. Notably, this study pioneers the use of style transformation in spatiotemporal fusion to achieve spatial-spectral compensation, effectively overcoming the constraints of limited prior image information. Experimental results demonstrate that GCM-PDA not only achieves competitive fusion performance but also exhibits strong robustness across diverse conditions.

Semantic change detection (CD) not only helps pinpoint the locations where changes occur, but also identifies the specific types of changes in land cover and land use. Currently, the mainstream approach for semantic CD (SCD) decomposes the task into semantic segmentation (SS) and CD tasks. Although these methods have achieved good results, they do not consider the incentive effect of task correlation on the entire model. Given this issue, this article further elucidates the SCD task through the lens of multitask learning theory and proposes a semantic change detection network based on boundary detection and task interaction (BT-SCD). In BT-SCD, the boundary detection (BD) task is introduced to enhance the correlation between the SS task and the CD task in SCD, thereby promoting positive reinforcement between SS and CD tasks. Furthermore, to enhance the communication of information between the SS and CD tasks, the pixel-level interaction strategy and the logit-level interaction strategy are proposed. Finally, to fully capture the temporal change information of the bitemporal features and eliminate their temporal dependency, a bidirectional change feature extraction module is proposed. Extensive experimental results on three commonly used datasets and a nonagriculturalization dataset (NAFZ) show that our BT-SCD achieves state-of-the-art performance. The code is available at https://github.com/TangYJ1229/BT-SCD.

Fine mapping of retrogressive thaw slumps (RTSs) holds paramount significance in the study of permafrost degradation and carbon exchange. We propose a lightweight and enhanced semantic segmentation network (LessNet) for automatically mapping the RTSs from Sentinel-2 images. LessNet is constructed on the encoder-decoder framework with innovative incorporation of attention mechanism and dual-level semantic features fusion. The lightweight architecture of LessNet eliminates the need for pre-training, and the network hyperparameters are automatically updated based on the training dataset, which allows for fast convergence of supervised learning. Experiments conducted in the Beiluhe region of the Tibetan Plateau highlight the robustness and competitive performance of the model.

Supervised classification with hyperspectral remote-sensing images (HRSIs) plays an important role in practical applications. However, labeling samples with HRSIs for supervised classification is time-consuming and labor-intensive. In this letter, we propose a new sample enhancement approach to improve the classification performance of HRSIs. First, the uncertainty and representativeness of the sample are defined to achieve sample possibility measurement for each pixel, and some pixels with high possibility can be selected as candidate samples. Then, two rules related to label correlation analysis and spectral similarity are defined to further refine the candidate samples used for generating the final sample set. Finally, the above-mentioned steps are fused into an iterative algorithm to enhance and balance the training samples for each class. The feasibility of the proposed approach was verified by applying it to classification with two real HRSIs. A comparison with some typical traditional sample enhancement methods and widely used few-shot deep-learning methods indicated the advantages of the proposed approach for improving classification accuracies. The improvement achieved by our proposed approach is about 0.79% ~ 2.31% in terms of the overall accuracy (OA). The code of the proposed approach is available athttps://github.com/ImgSciGroup/2023-GRSL-SIEA.

The escalating surface ozone (O3) pollution in urban areas throughout China has raised significant concerns due to its detrimental impacts on public health, local environment and agriculture. Despite of numerous efforts in surface O3estimation, intricate geographical spatiotemporal interactions of the potential predictors has been largely overlooked. This limitation has significantly constrained the O3estimation accuracy. To address this issue, we proposed a novel deep neural Network, named Geo-STO3Net, to effectively integrate adjacent Geographical SpatioTemporal information from meteorological data and satellite observations into surface O3estimation. The Geo-STO3Net model used a spatial encoder based on the Residual Network, a temporal encoder based on the Transformer, and a feature decoder based on the Deep Neural Networks to comprehensively capture the intricate geographical spatiotemporal dependencies among the predictors. Our model achieved cross-validation (CV) R2value of 0.95, outperforming popular models. The Geo-STO3Net model demonstrated robust spatial and temporal transferability, as evidenced by R2values of 0.94 and 0.82 in external spatial and temporal validation on monthly scales, respectively. The Geo-STO3Net model’s proficiency in handling geographical spatiotemporal information led to substantial performance improvements compared to models lacking this feature, with improved CV R2values ranging from 0.01 to 0.18. Our findings also highlighted the severe O3pollution over the Yangtze River Delta (YRD) region in 2022, with average surface O3concentrations reaching 103.14 μg/m3. These evidences indicate our proposed Geo-STO3Net model can accurately estimate surface O3concentrations, and provide valuable insights into the development of effective control policies.

Cloud cover and long revisit cycle of satellites can cause gaps in optical images and pose a significant obstacle to the consistency of Earth observation missions. Recently, synthetic aperture radar (SAR)-to-optical image translation (S2OIT) has become an emerging approach to reconstruct the missing information of optical remote sensing images. However, the previous studies ignored the mechanism difference between SAR and optical data and produced color distortion, image blurriness, and texture detail loss in the generated optical images. To tackle these challenges, we propose a multitemporal S2OIT network (MTS2ONet) for high-quality optical image generation. The proposed model comprises two subnetworks: change feature extraction subnetwork (Change_Extractor) and the S2OIT subnetwork (S2O_Translator). The first subnetwork is tasked with extracting change features from SAR images captured at dates T and${T} +1$, and then translating them from the SAR domain to the optical domain. Subsequently, the S2O_Translator integrates the optical image at date${T} +1$with the change features extracted by the Change_Extractor to generate the optical image at date T. In addition, we produce a dual-temporal SAR-optical dataset called DTSEN1-2 for model evaluation. Experiments on the DTSEN1-2 dataset reveal that our method is superior to the state-of-the-art (SOTA) methods with the metrics peak-signal-to-noise ratio (PSNR; 36.0435), structural similarity index measure (SSIM; 0.9896), learned perceptual image patch similarity (LPIPS; 0.0443), and root mean square error (RMSE; 0.0174) and exhibits preferable results in visual effects. Our dataset and codes can be accessed via the following link:https://github.com/hopeupup/MTS2ONet.

Recently, the adversarial domain adaptation (ADA) methods have been widely investigated and applied in cross-domain hyperspectral image (HSI) classification. However, most ADA algorithms aim to align the cross-domain distribution without focusing on the class separability of the aligned target features and the information of samples within the domain. To address these issues, a new ADA framework based on calibrated prototype and dynamic instance convolution (CPDIC) is proposed in this paper for cross domain HSI classification. The CPDIC is composed of a generator, a calibrated discriminator and a classifier. The generator includes a static 3D convolutional network (SCN) and a dynamic instance convolutional network (DICN), where the SCN is used to extract coarse-grained features of HSI and the DICN can extract sample-specific fine-grained features using instance convolutions generated from dynamic instance convolution kernel generation (DCKG) module. As for the generator, the static and dynamic interactive feature extraction network extracts robust domain-invariant features with discriminability. The calibrated discriminator aligns the marginal distribution between domains and calibrate the predicted pseudo labels of target domain. For classification, a calibrated prototype loss (CPL) is introduced to align the class distribution across domains. The results of three cross-domain HSI classification tasks show that the proposed CPDIC outperforms existing unsupervised domain adaptation (UDA) algorithms.

The objective of change detection (CD) is to identify the altered region between dual-temporal images. In pursuit of more precise change maps, numerous state-of-the-art (SOTA) methods design neural networks with robust discriminative capabilities. The convolutional neural network (CNN)-transformer model is specifically designed to integrate the strengths of the CNN and transformer, facilitating effective coupling of feature information. However, previous CNN-transformer studies have not effectively mitigated the interference of feature distribution differences as well as pseudovariations between two images due to cloud occlusion, imaging conditions, and other factors. In this article, we propose a domain adaptive and interactive differential attention network (DA-IDANet). This model incorporates domain adaptive constraints (DACs) to mitigate the interference of pseudovariations by mapping the two images to the same deep feature space for feature alignment. Furthermore, we designed the interactive differential attention module (IDAM), which effectively improves the feature representation and promotes the coupling of interactive differential discriminant information, thereby minimizing the impact of irrelevant information. Experiments on four datasets demonstrate the superior validity and robustness of our proposed model compared to other SOTA methods, as evident from both quantitative analysis and qualitative comparisons. The code will be available online (https://github.com/Jyl199904/DA-IDANet).

Hyperspectral image (HSI) has fine spectral resolution and abundant spatial information to detect subtle differences between targets. However, it is heavily contaminated with noise due to sensor design and atmospheric radiative transfer, resulting in spectral shifts and spatial discontinuities. Current denoising methods usually establish constraints directly on the ground truth and denoised image, lacking supervision of intermediate parameters of the network, resulting in insufficient model constraints and poor convergence. In addition, existing methods do not consider spatial-spectral compensation, so the denoising results have obvious spatial-spectral distortion. To this end, we propose a novel multiscale spatial-spectral invertible compensation network (MSIC-Net) for HSI denoising. The method constructs an invertible spatial-spectral compensation (ISSC) module, which supervises intermediate features through inverse constraints, realizes the circulation of multiscale information, and improves the stability of the model. At the same time, we also introduce style transfer for spatial-spectral compensation, which uses its superior fine feature control ability to precisely compensate for the lost spatial and spectral detail features. The method is extensively validated experimentally and categorically on simulated and real datasets. The experimental results show that MSIC-Net outperforms other state-of-the-art denoising methods in quantitative and qualitative evaluations.

Hyperspectral coastal wetland classification requires an extensive quantity of labeled samples, which are hard to acquire. Therefore, a novel semi-supervised dynamic ensemble learning (SSDEL) framework is proposed to overcome the limitations of labeled samples in wetland hyperspectral classification. Firstly, a collaborative relationship is established between labeled and unlabeled samples in the sample augmentation stage. Based on this relationship, unlabeled samples were assigned to the region to which the most similar samples belonged. Then, multiple classifiers are trained using labeled samples and predict unlabeled samples in the same region to obtain higher confidence pseudo-label results. Secondly, based on the assumption that different classifiers should produce similar classification results for a specific target sample, an objective function is designed to unify the classification behavior of multiple classifiers. The representation coefficients of multiple classifiers in the same region are constrained by optimizing the objective function through thel2norm. Finally, a complete SSDEL framework is constructed by applying consistency learning again to the augmented samples. The proposed method is evaluated using three wetland hyperspectral images of China, and the experiments results demonstrate its effectiveness.

The integration of hyperspectral (HS) imagery and light detection and ranging (LiDAR) for land cover classification has become a significant research topic. Numerous existing methods aim to interactively fuse the complementary features of HS and LiDAR to enhance the classification accuracy. However, most existing studies overlook the fact that spectral, spatial, and elevation features of HS and LiDAR possess significant discriminative information for specific categories. The rough and simple interacting or stacking these features may hinder the effective expression of this significant discriminative information. Moreover, existing approaches neglect the shared spatial characteristics between HS and LiDAR. In this article, an uncertain category-aware fusion network (UCAFNet) is proposed to tackle the above challenges. Specifically, we proposed an uncertain category-aware fusion strategy (UCAFS) that dynamically weights the spectral, spatial, and elevation branches based on their respective capabilities in identifying different categories to achieve targeted information aggregation. Moreover, we introduce the spatial information purification module (SIPM) and adaptive weighted fusion module (AWFM), to extract and enhance shared spatial features from HS and LiDAR for effective integration. The experimental results on three public benchmark datasets demonstrate the superior performance of the proposed UCAFNet.

Contrastive learning achieves good performance on hyperspectral image classification (HSIC), but its application on cross-scene classification is still challenging due to domain shift. The emergence of domain adaptation (DA) techniques can reduce domain discrepancy and transfer a model between two domains. Recently, instance-level contrast adaptation methods can connect two related domains, and domain-invariant features are extracted. However, it is sensitive to noisy samples and only learns low-level discriminative features. To solve these problems, a novel domain invariant and compact prototype contrast adaptation (DIC-proCA) framework is proposed for HSIC. About the proposed DIC-proCA, the prototype is introduced into the contrastive learning framework, which serves as a representative embedding of semantically similar samples, has class representativeness and can alleviate the negative impact of outliers. Taking into account the class representativeness of the prototype and the discriminability of the sample itself, a bidirectional inter-domain instance-to-prototype contrastive loss is proposed. It explicitly expresses feature relationships between categories in different domains, and then extracts domain-invariant features. Meanwhile, the mining of compact discriminative features within the target domain is facilitated by instance-level contrastive learning after data augmentation. In addition, the strategy of label smoothing promotes the clusters in the domain to be more compact and evenly separated, making the model more generalizable. Three cross-scene HSIC tasks demonstrate that the proposed DIC-proCA exhibits superior performance compared to some advanced DA algorithms.

The hyperspectral imaging instrument makes a trade-off by sacrificing spatial resolution to achieve high spectral resolution. This compromise leads to a low signal-to-noise ratio, and hyperspectral images (HSIs) are often heavily contaminated with mixed noise, which is an inherent challenge. Previous research has achieved satisfactory results for natural image denoising; hyperspectral denoising has remained a formidable task. In this article, we introduce an innovative method called the multistage hybrid-denoising network for satellite hyperspectral images (SUC-MSDN). SUC-MSDN initially decomposes the noisy HSI into multiple scales and constructs a multistage denoising network by analyzing the spatial spectrum texture distribution characteristics of noise signals. Instead of simply stacking the output results from each scale, SUC-MSDN uses the denoising results from the low-scale network as prior knowledge for the high-scale denoising network to more accurately remove the final noise components. Extensive experimental datasets are used to validate the performance of SUC-MSDN. Experimental results show that SUC-MSDN outperforms benchmark methods and significantly enhances the accuracy of land cover mapping.

This paper proposes a novel deep learning-based framework for hyperspectral band selection, named Cross-Scene Self-Representative Network (CSSRnet). The proposed method leverages the rich labels of the source domain (SD) to guide the band selection in the target domain (TD). To our knowledge, CSSRnet is the first deep learning-based solution for cross-scene hyperspectral band selection. First, the CSSRnet employs contextual attention mechanism to capture the latent features of SD and TD. It combines the self-attention mechanism with convolutional operations to capture static and dynamic contextual information. Then, the self-representative layer provides the self-representative coefficient of SD and TD. Subsequently, the maximum mean difference is utilized to align the self-representative coefficients of both SD and TD. To enhance the representativeness and precision of these coefficients, we introduce different tasks for the SD and TD branches. Finally, a suitable band subset is selected based on a ranking method that evaluates each band’s importance by considering its self-representative coefficient matrix. Experiments are carried out to assess the efficacy of CSSRnet. These experiments focus on evaluating classification accuracy across various cross-scene datasets, the utility of cross-scene concepts, and the practical application in coastal wetland. Experimental results confirm the effectiveness of CSSRnet.

While fusion of hyperspectral images (HSIs) with low spatial resolution and multispectral images (MSIs) with high spatial resolution has achieved significant success, high-quality fusion between noisy images has always been challenging. In this article, we propose a domain transform model driven by deep learning for anti-noise hyperspectral and multispectral image fusion (DTAFN). This marks the first time that wavelet decomposition theory is combined with deep learning for noise reduction in hyperspectral and MSI fusion. DTAFN initially decomposes hyperspectral and MSIs into frequency components and constructs a novel feature interaction fusion module (FIFM). This module, while using MSIs to guide the removal of noise from HSIs, also achieves the fusion of spatial and spectral information. Furthermore, it maps the fused features to a lower dimensional subspace to enhance computational efficiency. Additionally, we introduce a spatial-spectral self-attention mechanism to optimize the reconstructed frequency components using the subspace features. In the end, the wavelet inverse transform is used to reconstruct the clean fused image. It is worth noting that the extraction of the subspace is considered a process of nonlinear low-rank component extraction, which, to a certain extent, suppresses noise signals. Numerous experiments of mixed noise image fusion are carried out, and the experimental results show that DTAFN can obtain high-quality fusion results, is robust, and superior to the state-of-the-art methods.

Spectral super-resolution (SSR) aims to improve the spectral resolution of images from multispectral imagery or even red, green, blue (RGB) images. However, the majority of existing SSR methods do not fully exploit the spatial and texture features in RGB images, which would lead to the image unreal and distort of the high-frequency details in the reconstructed SSR images. In this study, a hybrid spectral and texture attention pyramid network (STANet) is proposed to reconstruct hyperspectral images (HSIs) with RGB bands of remote sensing images as input. More specifically, a learnable texture feature extraction module is proposed, aiming to make full use of the texture features in the RGB images, which are important in the subsequent spectral reconstruction. Furthermore, to better reconstruct the correlations between various spectral channels, a spatial-spectral-constrained cross-attention module is introduced. Finally, a novel spectral-texture fusion method is proposed, which successfully alleviates the problem of insufficient deep interaction among multiple deep features. On three remote sensing datasets, STANet demonstrates state-of-the-art performance, with its peak signal-to-noise ratio (PSNR) exceeding the suboptimal methods by 0.7266, 0.6724, and 0.6 dB, respectively. The results of the land-cover classification experiment using the reconstructed HSI further demonstrated the performance of the STANet algorithm.

Hyperspectral image (HSI) change detection (CD) has gained significant attention in the field of remote sensing. Current CD methods typically extract features based on spatial or spectral correlations between bitemporal HSIs, which often overlook the difference information, leading to a decrease in CD accuracy. Furthermore, these algorithms do not fully consider the alignment of features between images across different channel and spatial dimensions. To tackle these issues, we propose a novel approach called the differential information-enhanced feature exchange network (DIEFEN) for HSI CD, which leverages the difference information between images and enhances the alignment of bitemporal image features to improve CD accuracy. Specifically, an enhanced differential multihead attention (EDMA) module is proposed to utilize difference information to guide the feature aggregation of bitemporal images, effectively highlighting changing pixels and suppressing unchanging pixels. A feature focus and long-range attention (FFLA) module is designed to extract local and global features, and a channel-spatial interaction (CSI) module is constructed to align features and mitigate the impact of noise. Experimental results on three HSI CD datasets demonstrate that the proposed DIEFEN method outperforms several state-of-the-art methods. The source code of the proposed DIEFEN is released athttps://github.com/creativeXin/DIEFEN.

Currently, deep learning methods represented by convolutional neural networks (CNNs) or Transformers are of great interest in hyperspectral image (HSI) classification. And recent works show that hybrid models using CNN and Transformer modules are expected to achieve better performance than when they are used alone. However, these hybrid models applied to HSI classification consider the combination of 2D CNN and Transformer, which makes the models have high computational complexity. And the information of multiple spectral dimensions different from ordinary RGB images has not been fully excavated. Based on this, we propose DBCTNet, a double branch Convolution-Transformer network. Specifically, a MSpeFE module is used for multiscale spectral feature extraction at the early stage of the proposed network. Then a ConvTE block is designed to improve the original Transformer encoder, where a Conv spectral projection unit and a convolutional multihead self-attention (CMHSA) unit are proposed to extract spatial and global spectral features. A double branch module is further built based on 3D CNN and ConvTE. This module can fully integrate spatial and local-global spectral features, while also having low computational complexity. Experiment results on four public datasets, Pavia University, Houston, WHU-Hi-LongKou and HuangHeKou, show that DBCTNet achieves satisfactory performance with a small number of parameters and relatively excellent efficiency compared to nine other networks. The implement of DBCTNet will be available publicly at https://github.com/xurui-joei/DBCTNet.

Graph convolutional networks (GCNs) have attracted significant attention in hyperspectral image (HSI) change detection (CD) due to their capability to perform shape-adaptive convolutions and capture complex patterns within HSIs. Existing GCN-based methods typically preprocess bitemporal HSIs into graphs using a specific superpixel segmentation. However, this preprocessing step limits the modeling of spatial topologies to a fixed scale. Besides, these methods do not consider distribution shifts between bitemporal HSIs. To overcome these limitations, this article proposes a fine–coarse–fine progressive graph framework with distribution alignment (FCFDA) to learn progressive features across multilevel graphs for HSI-CD. Specifically, for each bitemporal HSI, we generate multiple hierarchical segmentations ranging from fine to coarse by gradually merging neighboring superpixels and subsequently transforming these segmentations into multilevel graphs. Second, instead of simply concatenating features from different hierarchies, FCFDA integrates them progressively from fine to coarse and then back to fine, generating subtle features tailored to the pixel-wise CD task. Finally, an effective distribution alignment (DA) method is designed to align the feature space of the bitemporal HSIs, thus mitigating the adverse effects of distribution shifts. Experiments conducted on real HSI-CD datasets demonstrate the effectiveness and superiority of the FCFDA.

Graph neural networks (GNNs) have garnered significant attention in hyperspectral image (HSI) change detection (CD). However, existing GNN-based methods extract features by aggregating neighborhood information, which is essentially a low-pass Laplacian smoothing operation and tends to diminish change information between bitemporal HSIs. In addition, these methods rely on fixed hand-crafted graphs, and thus cannot capture complex structures of HSIs well. To address these deficiencies, this paper develops a Sharpening-Aware Graph Network (SAGN) for achieving high-quality HSI CD. Firstly, to counteract the weakening of differences caused by Laplacian smoothing, this paper proposes a novel Laplacian sharpening-based graph convolution (LSGC) module to accentuate change information between bitemporal HSIs. Secondly, instead of using “similarity graphs”, this paper constructs untied “difference graphs” for bitemporal HSIs to model dissimilarities between changed pixels and their neighbors. The SAGN can dynamically update graph structures across all layers, aiming to further maximize the divergence. Finally, a joint loss function, incorporating modified cross-entropy loss and contrastive loss, is devised to enhance inter-class discrimination of learned features and alleviate the issues stemming from imbalanced labeled samples. Experiments on various HSI CD datasets demonstrate the effectiveness and superiority of the proposed SAGN.

In recent years, few-shot learning (FSL) has made significant progress in hyperspectral image classification (HSIC) by transferring metaknowledge from a labeled source domain to a target domain with very limited labeled samples. Considering that natural images have rich spatial texture information, heterogeneous FSL (HFSL) by using natural images as the source domain and hyperspectral image (HSI) as the target domain has shown excellent performance. However, some problems also exist in the HFSL, such as poor generalization ability from natural images to HSIs, prototype instability due to limited labeled samples, and domain shift between different types of images. To address these problems, we propose a mutual information enhancement FSL (MIEFSL) method for HSIC, which mainly contains three modules, i.e., mutual information enhancement (MIE), intradomain prototype rectification (IPR), and interdomain distribution alignment (IDA). In order to improve the generalization ability of the network and preserve the raw data information as much as possible, an MIE module is designed to maximize the mutual information (MI) between the support set samples and their corresponding masked samples. To stabilize the prototypes, an IPR module is constructed through a distribution expansion strategy. In addition, to alleviate domain shifts between different types of images, an IDA is performed between source and target domains. Experimental results demonstrate that the proposed MIEFSL outperforms existing state-of-the-art FSL methods and achieves the overall accuracy (OA) of 78.34%, 90.31%, and 91.72% on Indian Pines (IP), University of Pavia (UP), and Salinas (SA) in the case of only five labeled samples, respectively.

Labeled samples are important in achieving land cover change detection (LCCD) tasks via deep learning techniques with remote sensing images. However, labeling samples for change detection with bitemporal remote sensing images is labor-intensive and time-consuming. Moreover, manually labeling samples between bitemporal images requires professional knowledge for practitioners. To address this problem in this article, an iterative training sample augmentation (ITSA) strategy to couple with a deep learning neural network for improving LCCD performance is proposed here. In the proposed ITSA, we start by measuring the similarity between an initial sample and its four-quarter-overlapped neighboring blocks. If the similarity satisfies a predefined constraint, then a neighboring block will be selected as the potential sample. Next, a neural network is trained with renewed samples and used to predict an intermediate result. Finally, these operations are fused into an iterative algorithm to achieve the training and prediction of a neural network. The performance of the proposed ITSA strategy is verified with some widely used change detection deep learning networks using seven pairs of real remote sensing images. The excellent visual performance and quantitative comparisons from the experiments clearly indicate that detection accuracies of LCCD can be effectively improved when a deep learning network is coupled with the proposed ITSA. For example, compared with some state-of-the-art methods, the quantitative improvement is 0.38%-7.53% in terms of overall accuracy. Moreover, the improvement is robust, generic to both homogeneous and heterogeneous images, and universally adaptive to various neural networks of LCCD. The code will be available at https://github.com/ImgSciGroup/ITSA.

In recent years, with the continuous development of deep learning (DL), neural networks have demonstrated good results in large-sample hyperspectral image (HSI) classification. However, in practice, labels are often limited. In order to use fewer labeled samples without degrading the classification performance, this letter proposes a new semi-supervised classification method named prototype and active learning network (PALN), which integrates DL, active learning (AL) and prototype learning (PL) into a framework. After training the DL network with a small number of available labels, samples with high uncertainty are selected by AL to assign true labels, while samples more similar with prototypes are chosen by PL with their pseudo labels, and all selected samples are appended to the training set for the next training. Compared with existing classification methods, our method achieves good performance on two hyperspectral datasets.

Land cover change detection (LCCD) with bitemporal remote sensing images has been widely used in practical applications. However, when the bitemporal images are multimodal remote sensing images (MRSIs) which are acquired with different sensors, the change detection performance may be unsatisfactory, because MRSIs cannot be compared directly to generate a change magnitude and obtain a change detection map. Here a novel approach is proposed to overcome this problem, i.e., the Enhanced UNet (E-UNet) which learns deep shared features from MRSIs to achieve change detection with MRSIs. First, apre-event image to post-eventimage (P2P) transformation module based on classical Cycle-consistent Generative Adversarial Network (CGAN) is suggested to embed at the head of the proposed E-UNet to translate the pre-event image to a post-event image one. Then, multi-scale convolutions are added at each encoding layer to capture the various shapes and sizes of ground targets. Finally, a Polarized Self-Attention (PSA) module is employed before beginning the decoding progress of E-UNet with an aim to pay extra attention to changed areas. Compared with five typical state-of-the-art methods, experimental results based on two pairs of MRSIs well demonstrated the feasibility and advantages of the proposed E-UNet for LCCD with MRSIs in terms of visual observations and quantitative evaluations. For example, the improvement is 4.19% and 4.75% in terms of the overall accuracy for the Sardinia dataset and California dataset, respectively. The code of the proposed approach can be found at https://github.com/ImgSciGroup/E-UNet.

Multispectral (MS)-hyperspectral (HS) image fusion, which aims to enhance the spatial resolution of low spatial resolution HS images with a high spatial resolution MS has provided a wide range of applications in remote sensing. However, relatively long revisit cycles of HS satellites and irresistible weather factors cause the acquisition of HS and MS images at the same time difficult. Most of the existing approaches neglect the temporal difference between MS and HS images, and perform weakness in the challenging case with diverse temporal difference spans. In this paper, we propose a novel image fusion strategy with embedding a stage of feature matching before interaction. On the one hand, we explore the role of spectral correlation modeling between HS and MS images, which accounts for the utilization of available spatial information from MS images. On the other hand, we design a feature aggregation module to fully exploit the nonlinear gaps and dependencies of heterogeneous data and utilize adaptive gains to realize complementary information projection and fusion. We build Dongying (DY) and Yellow River Estuary (YRE) remote sensing datasets based on Sentinel-2 and ZiYuan(ZY)-1 02D satellites with diverse temporal difference spans. The extensive experiments demonstrate that our method is robust to the span of temporal difference and shows superior performance over the existing methods visually and quantitatively.

Spatial-temporal-spectral fusion (STSF) is highly desirable to generate dense-time image series with high spatial and spectral resolution by integrating the complementary advantages of multi-source and multi-temporal observations. However, most existing STSF methods are still limited to the assumption of linear temporal, spatial and spectral relations. In addition, the STSF methods on Landsat and MODIS data are insufficient to characterize the inherent properties of the current spaceborne hyperspectral images with low spatial and temporal resolutions. For these, we propose a progressive STSF network (PSSTFN) by interestingly integrating spatial-spectral fusion and spectral-temporal fusion into a unified end-to-end STSF framework. Specifically, in the spatial-spectral fusion stage, we obtain the hierarchical features with different receptive fields and propose a multi-attention guided module for joint learning and refinement of spatial-spectral features. In the spectral-temporal fusion stage, a feature insertion module is presented to embed the difference images into the resulting spatial-spectral features, and the estimation from deeper layers is cascaded for more reliable spatial information. We build Dongying (DY) and Yellow River Estuary (YRE) remote sensing datasets based on Sentinel-2 and ZiYuan(ZY)-1 02D satellites for verification, and the experimental results on reduce- and full-resolution data demonstrate the superior performance of our method over the existing methods visually and quantitatively.

Deep learning methods face significant challenges in practical cross-scene classification tasks of hyperspectral images, primarily due to the difficulty of acquiring labels and the issue of inconsistent distribution caused by spectral drift. To tackle the above issues, we propose a deep dynamic adaptation network based on joint correlation alignment (DDAN-JCA) for cross-scene hyperspectral image classification. First, the dual-channel residual network (DCRN) and the attention mechanism module (AMM) are employed to extract spatial-spectral joint features from both source domain and target domain. Then, the method of correlation alignment (CORAL) is employed to minimize the marginal distribution discrepancy between two domains and further reduce the conditional distribution discrepancy of each class. Finally, a dynamic distribution adaptation strategy is used to dynamically adjust the importance of marginal distribution and conditional distribution by using a balance factor. DDAN-JCA can achieve unsupervised classification without using target labels. The performance of DDAN-JCA has been validated using three hyperspectral datasets, and the experimental results demonstrate that DDAN-JCA significantly enhances classification accuracy and exhibits greater robustness compared to state-of-the-art methods.

Spatio-temporal-spectral fusion aims to produce high spatio-temporal-spectral resolution images by integrating the complementary spatial, temporal, and spectral advantages of multi-source remote sensing images. However, on one hand, existing spatio-temporal-spectral fusion methods are insufficient to exploit the inherent complex nonlinear spatial, temporal, and spectral relationship among multisource and multitemporal observations. On the other hand, since the unavailability of real high spatio-temporal-spectral resolution images, it is difficult to adopt deep learning methods with supervised training. In this paper, we propose an effective Unsupervised Spatio-Temporal-Spectral Fusion Model (USTSFM) with dual-task interactive learning to alleviate these problems. The proposed USTSFM has two branches: the Spatio-Temporal-Spectral Mapping (STSM) branch is to describe the temporal relationship, and the Spectral Super Resolution (SSR) branch is to model the spectral relationship. Moreover, the spatial-spectral interaction compensation block is designed to make the two branches compensate and benefited from each other. This intrinsically related and mutually facilitated strategy allows the USTSFM to sufficiently exploit the inherent spatial, temporal, and spectral relationship. In addition, a shared reconstruction module is meticulously designed for the two tasks, which not only reduces the parameters but also allows the supervised task to guide the convergence of the unsupervised task, boosting the stability of unsupervised training. The qualitative and quantitative results demonstrated the proposed USTSFM has richer spatial details and more accurate predictions than the other state-of-the-art methods.

Deep learning has emerged as a powerful method for hyperspectral image (HSI) classification. However, a significant prerequisite for HSI classification using deep learning is enough labeled samples, which is both time-consuming and labor-intensive. Yet, labeled samples are essential for training deep learning models. This paper proposes an HSI classification method based on the self-supervised learning of spectral masking (SSLSM). The method mainly includes two steps: self-supervised pre-training and fine-tuning. First, considering the rich spectral information of HSI, we propose masked spectral reconstruction as the pretext task. The unmasked data is input into the encoder and decoder sequentially, which are composed of a multi-layer transformer, for feature learning for masked spectral reconstruction. Second, we use reference samples to fine-tune the network, and the encoder and decoder are innovatively cascaded for deep semantic feature extraction, which can further improve the ability of feature extraction in the downstream classification tasks. Experiment results show that, compared with other methods, the SSLSM obtains the highest classification accuracy of 96.52%, 97.03%, and 96.70% on the Indian Pines dataset, Pavia University dataset, and Yancheng Wetlands dataset, respectively. Our method can also be applied to other HSI datasets, and the codes will be available from https://github.com/CIRSM-GRoup/2023-TGRS-SSLSM.

Deep learning has been extensively used for hyperspectral image (HSI) classification with significant success, but the classification of high-dimensional HSI datasets with a limited amount of labeled samples is still a great challenge. Few-shot learning (FSL) has shown excellent performance in solving small-sample classification problems. However, most of the existing FSL methods usually suffer from the prototype instability and domain shift. In order to address these problems, this paper proposes a category-specific prototype self-refinement contrastive learning (CPSRCL) method for cross-domain FSL of HSIs. Our method uses a supervised contrastive learning (SCL) strategy to promote intra-class compactness and inter-class dispersion of features in the metric space. To stabilize and refine the prototypes of the support set, a category-specific prototype self-refinement (CSPSR) module is designed to adaptively learn different updating rules for different category prototypes using rich labeled information in the query set. Furthermore, a local discriminative domain adaptation (LDDA) method is constructed to align the global distribution between source and target domains while preserving domain-specific discriminative information. Experimental results on four public HSI datasets demonstrate that CPSRCL outperforms existing FSL and deep learning methods for HSI classification.

Recently, prototypical network based few-shot learning (FSL) has been introduced for small-sample hyperspectral image (HSI) classification and shown good performance. However, existing prototypical-based FSL methods have two problems: prototype instability and domain shift between training and testing datasets. To solve these problems, we propose a refined prototypical contrastive learning network for few-shot learning (RPCL-FSL) in this paper, which incorporates supervised contrastive learning and FSL into an end-to-end network to perform small-sample HSI classification. To stabilize and refine the prototypes, RPCL-FSL imposes triple constraints on prototypes of the support set, i.e., contrastive learning (CL), self-calibration (SC) and cross-calibration (CC) based constraints. The CL module imposes internal constraint on the prototypes aiming to directly improve the prototypes using support set samples in the CL framework, and the SC and CC modules impose external constraints on the prototypes by using the prediction loss of support set samples and the query set prototypes, respectively. To alleviate domain shift in the FSL, a fusion training strategy is designed to reduce the feature differences between training and testing datasets. Experimental results on three HSI datasets demonstrate that the proposed RPCL-FSL outperforms existing state-of-the-art deep learning and FSL methods.

Wetland contains numerous features, and label acquisition is time-consuming, laborious, and inaccurate. Coastal wetland land cover classification with limited labeled training samples has become a significant challenge. In this study, a novel probabilistic ensemble sample selection framework (ProESS) is proposed for coastal wetland land cover classification. First, a sample probabilistic confidence index (SPCI) is proposed, which is defined by probabilistic output of each base classifier. Then the prediction confidences of unknown samples are obtained by joint probability of ensemble base classifiers, which can select high-quality samples to improve classification performance. However, the low accuracy of base classifiers will affect the confidence of samples selected by SPCI, thus reducing the classification accuracy. Based on this observation, an adaptive representation coefficient consistency learning (AdaRCCL) is proposed to help define SPCI. Finally, a ProESS is constructed through SPCI and AdaRCCL which can obtain training samples with high confidence from unknown samples. To evaluate the effectiveness of proposed methods, the three wetland hyperspectral datasets of China, i.e., Yangtze River Delta, Jiangsu Dafeng Natural Reserve, and Yellow River Delta, are used for classification experiments in the paper. Experimental results show that the proposed algorithms achieve higher performance and are robust to parameters in comparison to the baseline and the state-of-the-art ensemble algorithms. The extensibility and transferability of proposed methods are also discussed in the paper. Better results on multiple machine learning models with new samples show the extensibility of SPCI and ProESS. The great performance on Botswana dataset also demonstrates the transferability of proposed methods.

Deep learning techniques have become popular in land cover change detection (LCCD) with remote sensing images (RSIs). However, many existing networks mostly concentrate on learning deep features but without considering the effect of different features’ attention and fusion strategy on detection performance. In this paper, a novel hierarchical attention feature fusion (HAFF)-based network for LCCD with RSIs is proposed. In the proposed HAFF-based network, novel multi-scale convolution fusion filters (MCFFs) explore the global semantic feature of the interested targets from multi-perspectives ways. To achieve that objective, the proposed MCFFs are composed by a well-known position attention module (PAM) and a novel multi-perspectives feature filter block with different kernel sizes. In addition, a compound loss function was proposed for balancing the impact from the features at different levels in terms of backpropagation error. Experiments conducted on six pairs of real RSIs, including three pairs of homogeneous images and three pairs of heterogeneous images, confirmed the superiority of the proposed HAFF network over other cognate methods. Moreover, the ablation experiments further confirmed the feasibility and superiority of the proposed MCFFs, whereas quantitative observations indicated that competitive improvements are achieved by the proposed MCFFs in terms of all the evaluation indicators. The code for the proposed approach will be available at https://github.com/ImgSciGroup/HAFF.

Samples play a crucial role in the supervised classification of remote sensing images. However, labeling large samples for training a classifier or deep learning network is not only time-consuming but also labor-intensive. In this paper, a novel land cover classification with nonparametric sample augmentation is proposed to improve the performance of hyperspectral remote sensing images (HRSIs) classification. First, initial samples with limited quantity are selected randomly from the ground truth map. Second, based on the gray image, a nonparametric adaptive region generation (NARG) algorithm is developed for utilizing the contextual information around each sample. Then, an nonparametric sample augmentation algorithm is developed with NARG to explore reliable samples iteratively around each initial sample. Finally, the above steps are fused into an iterative progress to obtain the final classification map. Compared with some typical traditional methods and some widely used deep learning methods based on four real HRSIs, our proposed approach exhibits some advantages in improving the visual performance and quantitative accuracies of HRSIs classification, such as the improvement is about 2.0% ~ 10.34% for four real HRSIs in term of the overall accuracy.

Land cover change detection (LCCD) using bitemporal remote sensing images is a crucial task for identifying the change areas on the Earth’s surface. However, the utilization of hyperspectral remote sensing images (HRSIs) introduces challenges as the detection performance is affected by the spectral noise and deducing change detection accuracies. In this work, we concentrated on utilizing spatial-contextual information to improve the change detection performance while using HRSIs. First, a band selection approach is used to minimize the spectral redundancy of HRSIs. Second, an iterative spatial-adaptive filter is proposed to smooth the noise of HRSIs. Thereafter, the change magnitude between bitemporal HRSIs is measured by coupling change vector analysis and the adaptive region around each pixel, resulting in a change magnitude image (CMI). Subsequently, the CMI is divided into a binary change detection map by using an Ostu threshold method. The experimental results on three pairs of real HRSIs efficiently demonstrated the feasibility and superiorities of the proposed approach compared with six state-of-art methods. For example, the improvement rates are approximately 0.43%-11.83% and 1.05%-15.41% for overall accuracy and average accuracy, respectively. The code of our proposed approach will be available at: https://github.com/ImgSciGroup/2023-HSICD.

Spectral-spatial features are important for ground target identification and classification with High Spatial Resolution Remotely Sensed (HSRRS) Imagery. In this paper, two novel features, named the Gaussian-Weighting Spectral (GWS) feature and the Area Shape Index (ASI) feature, are proposed to complement the deficiency of the basic image feature for land cover classification with HSRRS imagery. The proposed GWS feature is an adaptive region-based feature that aims to improve the spectral homogeneity of a local area surrounding a pixel. Additionally, it is well known that the spectral feature is inadequate for classifying HSRRS imagery. Therefore, one spatial feature called the ASI feature is proposed here to describe the relationship between the area and shape for an adaptive region around each pixel. The proposed GWS and ASI features coupled with the basic red-green-blue feature are fed into a supervised classifier to obtain the final classification map. Experiments based on four real HSRRS images demonstrate that the proposed GWS and ASI features are capable of improving classification accuracies compared with some cognate state of the art methods. Moreover, the experiments also reveal that the proposed spectral-spatial features can complement each other for enhancing the classification performance with HSRRS images.

Cross-scene hyperspectral image classification (HSIC) is a challenging topic in remote sensing, especially when there are no labels in target domain. Domain adaptation (DA) techniques for cross-scene HSIC aim to label a target domain by associating it with a labeled source domain. Most existing DA methods learn domain-invariant features by reducing feature distance across domains. Recently, contrastive learning has shown excellent performance in computer vision tasks, but there is little or no research on the performance of cross-scene HSIC. Considering that its idea is similar to reducing feature distance, this paper attempts to explore whether contrastive learning can achieve cross-scene HSIC. In this work, an instance-to-instance contrastive learning framework based on category matching (CLCM) is designed. The main idea is to take the category information as the premise in the feature space, regard the source sample as an anchor, and find its positive and negative matching samples across domains. The instance-level discriminative feature embeddings are learned through positive matching pairs attracting each other and negative matching pairs repelling each other. Among them, the target label is a pseudo-label. To further improve the quality of contrastive learning, it is considered to focus on extracting the spectral-spatial features of HSI to more accurately represent semantic information. Simultaneously, high-confidence target samples are screened to update the network. Three DA tasks confirm the effectiveness and feature discriminativeness of CLCM, while also providing new ideas for cross-scene image classification.

Benefiting from powerful feature extraction capabilities, convolutional neural networks (CNNs) have gained prominence in hyperspectral image (HSI) classification. Nevertheless, with restricted receptive fields of convolution kernels, CNN-based methods fail to learn complex characteristics of long-range sequences. Meanwhile, vision transformer allows us to learn long-range dependencies in a global view, but local region features are ignored. To overcome these limitations, we propose a novel method entitled global-local three-dimensional convolutional transformer network (GTCT), where 3-D convolution is embedded in a dual-branch transformer to simultaneously capture global-local associations in both spectral and spatial domains. In particular, the global-local spectral convolutional transformer (GECT) is designed to exploit global spectral sequence signatures and local spectral relationships between bands. Symmetrically, the global-local spatial convolutional transformer (GACT) is devised to exploit local spatial context features and global interactions among different pixels. In addition, multiscale global-local spectral-spatial information is adaptively fused with trainable weights by the weighted multiscale spectral-spatial feature interaction (WMSFI) module. It is worth noting that a spectral-spatial global attention mechanism (SSGAM) is incorporated into multi-head convolutional attention to further integrate discriminative spectral-spatial information. Extensive experiments on four HSI datasets, including GF-5 and ZY1-02D satellite hyperspectral images, demonstrate the superiority of the proposed GTCT method over other state-of-the-art algorithms with fewer parameters and lower floating-point operations (FLOPs) in practical applications.

Convolutional neural network (CNN) has achieved great success in hyperspectral image (HSI) classification. However, the local receptive field of CNN leads to the drawback in extracting long-distance features. Transformer has excellent global modeling ability and shows good performance for HSI classification. The existing Transformer-based methods usually ignore a problem that the spatial information varies under different channels. To well describe the cross-channel dependencies, a cross-channel dynamic spatial-spectral fusion transformer (CDSFT) is proposed in this article. In the proposed CDSFT, the multi-scale and multi-channel features are extracted and then cross-channel global features are extracted through transpose multi-head self-attention (TMHSA). Next, a dynamic feature enhancement module and a spectral spatial position attention module are designed to extract and enhance spectral-spatial joint features for classification. Experimental results on three well-known HSI datasets demonstrate the effectiveness of the proposed CDSFT method.

Image registration aims to eliminate the geometric deviation between multi-source data with the same range, and to promote the collaborative application of data. In recent years, spaceborne hyperspectral (HS) and multispectral (MS) data have been widely used in Earth observation. However, the difference in the number of bands, spatial resolution, and spectral resolution puts forward higher requirements on the registration algorithm. The key to HS and MS image registration is to extract more common key points, weaken and eliminate the difference of radiation and spatial texture information to build superior descriptors, and achieve high-precision matching of key points. This paper introduces a new robust HS and MS registration method based on common deep feature subspaces. We first construct the common deep feature subspaces extraction network to extract consistent edge features and common subspace images of the image pair. Then, Harris algorithm is used to extract key points from consistent edge features between images, which reduces the impact of spatial resolution differences between images. Besides, the SIFT descriptor and subspace images are used to describe key points, which reduces the impact of radiation differences between images. Finally, Euclidean distance is used for the initial matching of key points, and the affine matrix is calculated after the outliers are eliminated, and image registration is performed. We perform experiments on spaceborne HS and MS datasets of different spatial resolutions and comparisons with state-of-the-art methods. Experimental results show that our method can obtain satisfactory registration results and is robust.

Protection of wetlands is important for ecosystem in recent years, and the classification of wetland ground cover is the foundation of investigation and protection work. Probabilistic collaborative representation classifier (ProCRC) is one of the best performing classifiers which has been applied in hyperspectral image (HSI) classification. However, its performance is greatly limited for wetland data where spectrums are highly similar. Moreover, the complex distribution of ground objects in wetlands have not been wisely utilized in the classification. In this article the intrinsic mechanism of ProCRC is found and its kernel version is proposed to solve the problems of wetlands classification. Then, a new ensemble learning strategy that considers neighborhood information are proposed, which largely alleviates the problem of sample collection in wetlands. Under the guidance of this strategy, two specific ensemble learning algorithms, i.e., LNE and LNSAE, are proposed. The superiority of proposed methods is validated using three typical HSI data sets of China coastal wetland with few samples.

Due to sensor design limitations and the influence of weather factors, it is currently challenging to obtain remote sensing images with high temporal, spatial, and spectral resolution. Spatial-temporal-spectral fusion aims to integrate the temporal, spatial, and spectral information from multiple sources of remote sensing images to reconstruct a remote sensing image with high temporal, spatial, and spectral resolution. Existing methods typically require at least three types of data to achieve spatial-temporal-spectral fusion. However, acquiring remote sensing data observed at the same time poses significant difficulties. The major challenge lies in effectively utilizing hyperspectral images with low spatial and temporal resolution and multispectral images with high temporal and spatial resolution to reconstruct remote sensing images with high temporal, spatial, and spectral resolution. To address the aforementioned issues, we propose a novel unsupervised 3D tensor subspace decomposition network. Our method incorporates the theory of 3D tensor subspace decomposition, utilizing a 3D hyperspectral/multispectral tensor subspace extraction network to predict the hyperspectral tensor subspace features with low spatial resolution missing at other times (To better understand, the missing moment is defined as time 2). Subsequently, the 3D hyperspectral tensor subspace reconstruction network is employed along with the time 2 hyperspectral tensor subspace features with low spatial resolution and the time 2 multispectral image to reconstruct the time 2 hyperspectral image with high spatial resolution. In the experiment, we utilize three simulated datasets and two real datasets to evaluate the fusion performance of our proposed method. The results demonstrate that our method achieves high-quality fusion results and exhibits comparable performance, and has robustness and practicality.

Spatial-temporal-spectral fusion (STSF) of remote sensing imagery can produce data with the highest spatial and spectral resolution, only as well as fine temporal resolution, by integrating images with complementary information in both the temporal and spectral domains. Accuracy of temporal variation is an important guarantee for achieving fidelity fusion in STSF. However, current STSF methods estimate the temporal variation only by utilizing the temporal variation between observed multispectral image (MSI) and the relationship between MSI and hyperspectral image (HSI), which is difficult to obtain accurate temporal variation. To address this problem, this paper proposes a coupled temporal variation information estimation and resolution enhancement for remote sensing image spatial-temporal-spectral fusion (CTVRE-STSF). The temporal variation information estimation model estimates the temporal variation of the target image, while the resolution enhancement model provides additional constraints for estimating the temporal variation. For the temporal variation information reconstruction model, we build a temporal variation information estimation based on a generalized linear mixed model and use the temporal variation between MSIs. In addition, a resolution enhancement model is constructed to estimate the temporal variation of the target image by incorporating relevant prior knowledge. The introduction of the resolution enhancement model in the prior provides additional constraints on the estimation of the temporal variation high-dimensional information, thus facilitating the resolution improvement. Experimental results on two real datasets demonstrate the effectiveness and superiority of our proposed method over current state-of-the-art methods, especially in terms of spectral fidelity.

The pursuit of super-resolution (SR) with large upscaling factors such as 8×, for enhancing the spatial resolution of low-resolution (LR) remote sensing images is a persistent and challenging problem. To address this issue, we propose the Progressive Feature Enhancement SR (PFESR) network with an 8× upscaling factor. Given the limited high-frequency information provided by a single LR image, we propose an improved style transfer technology to generate auxiliary details that aid in the recovery of high-resolution (HR) images. Additionally, multi-scale texture features are extracted through the Visual Geometry Group (VGG) feature extraction (VFE) block. To efficiently fuse various features, we combine hard and soft attention mechanisms. Finally, we use a hierarchical fusion block to address the progressive fusion problem of multiple scale features. Experiments on three datasets demonstrate that our method achieves state-of-the-art performance and exhibits good robustness in 8× and higher scale SR tasks.

Graph convolutional network (GCN) has recently attracted great attention in hyperspectral image (HSI) classification due to its strong ability to aggregate information of neighborhood nodes. However, a GCN model usually suffers from the over-smoothing problem (i.e., all nodes’ representations converge to a stationary point) when the number of GCN layers is increased. In addition, GCNs always work on superpixel-level nodes to reduce computational cost, so pixel-level features cannot be well captured. To deal with these problems, a novel two-branch deeper GCN (TBDGCN) is proposed to combine the advantages of superpixel-based GCN and pixel-based CNN, which can simultaneously extract superpixel-level and pixel-level features of HSIs. In the GCN branch, a GCN module with the DropEdge technique and residual connection is designed to alleviate over-smoothing and over-fitting problem, which results in a deeper network structure with more than ten layers. In the CNN branch, to capture spatial positional information and channel information, a mixed attention mechanism is constructed to extract attention-based spectral-spatial features. The features of the GCN and CNN branches are then fused for classification. Experimental results on three benchmark HSI data sets show that the classification performance of our TBDGCN is better than existing GCN models especially in the case of small sample size.

Deep learning (DL) based methods represented by convolutional neural networks (CNNs) are widely used in hyperspectral image classification (HSIC). Some of these methods have strong ability to extract local information, but the extraction of long-range features is slightly inefficient, while others are just the opposite. For example, limited by the receptive fields, CNN is difficult to capture the contextual spectral-spatial features from a long-range spectral-spatial relationship. Besides, the success of DL-based methods is greatly attributed to numerous labeled samples, whose acquisition are time-consuming and cost-consuming. To resolve these problems, a hyperspectral classification framework based on multi-attention Transformer (MAT) and adaptive superpixel segmentation-based active learning (MAT-ASSAL) is proposed, which successfully achieves excellent classification performance, especially under the condition of small-size samples. Firstly, a multi-attention Transformer network is built for HSIC. Specifically, the self-attention module of Transformer is applied to model long-range contextual dependency between spectral-spatial embedding. Moreover, in order to capture local features, an outlook-attention module which can efficiently encode fine-level features and contexts into tokens is utilized to improve the correlation between the center spectral-spatial embedding and its surroundings. Secondly, aiming to train a excellent MAT model through limited labeled samples, a novel active learning (AL) based on superpixel segmentation is proposed to select important samples for MAT. Finally, to better integrate local spatial similarity into active learning, an adaptive superpixel (SP) segmentation algorithm, which can save SPs in uninformative regions and preserve edge details in complex regions, is employed to generate better local spatial constraints for AL. Quantitative and qualitative results indicate that the MAT-ASSAL outperforms seven state-of-the-art methods on three HSI datasets.

The improvement of temporal resolution of hyperspectral (HS) data is a fundamental and challenging problem. In this paper, we propose a Temporal-Spectral fusion method based on Generative Adversarial Network (TSF-GAN). First, the generator is used to train the nonlinear relationship between multispectral (MS) and HS data pairs at time T1 and T3, and we map the relationship to the MS data at T2 to obtain the HS data. Second, the discriminator is used to identify whether the differential image of HS data at different times is consistent with that of MS data, and whether the HS data at time T2 after spectral down-sampling is consistent with that of MS data at time T2. Preliminary experimental results demonstrate that the proposed TSF-GAN achieves comparative fidelity and has strong practicability.

Domain adaptation (DA) aims to use a well-labeled source domain to predict the labels of the unlabeled or poor-labeled target domain. Most of the existing DA methods focus on the use of feature-level or sample-level information. Recent studies have shown that domain discriminative information is also important for classification. To jointly exploit feature-level information and discriminative information, a new DA method called distribution alignment and discriminative feature learning (DADFL) is proposed for hyperspectral image (HSI) classification in this letter. DADFL incorporates category-discriminative information preservation and structured prediction (SP)-based pseudolabeling into a unified framework to simultaneously reduce distribution and subspace differences between domains. Experimental results on three hyperspectral DA tasks show that the classification performance of the proposed DADFL is better than that of existing DA methods.

Deep neural networks have achieved promising performance for hyperspectral image (HSI) classification. However, due to the limitation of the available labeled samples, the traditional deeper and wider neural networks usually cause the overfitting problem and lose the detailed information. To solve this problem, a brain-like structure, namely spatial attention-driven recurrent feedback convolutional neural network (SARFNN), is proposed by utilizing the recurrent feedback and attention mechanism structures, from which two deep models are further developed for HSI classification. First, a 2-D SARFNN (SARF2DNN) model is developed to learn the spatial features from HSI data. After that, to better exploit the 3-D characteristic, the 3-D version is extended from SARF2DNN, thus constructing an SARF3DNN model to extract joint spatial-spectral features. Moreover, with the help of the idea of brain-likeness, the recurrent feedback module is designed to recover information loss caused by deeper structure and the dimension reduction operation. The experimental results conducted on two HSI data sets show that our SARFNN architecture can achieve more competitive performance than other state-of-the-art algorithms.

Zi Yuan (ZY)-1 02D satellite simultaneously provides the low spatial resolution (LR) and narrow swath-width hyperspectral (HS) image, the moderate spatial resolution (MR) multispectral (MS) image with a wider swath width, and the high spatial resolution (HR) panchromatic (PAN) image with the same wide swath width to the MR MS. How to comprehensively integrate their complementary advantages to obtain the wide swath-width and high-fidelity HR HS image is interesting but challenging. In this paper, we propose an integrated fusion method for the HR PAN, MR MS, and LR HS images with different swath widths, to generate the optimal wide swath-width HR HS image. The proposed method is based on the encoder-decoder learning framework. In the proposed fusion framework, a novel multi-branch encoder structure with an enhanced HS-encoder module and the multilevel spatial-spectral aggregation block is designed, by considering the difference in the spatial and spectral resolution among the multi-sensor images. The experiments on synthetic and real datasets from both qualitative and quantitative aspects demonstrated the competitive performance of the proposed method.

Pansharpening aims to sharpen a low spatial resolution (LR) multispectral (MS) image using a high spatial resolution (HR) panchromatic (PAN) image to obtain the HR MS image. Though large numbers of pansharpening methods have been proposed, and many advanced methods have shown high quantitative results, few of them are widely used in real applications. This may be attributed to their instability for different images with different ground surface features, or the complexity to be implemented and the time-consuming process for some state-of-the-art methods. In this letter, we proposed a simple, adjustable, and robust fusion (SARF) method. In the proposed method, a spatial-spectral coenhanced strategy was proposed, and several details of the proposed fusion model were specifically designed for the “simple, adjustable, robust” features. It was tested and verified by four-band and eight-band MS images based on reduced resolution (RR) and full resolution (FR) experiments. The experimental results demonstrated the promising spatial visuality of the proposed method, and the spectral fidelity was more robust than most of component substitution (CS)-based and multiresolution analysis (MRA)-based methods.

Nonnegative matrix factorization (NMF) is a widely used hyperspectral unmixing model which decomposes a known hyperspectral data matrix into two unknown matrices, i.e., endmember matrix and abundance matrix. Due to the use of least-squares loss, the NMF model is usually sensitive to noise or outliers. To improve its robustness, we introduce a general robust loss function to replace the traditional least-squares loss and propose a general loss-based NMF (GLNMF) model for hyperspectral unmixing in this letter. The general loss function is a superset of many common robust loss functions and is suitable for handling different types of noise. Experimental results on simulated and real hyperspectral data sets demonstrate that our GLNMF model is more accurate and robust than existing NMF methods.

This letter presents a multiscale low-rank decomposition (MSLRD) method to extract multiscale spatial structures from hyperspectral images. The MSLRD assumes that ground objects have divergent characteristics in changing spatial scales. It decomposes each band image into a series of block-wise matrices, where these low-rank blocks take detailed spatial structures at multiple scales. It formulates the low-rank matrix decomposition problem into minimizing the ranks of all block matrices and adopts the alternative direction of the multiplier method to optimize it. Experiments on Indian Pines and Pavia University data sets show that the MSLRD can greatly improve the classification performance of regular classification on spectral features (i.e., all bands) and perform better than five state-of-the-art spatial feature extraction methods.

We propose a new method for retrieving 3-D deformation maps by combining the single Interferometric Synthetic Aperture Radar (InSAR)-line of sight (LOS) data and the physical property of crustal deformation, where the spatial coherence of the crustal deformation direction in elastic senses caused by fault slip is considered. First, fault slip and geometry parameters are inverted based on a simplified elastic dislocation model using the InSAR-LOS data. The continuous 3-D surface deformation direction vectors are then estimated by the model parameters. Finally, with the constraints of direction, InSAR-LOS is transformed into the 3-D components, and the 3-D coseismic deformation field is obtained. We implement the L-band SAR data from Advanced Land Observing Satellite (ALOS)/Phased Array L-band SAR (PALSAR) for the 2008 Wenchuan earthquake to testify our method. For the purpose of comparison, four representative models are constructed, ranging from rough to fine. The experimental results explain that the spatial consistency of our method agrees well with the field investigation and Global Navigation Satellite System (GNSS) observations and shows more application potentials than forward modeling.

With the fast development of remote sensing platforms and sensors technology, change detection with heterogeneous remote sensing images (Hete-CD) has become an attractive topic in recent years and plays a vital role in land cover change detection for responding to natural disaster emergencies when homogeneous images are unavailable. Although Hete-CD has been developed for about three decades, and various related methods have been developed and applied successfully in practice, a systematic and comprehensive review of the current achievements regarding Hete-CD remains lacking. Therefore, in this article, we first present an overview of Hete-CD in terms of the related literature. Second, the major techniques of Hete-CD are reviewed in terms of publicly available datasets, the taxonomy of major techniques, results, performance, and quantitative evaluation. Then, some classical methods are selected for comparison and discussion. Finally, based on the discussion and literature review, challenges, opportunities, and future directions for Hete-CD are concluded. The review aims to provide a “one-stop-shop” understanding of the problems with the categories of existing approaches, open opportunities and challenges, and potential future directions for Hete-CD.

The maintenance of spectral variability between subclass objects and the relationship between hyperspectral (HS) bands have been a fundamental but challenging problem for fusing low spatial resolution (LR) HS and high spatial resolution (HR) multispectral (MS) images. This article presents a locally optimized image segmentation fusion (LOISF) framework for HS super-resolution reconstruction. First, LR HS and HR MS are clustered and segmented, and the label attributes of the segmented objects are identified by the prior information. Then, a novel joint fusion model for different typical ground objects is constructed based on spectral unmixing. The fusion problem is formulated mathematically as a convex optimization of a Frobenius norm, which includes spatial, spectral, and index constraints, with an alternating-directions’ optimization featuring linearization providing the solution. Experimental results demonstrate that the proposed LOISF preserves both spatial details and texture, achieving high spectral fidelity, and yielding significantly improved image quality compared to other state-of-the-art fusion methods.

Deep learning methods have shown considerable potential for hyperspectral image (HSI) classification, which can achieve high accuracy compared with traditional methods. However, they often need a large number of training samples and have a lot of parameters and high computational overhead. To solve these problems, this article proposes new network architecture, LiteDepthwiseNet, for HSI classification. Based on 3-D depthwise convolution, LiteDepthwiseNet can decompose standard convolution into depthwise convolution and pointwise convolution, which can achieve high classification performance with minimal parameters. Moreover, we remove the ReLU layer and batch normalization layer in the original 3-D depthwise convolution, which is likely to improve the overfitting phenomenon of the model on small-sized data sets. In addition, focal loss is used as the loss function to improve the model’s attention on difficult samples and unbalanced data, and its training performance is significantly better than that of cross-entropy loss or balanced cross-entropy loss. Experiment results on five benchmark hyperspectral data sets show that LiteDepthwiseNet achieves state-of-the-art performance with a very small number of parameters and low computational cost.

The monitoring of coastal wetlands is of great importance to the protection of marine and terrestrial ecosystems. However, due to the complex environment, severe vegetation mixture, and difficulty of access, it is impossible to accurately classify coastal wetlands and identify their species with traditional classifiers. Despite the integration of multisource remote sensing data for performance enhancement, there are still challenges with acquiring and exploiting the complementary merits from multisource data. In this article, the depthwise feature interaction network (DFINet) is proposed for wetland classification. A depthwise cross attention module is designed to extract self-correlation and cross correlation from multisource feature pairs. In this way, meaningful complementary information is emphasized for classification. DFINet is optimized by coordinating consistency loss, discrimination loss, and classification loss. Accordingly, DFINet reaches the standard solution-space under the regularity of loss functions, while the spatial consistency and feature discrimination are preserved. Comprehensive experimental results on two hyperspectral and multispectral wetland datasets demonstrate that the proposed DFINet outperforms other competitive methods in terms of overall accuracy.

This article proposes a dual global–local attention network (DGLAnet), which is an end-to-end unsupervised band selection (UBS) method that fully utilizes spatial and spectral information in both global and local aspects. The DGLAnet assumes that BS can be realized using the hyperspectral image (HSI) reconstruction process. First, the DGLAnet implements a dual attention module to obtain spatial–spectral and global–local features to reweight the HSI data. It adopts bi-directional relations to grasp spatial and spectral features from a global perspective. Meanwhile, the DGLAnet extracts local features through max-pooling and mean-pooling and then merges them via the convolution operation. Global–local features are utilized to learn attention to recalibrate the original data, and the reconstruction module is adopted to restore the original image from the reweighted HSI data. Finally, a proper band subset is selected by the constructed band evaluation index. Experiments on three hyperspectral data show that the DGLAnet outperforms other state-of-the-art methods and uses all bands with a lower computational cost.

Recent studies have shown that deep domain adaptation (DA) techniques have good performance on cross-domain hyperspectral image (HSI) classification problems. However, most existing deep HSI DA approaches directly use deep networks to extract features from the data, which ignores the detailed information of HSI in spectral and spatial dimensions. To effectively exploit the spectral–spatial joint information for DA of HSIs, we propose a two-branch attention adversarial DA (TAADA) network in this article. In the TAADA network, a two-branch feature extraction (TBFE) subnetwork is first designed as a generator to extract the attention-based spectral–spatial features. Then, a discriminator based on two classifiers with the multilayer FC-BN-ReLU-Dropout structure is constructed. Based on adversarial learning between the generator and the discriminator, the ability of discriminative feature extraction and cross-domain classification is improved simultaneously. Finally, the TAADA network can adjust the distribution between the source and target domains and extract domain-invariant features. Experimental results on three cross-scene HSI classification tasks show that our proposed TAADA outperforms some existing DA methods.

With the development of hyperspectral imaging technology, hyperspectral images (HSIs) have become important when analyzing the class of ground objects. In recent years, benefiting from the massive labeled data, deep learning has achieved a series of breakthroughs in many fields of research. However, labeling HSIs requires sufficient domain knowledge and is time-consuming and laborious. Thus, how to apply deep learning effectively to small labeled samples is an important topic of research in HSI classification. To solve this problem, we propose a semisupervised Siamese network that embeds Siamese network into a semisupervised learning scheme. It integrates an autoencoder module and a Siamese network to, respectively, investigate information in a large amount of unlabeled data and rectify it with a limited labeled sample set, which is called 3DAES. First, the autoencoder method is trained on the massive unlabeled data to learn the refinement representation, creating an unsupervised feature. Second, based on this unsupervised feature, limited labeled samples are used to train a Siamese network to rectify the unsupervised feature to improve feature separability among various classes. Furthermore, by training the Siamese network, a random sampling scheme is used to accelerate training and avoid imbalance among various sample classes. Experiments on three benchmark HSI datasets consistently demonstrate the effectiveness and robustness of the proposed 3DAES approach with limited labeled samples. For study replication, the code developed for this study is available athttps://github.com/ShuGuoJ/3DAES.git.

Due to the detailed spectral information through hundreds of narrow spectral bands provided by hyperspectral image (HSI) data, it can be employed to accurately classify diverse materials of interest, which is one of the core applications of hyperspectral remote sensing technology. In recent years, with the rapid development of deep learning, convolutional neural networks (CNNs) have been successfully applied in many fields, including HSI classification. However, the random gradient descent-based parameter updating scheme is too general and leading to the inefficiency of CNN models. Moreover, the high dimensionality and limited training samples of HSI data also exacerbate the overfitting problem. To tackle these issues, in this article, a novel deep network with multilayer and multibranch architecture, named 3-D Gabor CNN (3DG-CNN), is proposed for HSI classification. More precisely, since the predefined 3-D Gabor filters in multiple scales and orientations could well characterize the internal spatial–spectral structure of HSI data from various perspectives, the 3-D Gabor-modulated kernels (3-D GMKs) are employed to replace the random initialization kernels. Moreover, the specially designed multibranch architecture enables the network to better integrating the scalable property of 3-D Gabor filters; thus, the representative ability and robustness of the extracted features can be greatly improved. Alternatively, the number of network parameters is substantially reduced due to the incorporation of 3-D Gabor modulation, relieving the training complexity and also alleviating the training process from overfitting. Experimental results on four real HSI datasets (including two newly released ones in the literature) have demonstrated that the proposed 3DG-CNN model can achieve better performance than several widely used machine-learning-based and deep-learning-based approaches. For the sake of reproducibility, the codes of the proposed 3DG-CNN model are available athttp://jiasen.tech/papers/.

Nowadays, convolutional neural networks (CNNs) are widely used in the field of hyperspectral image (HSI) classification. However, a major feature of HSIs is their rich spectral–spatial information with hundreds of continuous bands. This inevitably incurs the problems of high computational cost for network optimization and high interredundancy in the convolution kernels. To solve these problems, in this article, we rethink HSIs from the spectral perspective and introduce a lightweight operator called involution, which can effectively solve the above limitations. Different from traditional convolution kernels, the involution kernels pay more attention to the features of the channels but usually ignore the spatial features in the receptive field. To incorporate both spatial and spectral information, we construct a dual-pooling layer and design a novel involution-2D operator and its more lightweight version, involution-1D operator. Finally, an efficient spatial–spectral interaction network (ESSINet) for HSI classification is proposed based on these two new operators, which can make the spatial–spectral information in HSIs interact more closely. Extensive experimental results on four public datasets demonstrate the effectiveness and efficiency of the proposed ESSINet over some state-of-the-art CNN-based networks.

Land cover change detection (LCCD) using remote sensing images (RSIs) plays an important role in natural disaster evaluation, forest deformation monitoring, and wildfire destruction detection. However, bitemporal images are usually acquired at different atmospheric conditions, such as sun height and soil moisture, which usually cause pseudo and noise change into the change detection map. Changed areas on the ground also generally have various shapes and sizes, consequently making the utilization of spatial contextual information a challenging task. In this paper, we design a novel neural network with spatial-spectral attention mechanism and multi-scale dilation convolution modules. This work is based on the previously demonstrated promising performance of convolutional neural network for LCCD with RSIs and attempts to capture more positive changes and further enhance the detection accuracies. The learning of the proposed neural network is guided with a change magnitude image. The performance and feasibility of the proposed network are validated with four pairs of RSIs that depict real land cover change events on the Earth’s surface. Comparison of the performance of the proposed approach with that of five state-of-art methods indicates the superiority of the proposed network in terms of 10 quantitative evaluation metrics and visual performance. Such as, the proposed network achieved an improvement about 0.08%~14.87% in terms of OA for Dataset-A.

Landslide inventory mapping (LIM) is an important application in remote sensing for assisting in the relief of landslide geohazards. However, while conducting LIM tasks performing change detection analysis using bi-temporal very high-resolution (VHR) remote sensing images, due to landslide usually occurred in a mountain area, the phenological difference and outcrop rock may bring pseudo-changes to LIM results. In this paper, a novel change detection approach based on Adaptive Region Shape Similarity (ARSS) is proposed for LIM with VHR remote sensing images to improve detection performance. First, an adaptive region around each pixel is extended to explore the contextual information. Then, direction lines within an adaptive region are defined to describe the shape of the adaptive region. Finally, the pixels located on each direction line are taken into account to build the corresponding histogram. The shape similarity between the pairwise histogram curves is measured by using the Discrete Frchet Distance (DFD). Once the bi-temporal images are processed by using the abovementioned steps, a change magnitude image (CMI) is generated, while a threshold is then used to obtain a final binary change map. The proposed approach is applied to three pairs of landslide sites images acquired with aerial plane and one land use change dataset acquired by Quick Bird Satellite. Compared with ten state-of-the-art methods, the proposed approach achieved LIMs and detection results with higher accuracies and better performance.

Pansharpening methods have been developed for nearly 40 years; however, how to quantitatively evaluate the quality of pansharpened images at full resolution (FR) is probably the most debated topic in this field due to the inherent unavailable of the real HR MS reference image. In this article, a novel blind FR quality evaluation method for pansharpening is proposed. In the proposed method, spatial and spectral features that are sensitive to spatial and spectral distortions of fused images are comprehensively considered and jointly learned based on online multivariate Gaussian (MVG) to construct the evaluation model. It directly outputs the quality of fused images, rather than the stepwise evaluation of spectral score, spatial score, and final overall quality score by the weighted combination of them, which may introduce contradictory results. First, a pristine benchmark evaluation model is established on the spatial features from the original high-spatial-resolution (HR) panchromatic (PAN) image and the spectral invariant assumption between ideal fused and original multispectral (MS) images. Second, a testing evaluation model for the fused image is founded. Finally, the quality of the fused image is measured based on the distance between the testing and benchmark models. The experimental results demonstrated the superior performance of the proposed method. Furthermore, the proposed method can be generalized to other interesting tasks, such as the nonreference evaluation for pansharpening with missing information and the nonreference evaluation for hyperspectral image fusion. The source code is available onhttps://github.com/yyxhpkq/MQNR.

The nonoverlapped spectrum range between low spatial resolution (LR) hyperspectral (HS) and high spatial resolution (HR) multispectral (MS) images has been a fundamental but challenging problem for MS/HS fusion. The spectrum of HS data is generally 400–2500 nm, and the spectrum of MS data is generally 400–900 nm; how to obtain the high-fidelity HR HS fused image within the whole spectrum of 400–2500 nm? In this article, we proposed a band divide-and-conquer framework (BDCF) to solve the problem, by comprehensively considering spectral fidelity, spatial enhancement, and computational efficiency. First, the spectral bands of HS were divided into overlapped and nonoverlapped bands according to the spectral response between HS and MS. Then, a novel improved component substitution (CS)-based method by combing neural network was proposed to fuse the overlapped bands of LR HS. Then, a mapping-based method with the neural network was presented to construct the complicated nonlinear relationship between overlapped and nonoverlapped bands of the original LR HS data. The trained network was mapped to the fused overlapped HR HS bands to estimate the nonoverlapped HR HS bands. Experimental results on two simulated data sets and two realistic data sets of Gaofen (GF)-5 LR HS, GF-1 MS, and Sentinel-2A MS show that the proposed BDCF has superior performance in both high spectral fidelity and sharp spatial details, and it obtained competitive fusion behaviors compared with other state-of-the-art methods. Moreover, BDCF has relatively higher computational efficiency than optimal solution-based methods and deep learning-based fusion methods.

Fusing low spatial resolution (LR) hyperspectral (HS) data and high spatial resolution (HR) multispectral (MS) data aims to obtain HR HS data. However, due to bad weather and the aging of sensor equipment, HS images usually contain a lot of noise, e.g., Gaussian noise, strip noise, and mixed noise, which would make the fused image have low quality. To solve this problem, we propose the multiscale low-rank deep back projection fusion network (MLR-DBPFN). First, HS and MS are superimposed, and multiscale spectral features of the stacked image are extracted through multiscale low-rank decomposition and convolution operation, which effectively removes noisy spectral features. Second, the upsampling and downsampling network mechanisms are used to extract the multiscale spatial features from each layer of spectral features. Finally, the multiscale spectral features and multiscale spatial features are combined for network training, and the weight of the noisy spectrum features is reduced through the network feedback mechanism, which suppresses the noisy spectrum and improves the noisy HS fusion performance. Experimental results on datasets of different noise demonstrate that MLR-DBPFN has superior spatial and spectral fidelity, comparative fusion quality, and robust antinoise performance compared with state-of-the-art methods.

This article proposes a multiscale spectral features graph fusion (MSFGF) method for selecting proper hyperspectral bands. The MSFGF regards that the selected bands should reflect diagnostic spectral information of ground objects at different scales, and it explores band selection from the aspect of multiple spatial scales. First, it adopts the multiscale low-rank decomposition (MSLRD) model to find multiscale spectral features of different ground objects. The model considers divergent spatial structures or spatial correlations of ground objects at different scales, and factorizes the hyperspectral data cube into a series of low-rank block-wise data cubes, where the blocks take spatial structures of different ground objects at increasing scales. Second, the MSFGF presents the multiscale sparse spectral clustering (MSSC) model to fuse the separate connected graphs of multiscale spectral features into a consensus graph. The consensus graph combines the complementary information of multiscale spectral features and helps to reveal the intrinsic clustering structure of all spectral bands. Finally, the MSFGF utilizes spectral clustering to find clusters from the consensus graph and selects representative bands. Experimental results on three widely used hyperspectral data prove the superiority of MSFGF in selecting bands, where it outperforms other seven state-of-the-art methods in classification with an acceptable computational cost.

In the above article[1],Fig. 19was incorrectly placed. The correct image and caption are provided here:

Image fusion effectively solves the trade-off between spatial resolution, temporal resolution, and spectral resolution of remote sensing sensors. However, most of existing methods focus on the fusion of two of the spatial, temporal, and spectral metrics of remote sensing images. The few spatial-temporal-spectral fusion (STSF) methods available are mainly for fusing MODIS and Landsat images, which are not suitable for the characteristics of the spaceborne hyperspectral images with low temporal resolution, such as Hyperion, ZY-1 02D, and PRISMA. For this purpose, we proposed a novel generalized linear spectral mixing model for spatial-temporal-spectral fusion (GLMM-STSF). In the method, the GLMM is introduced into the STSF problem, and the temporal variations of images at different times are transferred to the endmember and abundance matrix variations of images for estimation. To the best of our knowledge, for the first time, the STSF task of remote sensing images is handled from the perspective of spectral unmixing. Compared with existing STSF fusion methods, our method targets the task of fusing spaceborne HSI with low temporal and spatial resolutions with multispectral image featured by high temporal and spatial resolutions. Taking the STSF of ZY-1 02D hyperspectral and Sentinel-2 multispectral real datasets as an example, comparisons with related state-of-the-art methods demonstrate that our proposed method achieves superior fusion performance.

The presence of mixed pixels in the hyperspectral data makes unmixing to be a key step for many applications. Unsupervised unmixing needs to estimate the number of endmembers, their spectral signatures, and their abundances at each pixel. Since both endmember and abundance matrices are unknown, unsupervised unmixing can be considered as a blind source separation problem and can be solved by nonnegative matrix factorization (NMF). However, most of the existing NMF unmixing methods use a least-squares objective function that is sensitive to the noise and outliers. To deal with different types of noises in hyperspectral data, such as the noise in different bands (band noise), the noise in different pixels (pixel noise), and the noise in different elements of hyperspectral data matrix (element noise), we propose three self-paced learning based NMF (SpNMF) unmixing models in this article. The SpNMF models replace the least-squares loss in the standard NMF model with weighted least-squares losses and adopt a self-paced learning (SPL) strategy to learn the weights adaptively. In each iteration of SPL, atoms (bands or pixels or elements) with weight zero are considered as complex atoms and are excluded, while atoms with nonzero weights are considered as easy atoms and are included in the current unmixing model. By gradually enlarging the size of the current model set, SpNMF can select atoms from easy to complex. Usually, noisy or outlying atoms are complex atoms that are excluded from the unmixing model. Thus, SpNMF models are robust to noise and outliers. Experimental results on the simulated and two real hyperspectral data sets demonstrate that our proposed SpNMF methods are more accurate and robust than the existing NMF methods, especially in the case of heavy noise.

Recently, multiscale spatial features have been widely utilized to improve the hyperspectral image (HSI) classification performance. However, fixed-size neighborhood involving the contextual information probably leads to misclassifications, especially for the boundary pixels. Additionally, it has been demonstrated that deep neural network (DNN) is practical to extract representative features for the classification tasks. Nevertheless, under the condition of high dimensionality versus small sample sizes, DNN tends to be over-fitting and it is generally time-consuming due to the deep-level feature learning process. To alleviate the aforementioned issues, we propose a multiscale context-aware ensemble deep kernel extreme learning machine (MSC-EDKELM) for efficient HSI classification. First, the scene of the HSI data set is over-segmented in multiscale via using the adaptive superpixel segmentation technique. Second, superpixel pattern (SP) and attentional neighboring superpixel pattern (ANSP) are generated by leveraging the superpixel maps, which can automatically comprise local and global contextual information, respectively. Afterward, an ensemble deep kernel extreme learning machine (EDKELM) is presented to investigate the deep-level characteristics in the SP and ANSP. Finally, the category of each pixel is accurately determined by the decision fusion and weighted output layer fusion strategy. Experimental results on four real-world HSI data sets demonstrate that the proposed frameworks outperform some classic and state-of-the-art methods with high computational efficiency, which can be employed to serve real-time applications.

Classifying different land cover types with hyperspectral image (HSI) is significant for restoring and protecting natural resources and maintaining ecological services in coastal wetlands. This paper proposes a multi-domain features fusion convolutional neural network (MDF-CNN) based classification method for hyperspectral images of coastal wetlands. This method adopts inter-class sparsity based discriminative least square regression (ICS_DLSR) to learn a more compact and discriminative transformation, as well as fuse the high-level features of the original domain and the regression domain to obtain higher classification accuracy. Experimental results demonstrate the effectiveness of the proposed method when compared with some recent classifiers. The MDF-CNN achieved state-of-the-art performance on two latest GF-5 HSI datasets of Coastal Wetland.

Non-negative matrix factorization (NMF) is a classical hyperspectral unmixing model which minimizes the Euclidean distance between the hyperspectral data matrix and its low rank approximation (i.e., the product of endmember matrix and abundance matrix), and it fails when applied to noisy data because the loss function is sensitive to outliers. In this paper, we propose a Cauchy NMF (CauchyNMF) model for hyperspectral unmixing which uses a Cauchy loss function (CLF) to replace the traditional least-squares loss. Compared with the least-squares loss, CLF can penalize the noise term for suppressing the large noise mixed in the real data and thus is much more robust. Experimental results on simulated and real hyperspectral data sets demonstrate that our proposed CauchyNMF method is more accurate and robust than existing NMF methods, especially in the case of heavy noise.

This letter presents a correntropy-based sparse spectral clustering (CSSC) method to select proper bands of a hyperspectral image. The CSSC first constructs an affinity matrix with the correntropy measure which considers the nonlinear characteristics of hyperspectral bands and can suppress effects from noise or outliers in measuring band similarity. The CSSC imposes the sparsity and block diagonal constraint on spectral clustering, which can further improve band clustering performance. Bands are finally selected from each cluster on the connected graph. Experimental results on two widely used hyperspectral images show that the CSSC behaves better than spectral clustering and other several state-of-the-art methods in band selection.

This letter proposes a lateral-slice sparse tensor robust principal component analysis (LSSTRPCA) method to remove gross errors or outliers from hyperspectral images so as to promote the performance of subsequent classification. The LSSTRPCA assumes that a three-order hyperspectral tensor has a low-rank structure, and gross errors or outliers are sparsely scattered in a 2-D space (i.e., lateral-slice) of the tensor. It formulates a low-rank and sparse tensor decomposition problem into a convex problem and then implements the inexact augmented Lagrange multiplier method to solve it. The experiments on two hyperspectral data sets show that the LSSTRPCA can successfully remove outliers or gross errors and achieve higher accuracies than both the original robust principal component analysis (RPCA) and tensor robust principal component analysis (TRPCA).

This article presents a fast and latent low-rank subspace clustering (FLLRSC) method to select hyperspectral bands. The FLLRSC assumes that all the bands are sampled from a union of latent low-rank independent subspaces and formulates the self-representation property of all bands into a latent low-rank representation (LLRR) model. The assumption ensures sufficient sampling bands in representing low-rank subspaces of all bands and improves robustness to noise. The FLLRSC first implements the Hadamard random projections to reduce spatial dimensionality and lower the computational cost. It then adopts the inexact augmented Lagrange multiplier algorithm to optimize the LLRR program and estimates sparse coefficients of all the projected bands. After that, it employs a correntropy metric to measure the similarity between pairwise bands and constructs an affinity matrix based on sparse representation. The correntropy metric could better describe the nonlinear characteristics of hyperspectral bands and enhance the block-diagonal structure of the similarity matrix for correctly clustering all subspaces. The FLLRSC conducts spectral clustering on the connected graph denoted by the affinity matrix. The bands that are closest to their separate cluster centroids form the final band subset. Experimental results on three widely used hyperspectral data sets show that the FLLRSC performs better than the classical low-rank representation methods with higher classification accuracy at a low computational cost.

This paper presents a robust multi-feature spectral clustering (RMSC) method for hyperspectral band selection. The RMSC combines four features of hyperspectral bands and formulates a low-rank and sparse decomposition program to estimate the integrated similarity matrix for spectral clustering. The integrated similarity matrix is assumed to represent the total similarity information of band information entropy, band correlation and band divergence and alleviate negative effects from noise or outliers in hyperspectral images. After that, spectral clustering is implemented on the integrated similarity matrix to select appropriate bands. Experimental results on the Indian Pines datasets show that the RMSC could greatly improve the classification accuracy of spectral clustering and meanwhile outperform state-of-the-art band selection methods.

This paper aims to compare the performance of GaoFen-1 (GF-1), GaoFen-5 (GF-5), Landsat-8 data in fine classification. An optimized sample selection method (OSSM) is developed to ensure the high quality of samples. This method adopts different band combination strategies to realize optimal selection of training samples under the aid of normalized vegetation index (NDVI), normalized water index (NDWI) and the components of Kauth-Thomas (KT) Transformation. After that, support vector machine (SVM) is implemented on these three types of data. Experimental results on China Dunhuang calibration field, Gansu Ying-mao-tuo exploration area and Gan River lower reaches datasets show that GF-5 data performs best in both qualitative and quantitative evaluation of fine classification thanks to its hyperspectral properties.

This paper proposes an unsupervised domain adaptation (DA) method called correlation alignment based on sparse matrix transform (CORAL-SMT) for hyperspectral image (HSI) classification. In CORAL-SMT, the covariance of source and target domain are constrained to have an eigen-decomposition that can be represented as a sparse matrix transform. Under maximum likelihood framework, based on greedy minimization strategy, the covariances can be efficiently estimated and are always positive definite. The proposed method is compared with some classical unsupervised domain adaptation methods. Experimental results on the City of Pavia hyperspectral data set demonstrate the effectiveness of CORAL-SMT.

A clear bias exists in the widely used gridded precipitation products (GPPs) that result from factors about topography, climate, and retrieval algorithms. Many existing optimization works have a deficiency in validation and domain sizes, which makes the evaluation and corrections of the Climate Hazards group InfraRed Precipitation with Station data (CHIRPS) product still challenging. In this letter, we propose a bias-correction approach that combines coarser-resolution gauge-based precipitation with a probability distribution function (PDF) to improve the accuracy of CHIRPS. The data from 27 local precipitation gauges in Shanghai are utilized to testify the performance of our method. Results explain that daily corrected CHIRPS (Cor-CHIRPS) product has higher accuracy than CHIRPS compared with ground truths (GrTs) in terms of both error statistics and detection capability, particularly in spring, autumn, and winter. Moreover, Cor-CHIRPS better captures the frequencies of precipitation events and well depicts the spatial characteristics of the annual precipitation.

Domain adaptation, which aims at learning an accurate classifier for a new domain (target domain) using labeled information from an old domain (source domain), has shown promising value in remote sensing fields yet still been a challenging problem. In this letter, we focus on knowledge transfer between hyperspectral remotely sensed images in the context of land-cover classification under unsupervised setting where labeled samples are available only for the source image. Specifically, a discriminative transfer joint matching (DTJM) method is proposed, which matches source and target features in the kernel principal component analysis space by minimizing the empirical maximum mean discrepancy, performs instance reweighting by imposing an ℓ2,1-norm on the embedding matrix, and preserves the local manifold structure of data from different domains and meanwhile maximizes the dependence between the embedding and labels. The proposed approach is compared with some state-of-the-art feature extraction techniques with and without using label information of source data. Experimental results on two benchmark hypersepctral data sets show the effectiveness of the proposed DTJM.

The optical remote sensing images not only have to make a fundamental tradeoff between the spatial and spectral resolutions, but also are inevitable to be polluted by the clouds; however, the existing pansharpening methods mainly focus on the resolution enhancement of the optical remote sensing images without cloud contamination. How to fuse the cloud-contaminated images to achieve the joint resolution enhancement and cloud removal is a promising and challenging work. In this paper, a pansharpening method for the challenging cloud-contaminated very high-resolution remote sensing images is proposed. Furthermore, the cloud-contaminated conditions for the practical observations with all the thick clouds, the thin clouds, the haze, and the cloud shadows are comprehensively considered. In the proposed methods, a two-step fusion framework based on multisource and multitemporal observations is presented: 1) the thin clouds, the haze, and the light cloud shadows are proposed to be first jointly removed and 2) a variational-based integrated fusion model is then proposed to achieve the joint resolution enhancement and missing information reconstruction for the thick clouds and dark cloud shadows. Through the proposed fusion method, a promising cloud-free fused image with both high spatial and high spectral resolutions can be obtained. To comprehensively test and verify the proposed method, the experiments were implemented based on both the cloud-free and cloud-contaminated images, and a number of different remote sensing satellites including the IKONOS, the QuickBird, the Jilin (JL)-1, and the Deimos-2 images were utilized. The experimental results confirm the effectiveness of the proposed method.

In this paper, a self-paced joint sparse representation (SPJSR) model is proposed for the classification of hyperspectral images (HSIs). It replaces the least-squares (LS) loss in the standard joint sparse representation (JSR) model with a weighted LS loss and adopts a self-paced learning (SPL) strategy to learn the weights for neighboring pixels. Rather than predefining a weight vector in the existing weighted JSR methods, both the weight and sparse representation (SR) coefficient associated with neighboring pixels are optimized by an alternating iterative strategy. According to the nature of SPL, in each iteration, neighboring pixels with nonzero weights (i.e., easy pixels) are included for the joint SR of a testing pixel. With the increase of iterations, the model size (i.e., the number of selected neighboring pixels) is enlarged and more neighboring pixels from easy to complex are gradually added into the JSR learning process. After several iterations, the algorithm can be terminated to produce a desirable model that includes easy homogeneous pixels and excludes complex inhomogeneous pixels. Experimental results on two benchmark hyperspectral data sets demonstrate that our proposed SPJSR is more accurate and robust than existing JSR methods, especially in the case of heavy noise.

This paper proposes a compressed columnwise robust principal component analysis (CCRPCA) method for hyperspectral anomaly detection. The CCRPCA improves the regular RPCA by using the Hadamard random projection and constraining the columnwise structure of sparse anomaly matrix. The Hadamard random projection reduces the computational cost of the hyperspectral data, and the columnwise sparse structure alleviates negative effects from the anomalies on the columns of the background. The sparse anomaly matrix and the background matrix are estimated by optimizing a convex program, and the anomalies are estimated from nonzero columns of the compressed sparse matrix. Preliminary experiment result from the San Diego dataset shows that the CCRPCA outperforms four state-of-the-art detection methods in both the receiver operating characteristic curve and the area under curve.

A fast and robust principal component analysis on Laplacian graph (FRPCALG) method is proposed to select bands of hyperspectral imagery (HSI). The FRPCALG assumes that a clean band matrix lies in a unified manifold subspace with low-rank and clustering properties, whereas sparse noise does not lie in the same subspace. It estimates the clean lowrank approximation of the original HSI band matrix while uncovering the clustering structure of all bands. Specifically, a structured random projection is adopted to reduce the high spatial dimensionality of the original data for computational cost saving, and then a Laplacian graph (LG) term is regularized into the regular robust principal component analysis (RPCA) to formulate the FRPCALG model for the submatrix of bands to be selected. The RPCA term ensures the clean and low-rank approximation of original data, and the LG term guarantees the clustering quality of a low-rank matrix in the low-dimensional manifold subspace. The alternating direction method of multipliers' algorithm is utilized to optimize the convex program of the FRPCALG. The K-means algorithm is to group all columns of submatrix into clusters, and corresponding bands closest to their cluster centroids finally constitute the desired band subset. Experimental results show that FRPCALG outperforms state-ofthe-art methods with lower computational cost. A moderate regularization parameter λ and a small μ could guarantee satisfying the classification accuracy of FRPCALG, and a small projected dimension greatly reduces the computational cost and does not affect the classification performance. Therefore, the FRPCALG can be an alternative method for hyperspectral band selection.

The land surface reflectance data are indispensable to generate many other land products. Global land surface reflectance data have been routinely produced from remote sensing sensors aboard different satellite platforms. However, the original data, especially the daily data, suffer from a large number of spatial gaps, which result from atmospheric contamination and instrument deficiencies. This seriously limits their further applications. Many composite products with less spatial gaps have been generated to solve the above problem, but they easily sacrifice their temporal resolutions of original data. Even worse, they cannot be directly implemented in realistic applications because of the noise and composite seams. This paper proposes a temporal-spatial reconstruction method (TSRM) to generate daily gapless and seamless land surface reflectance data. The TSRM integrates both temporal and spatial information for recovering different land cover types using three processing steps. First, spatial gaps are coarsely filled with multiyear weighted average (Step1). After that, all the gaps that are not filled in the first step are interpolated by using harmonic analysis of time series with true value constraint (Step2). Finally, the reconstructed results in the last step are seamlessly processed using the Poisson image editing method, and the seamless daily reflectance data set is generated (Step3). The Moderate Resolution Imaging Spectroradiometer reflectance data set (MOD09GA and MYD09GA) on two testing areas is selected to verify the performance of the proposed TSRM. Experimental results show that the TSRM has good performance with regard to maintaining the temporal and spatial integrity of the daily land surface reflectance data. Results on different testing sites also demonstrate that the TSRM preserves spectral integrity with clear seasonal trends for each spectral band.

A band-weighted support vector machine (BWSVM) method is proposed to classify hyperspectral imagery (HSI). The BWSVM presents an L1penalty term of band weight vector to regularize the regular SVM model. The L1norm regularization term guarantees the sparsity of band weights and describes potentially divergent contributions from different bands in modeling the binary SVM model. The BWSVM adopts the KerNel iterative feature extraction algorithm to minimize the nonconvex program. It linearizes nonlinear kernels and iteratively optimizes two convex subproblems with respect to both sample coefficients and band weights. The class label is determined by picking the largest sample coefficients from all its binary models of BWSVM. Two popular HSI data sets are utilized to testify the classification performance of BWSVM. Experimental results show that the BWSVM outperforms three state-of-the-art classifiers including SVM, random forest, and k-nearest neighbor.

Extracting subpixel land-cover change detection (SLCCD) information is important when multitemporal remotely sensed images with different resolutions are available. The general steps are as follows. First, soft classification is applied to a low-resolution (LR) image to generate the proportion of each class. Second, the proportion differences are produced by the use of another high-resolution (HR) image and used as the input of subpixel mapping. Finally, a subpixel sharpened difference map can be generated. However, the prior HR land-cover map is only used to compare with the enhanced map of LR image for change detection, which leads to a nonideal SLCCD result. In this letter, we present a new approach based on a back-propagation neural network (BPNN) with a HR map (BPNN_HRM), in which a supervised model is introduced into SLCCD for the first time. The known information of the HR land-cover map is adequately employed to train the BPNN, whether it predates or postdates the LR image, so that a subpixel change detection map can be effectively generated. In order to evaluate the performance of the proposed algorithm, it was compared with four state-of-the-art methods. The experimental results confirm that the BPNN_HRM method outperforms the other traditional methods in providing a more detailed map for change detection.

A sparse and low-rank near-isometric linear embedding (SLRNILE) method has been proposed to make dimensionality reduction and extract proper features for hyperspectral imagery (HSI) classification. The SLRNILE stands on the theory of the John-Lindenstrauss lemma, and tries to estimate a sparse and low-rank projection matrix that satisfies the restricted isometric property (RIP) condition on all secants of the HSI data. The RIP condition guarantees that the desired linear mapping near-isometrically preserves nearest neighbor points of all HSI pixels. Seeking the desired mapping is then modeled into minimizing a Lagrange multipliers formulation. The alternating direction method of multipliers framework is utilized to solve the above convex program, and column generation techniques are adopted to alleviate the computation memory burden during the optimization procedure. Five experiments on three widely used HSI data sets are designed to completely test the performance of SLRNILE, and experimental results are compared against those of six state-of-the-art feature extraction methods, including principal component analysis, Laplacian eigenmaps, locality preserving projections, neighborhood preserving embedding, sparse nonnegative matrix underapproximation, and random projections. The results show that SLRNILE performs best among all the seven methods, and its computational time is longest of all but still bearable for regular users. Therefore, the SLRNILE can be a good choice for feature extraction in HSI classification.

Remote sensing technique constructs a regression model to describe the relations between Aerosol optical depth (AOD) and the ground PM2.5concentration. However, potential effects from meteorological and geographical factors on the regression models have never been carefully investigated. The manuscript selects three main meteorological variables and two major geographical variables, and investigates their impacts in the performance of geographical weighted regression (GWR) and ordinary least square (OLS) models for estimating ground PM2.5concentration. Preliminary results on the case of Yangtze River Delta show that meteorological factors have more significant influence on the estimation of PM2.5concentration than geographical factors. Moreover, the observations tell that the GWR is more preferably to estimate PM2.5concentration than the OLS.

This manuscript proposes a symmetric sparse representation (SSR) method to extract pure endmembers from Hyperspectral imagery (HSI). The SSR assumes that the desired endmembers and all the HSI pixels can be sparsely represented by each other and it formulates the endmember extraction problem into finding archetypes in the minimal convex hull of the HSI data. The optimization program of SSR is solved by a simple projected gradient algorithm and the endmembers are initialized with the vector quantization scheme. Preliminary results on the popular Urban HSI data infer that the SSR performs better than several state-of-the-art methods (VCA, NFINDER, AVMAX, SVMAX, XRAY, OSP and H2NMF).