Synthetic Aperture Radar (SAR) images are crucial for maritime vessel detection; however, challenges such as blurred ship edges, strong land scattering interference, and angular regression mismatches across varying target sizes hinder accurate rotational localization. In this paper, an oriented decoupling target detection method (R-MCLST) is proposed to address these issues. The method integrates three key modules: a multi-channel positioning module (MC-PM) that employs distributed average pooling and additional coordinate channels to enhance orientation awareness; a soft threshold-based multilayer perceptron (ST-MLP) that effectively mitigates background interference while robustly extracting complex features; and a Gaussian distribution-based prediction box (GD-BPB) that transforms rotated bounding box encoding into a two-dimensional Gaussian distribution using KL divergence for adaptive parameter adjustment. Experimental evaluations on the R-SSDD and MR-HRSID datasets demonstrate that R-MCLST achieves superior performance, with the R-SSDD dataset yielding AP50 of 87.48%, AP75 of 34.96%, AP of 41.15%, and AR of 46.52%, and the MR-HRSID dataset yielding AP50 of 61.59%, AP75 of 4.96%, AP of 19.13%, and AR of 22.24%. Comparative analyses confirm that the proposed method outperforms current state-of-the-art networks in accurately localizing rotating targets under challenging SAR imaging conditions.

Ship detection in remote sensing images has been attracting a lot of attention due to its great application value in both military and civilian fields. However, ships in high-resolution remote sensing images are characterized by the remarkable features of multiscale, arbitrary orientation, and dense arrangement, which is a great challenge for fast and accurate target detection. In order to solve problems, we propose a YOLOV5-based oriented ship detection method of remote sensing images with contextual global attention mechanism and lightweight task-specific context decoupling (CGTC-RYOLO) in this article. First, a cross-stage partial context transformer (CSP-COT) module is introduced to capture global contextual spatial relations using multihead self-attention (MHSA) to verify their implications in implicit dependencies. Second, we propose an angle classification prediction branch in the YOLOV5 head network for detecting targets in any direction and design probability and distribution loss function (PrfoIoU) to optimize the regression effect. Third, the lightweight task-specific context decoupling (LTSCODE) for target detection is employed to replace the original head in the YOLOV5 model, which is used to solve the accuracy problem caused by YOLOV5’s hybridization of classification and localization. Ablation experiments demonstrate the importance and effectiveness of each module. Compared with the benchmark model, the CGTC-RYOLO has the 5.9%, 3.7%, and 4.3% mAP improvements on the DOTA-ship dataset, the HRSC2016 dataset, and the UCAS-AOD dataset, respectively. Moreover, the model’s generalization is also validated. Compared with state-of-the-artmethods, the CGTC-RYOLO can achieve better accuracy and fewer parameters.

Spatial-temporal-spectral fusion is dedicated to integrating the complementary advantages of multisource images to obtain fused image with all high spatial, high temporal and high spectral resolutions, which is promising but more challenging. On the one hand, traditional studies deployed on MODIS and Landsat data cannot be transferred to most spaceborne hyperspectral (HS) data with lower temporal resolution; on the other hand, the rigid time relation modeling in most existing studies exhibits weakness orienting to non-linear land-cover changes. In this paper, we propose a dual-task cascaded network for spatial-temporal-spectral fusion, with collaborative modeling on spatialspectral joint enhancement and temporal variation estimation in a unified framework. The spatial-spectral joint enhancement task was designed with an iterative alternating projection, meticulously crafted to address the scale variance among observation. Additionally, the spatial enhancement unit and error correction unit were coupled modeling to enhance the spatial and spectral fidelity. The temporal variation estimation on spectral fine tuning network was developed, to further enhance the temporal and spectral fidelity. Extensive experiments were implemented on Ziyuan(ZY)-1 02D HS data and Sentinel-2 multispectral (MS) data. Both qualitative and quantitative results demonstrated the competitive performance of the proposed method.

The integrated fusion of the high-spatial-resolution (HR) panchromatic image (PAN), the relative “moderate”-spatial-resolution (MR) multispectral image (MSI), and the low-spatial-resolution (LR) hyperspectral image (HSI), to generate the optimal HR HS fused image, is promising but challenging. On the one hand, existing mainstream fusion models mostly focus on the “pairwise fusion” between HR PAN, MR MSI, and LR HSI, which cannot sufficiently integrate their complementary spatial and spectral advantages. On the other hand, one of the few integrated fusion methods roughly introduced the MR MSI as a simple intermediate medium; however, the role of MSIs as a spatial and spectral “bridge” between HR PAN and LR HSI, generally characterized by significant scale differences, remains largely unexplored. To solve these problems, we proposed an integrated PAN–MSI–HSI fusion method from the perspective of MSIs, by comprehensively considering the scale difference among the multisource observations. In the proposed method, a spatial–spectral feature transfer network was designed by comprehensively exploring the spatial–spectral variations and connections among the HR PAN, MR MSI, and LR HSI. Then, a spatial–spectral joint reconstruction module was constructed to reconstruct the HR HSI with optimal spatial and spectral fidelity. Experiments were conducted on simulated and real datasets from qualitative and quantitative aspects. The experimental results demonstrated the competitive effectiveness over other state-of-the-art methods.

Spectral super-resolution (SSR) is the computational process of generating a high-dimensional hyperspectral image from a low-dimensional image through spectral reconstruction techniques. Recently, deep learning has demonstrated remarkable potential in the field of SSR, achieving impressive results. However, existing deep learning-based approaches often fail to deliver high-fidelity SSR outcomes. These methods tend to focus primarily on spectral information while paying insufficient attention to the critical role of spatial features. Furthermore, they lack effective strategies for capturing inter-band relationships, resulting in suboptimal spectral information modeling. To address these limitations, we propose a novel network for SSR, termed Bidirectional Spectral Attention Multi-Scale Aggregation Network (BiSANet). BiSANet features three U-Net-like branches and integrates two advanced attention mechanisms. The bidirectional spectral attention modules dynamically model inter-spectral dependencies through forward and reverse spectral feature extraction, enhanced by a weight-sharing strategy. Specifically, we reverse the spectral order of feature maps to activate complementary global trends and local details, overcoming the limitations of unidirectional modeling in traditional methods. Additionally, an independent spatial reconstruction branch with a dedicated loss function ensures precise spatial detail preservation. Experimental results demonstrate that BiSANet outperforms state-of-the-art methods across three benchmarks. For instance, on the DFC2018 Houston dataset, it achieves a 4.26% PSNR improvement and an 11.52% SAM reduction, highlighting its robustness and accuracy in spectral-spatial reconstruction.

High-resolution satellite imagery with dense temporal series is crucial for long-term surface change monitoring. Spatiotemporal fusion seeks to reconstruct remote sensing image sequences with both high spatial and temporal resolutions by leveraging prior information from multiple satellite platforms. However, significant radiometric discrepancies and large spatial resolution variations between images acquired from different satellite sensors, coupled with the limited availability of prior data, present major challenges to accurately reconstructing missing data using existing methods. To address these challenges, this paper introduces GCM-PDA, a novel generative compensation model with progressive difference attenuation for spatiotemporal fusion of remote sensing images. The proposed model integrates multi-scale image decomposition within a progressive fusion framework, enabling the efficient extraction and integration of information across scales. Additionally, GCM-PDA employs domain adaptation techniques to mitigate radiometric inconsistencies between heterogeneous images. Notably, this study pioneers the use of style transformation in spatiotemporal fusion to achieve spatial-spectral compensation, effectively overcoming the constraints of limited prior image information. Experimental results demonstrate that GCM-PDA not only achieves competitive fusion performance but also exhibits strong robustness across diverse conditions.

The escalating surface ozone (O3) pollution in urban areas throughout China has raised significant concerns due to its detrimental impacts on public health, local environment and agriculture. Despite of numerous efforts in surface O3estimation, intricate geographical spatiotemporal interactions of the potential predictors has been largely overlooked. This limitation has significantly constrained the O3estimation accuracy. To address this issue, we proposed a novel deep neural Network, named Geo-STO3Net, to effectively integrate adjacent Geographical SpatioTemporal information from meteorological data and satellite observations into surface O3estimation. The Geo-STO3Net model used a spatial encoder based on the Residual Network, a temporal encoder based on the Transformer, and a feature decoder based on the Deep Neural Networks to comprehensively capture the intricate geographical spatiotemporal dependencies among the predictors. Our model achieved cross-validation (CV) R2value of 0.95, outperforming popular models. The Geo-STO3Net model demonstrated robust spatial and temporal transferability, as evidenced by R2values of 0.94 and 0.82 in external spatial and temporal validation on monthly scales, respectively. The Geo-STO3Net model’s proficiency in handling geographical spatiotemporal information led to substantial performance improvements compared to models lacking this feature, with improved CV R2values ranging from 0.01 to 0.18. Our findings also highlighted the severe O3pollution over the Yangtze River Delta (YRD) region in 2022, with average surface O3concentrations reaching 103.14 μg/m3. These evidences indicate our proposed Geo-STO3Net model can accurately estimate surface O3concentrations, and provide valuable insights into the development of effective control policies.

Cloud cover and long revisit cycle of satellites can cause gaps in optical images and pose a significant obstacle to the consistency of Earth observation missions. Recently, synthetic aperture radar (SAR)-to-optical image translation (S2OIT) has become an emerging approach to reconstruct the missing information of optical remote sensing images. However, the previous studies ignored the mechanism difference between SAR and optical data and produced color distortion, image blurriness, and texture detail loss in the generated optical images. To tackle these challenges, we propose a multitemporal S2OIT network (MTS2ONet) for high-quality optical image generation. The proposed model comprises two subnetworks: change feature extraction subnetwork (Change_Extractor) and the S2OIT subnetwork (S2O_Translator). The first subnetwork is tasked with extracting change features from SAR images captured at dates T and${T} +1$, and then translating them from the SAR domain to the optical domain. Subsequently, the S2O_Translator integrates the optical image at date${T} +1$with the change features extracted by the Change_Extractor to generate the optical image at date T. In addition, we produce a dual-temporal SAR-optical dataset called DTSEN1-2 for model evaluation. Experiments on the DTSEN1-2 dataset reveal that our method is superior to the state-of-the-art (SOTA) methods with the metrics peak-signal-to-noise ratio (PSNR; 36.0435), structural similarity index measure (SSIM; 0.9896), learned perceptual image patch similarity (LPIPS; 0.0443), and root mean square error (RMSE; 0.0174) and exhibits preferable results in visual effects. Our dataset and codes can be accessed via the following link:https://github.com/hopeupup/MTS2ONet.

The objective of change detection (CD) is to identify the altered region between dual-temporal images. In pursuit of more precise change maps, numerous state-of-the-art (SOTA) methods design neural networks with robust discriminative capabilities. The convolutional neural network (CNN)-transformer model is specifically designed to integrate the strengths of the CNN and transformer, facilitating effective coupling of feature information. However, previous CNN-transformer studies have not effectively mitigated the interference of feature distribution differences as well as pseudovariations between two images due to cloud occlusion, imaging conditions, and other factors. In this article, we propose a domain adaptive and interactive differential attention network (DA-IDANet). This model incorporates domain adaptive constraints (DACs) to mitigate the interference of pseudovariations by mapping the two images to the same deep feature space for feature alignment. Furthermore, we designed the interactive differential attention module (IDAM), which effectively improves the feature representation and promotes the coupling of interactive differential discriminant information, thereby minimizing the impact of irrelevant information. Experiments on four datasets demonstrate the superior validity and robustness of our proposed model compared to other SOTA methods, as evident from both quantitative analysis and qualitative comparisons. The code will be available online (https://github.com/Jyl199904/DA-IDANet).

Hyperspectral image (HSI) has fine spectral resolution and abundant spatial information to detect subtle differences between targets. However, it is heavily contaminated with noise due to sensor design and atmospheric radiative transfer, resulting in spectral shifts and spatial discontinuities. Current denoising methods usually establish constraints directly on the ground truth and denoised image, lacking supervision of intermediate parameters of the network, resulting in insufficient model constraints and poor convergence. In addition, existing methods do not consider spatial-spectral compensation, so the denoising results have obvious spatial-spectral distortion. To this end, we propose a novel multiscale spatial-spectral invertible compensation network (MSIC-Net) for HSI denoising. The method constructs an invertible spatial-spectral compensation (ISSC) module, which supervises intermediate features through inverse constraints, realizes the circulation of multiscale information, and improves the stability of the model. At the same time, we also introduce style transfer for spatial-spectral compensation, which uses its superior fine feature control ability to precisely compensate for the lost spatial and spectral detail features. The method is extensively validated experimentally and categorically on simulated and real datasets. The experimental results show that MSIC-Net outperforms other state-of-the-art denoising methods in quantitative and qualitative evaluations.

The integration of hyperspectral (HS) imagery and light detection and ranging (LiDAR) for land cover classification has become a significant research topic. Numerous existing methods aim to interactively fuse the complementary features of HS and LiDAR to enhance the classification accuracy. However, most existing studies overlook the fact that spectral, spatial, and elevation features of HS and LiDAR possess significant discriminative information for specific categories. The rough and simple interacting or stacking these features may hinder the effective expression of this significant discriminative information. Moreover, existing approaches neglect the shared spatial characteristics between HS and LiDAR. In this article, an uncertain category-aware fusion network (UCAFNet) is proposed to tackle the above challenges. Specifically, we proposed an uncertain category-aware fusion strategy (UCAFS) that dynamically weights the spectral, spatial, and elevation branches based on their respective capabilities in identifying different categories to achieve targeted information aggregation. Moreover, we introduce the spatial information purification module (SIPM) and adaptive weighted fusion module (AWFM), to extract and enhance shared spatial features from HS and LiDAR for effective integration. The experimental results on three public benchmark datasets demonstrate the superior performance of the proposed UCAFNet.

The hyperspectral imaging instrument makes a trade-off by sacrificing spatial resolution to achieve high spectral resolution. This compromise leads to a low signal-to-noise ratio, and hyperspectral images (HSIs) are often heavily contaminated with mixed noise, which is an inherent challenge. Previous research has achieved satisfactory results for natural image denoising; hyperspectral denoising has remained a formidable task. In this article, we introduce an innovative method called the multistage hybrid-denoising network for satellite hyperspectral images (SUC-MSDN). SUC-MSDN initially decomposes the noisy HSI into multiple scales and constructs a multistage denoising network by analyzing the spatial spectrum texture distribution characteristics of noise signals. Instead of simply stacking the output results from each scale, SUC-MSDN uses the denoising results from the low-scale network as prior knowledge for the high-scale denoising network to more accurately remove the final noise components. Extensive experimental datasets are used to validate the performance of SUC-MSDN. Experimental results show that SUC-MSDN outperforms benchmark methods and significantly enhances the accuracy of land cover mapping.

This paper proposes a novel deep learning-based framework for hyperspectral band selection, named Cross-Scene Self-Representative Network (CSSRnet). The proposed method leverages the rich labels of the source domain (SD) to guide the band selection in the target domain (TD). To our knowledge, CSSRnet is the first deep learning-based solution for cross-scene hyperspectral band selection. First, the CSSRnet employs contextual attention mechanism to capture the latent features of SD and TD. It combines the self-attention mechanism with convolutional operations to capture static and dynamic contextual information. Then, the self-representative layer provides the self-representative coefficient of SD and TD. Subsequently, the maximum mean difference is utilized to align the self-representative coefficients of both SD and TD. To enhance the representativeness and precision of these coefficients, we introduce different tasks for the SD and TD branches. Finally, a suitable band subset is selected based on a ranking method that evaluates each band’s importance by considering its self-representative coefficient matrix. Experiments are carried out to assess the efficacy of CSSRnet. These experiments focus on evaluating classification accuracy across various cross-scene datasets, the utility of cross-scene concepts, and the practical application in coastal wetland. Experimental results confirm the effectiveness of CSSRnet.

While fusion of hyperspectral images (HSIs) with low spatial resolution and multispectral images (MSIs) with high spatial resolution has achieved significant success, high-quality fusion between noisy images has always been challenging. In this article, we propose a domain transform model driven by deep learning for anti-noise hyperspectral and multispectral image fusion (DTAFN). This marks the first time that wavelet decomposition theory is combined with deep learning for noise reduction in hyperspectral and MSI fusion. DTAFN initially decomposes hyperspectral and MSIs into frequency components and constructs a novel feature interaction fusion module (FIFM). This module, while using MSIs to guide the removal of noise from HSIs, also achieves the fusion of spatial and spectral information. Furthermore, it maps the fused features to a lower dimensional subspace to enhance computational efficiency. Additionally, we introduce a spatial-spectral self-attention mechanism to optimize the reconstructed frequency components using the subspace features. In the end, the wavelet inverse transform is used to reconstruct the clean fused image. It is worth noting that the extraction of the subspace is considered a process of nonlinear low-rank component extraction, which, to a certain extent, suppresses noise signals. Numerous experiments of mixed noise image fusion are carried out, and the experimental results show that DTAFN can obtain high-quality fusion results, is robust, and superior to the state-of-the-art methods.

Spectral super-resolution (SSR) aims to improve the spectral resolution of images from multispectral imagery or even red, green, blue (RGB) images. However, the majority of existing SSR methods do not fully exploit the spatial and texture features in RGB images, which would lead to the image unreal and distort of the high-frequency details in the reconstructed SSR images. In this study, a hybrid spectral and texture attention pyramid network (STANet) is proposed to reconstruct hyperspectral images (HSIs) with RGB bands of remote sensing images as input. More specifically, a learnable texture feature extraction module is proposed, aiming to make full use of the texture features in the RGB images, which are important in the subsequent spectral reconstruction. Furthermore, to better reconstruct the correlations between various spectral channels, a spatial-spectral-constrained cross-attention module is introduced. Finally, a novel spectral-texture fusion method is proposed, which successfully alleviates the problem of insufficient deep interaction among multiple deep features. On three remote sensing datasets, STANet demonstrates state-of-the-art performance, with its peak signal-to-noise ratio (PSNR) exceeding the suboptimal methods by 0.7266, 0.6724, and 0.6 dB, respectively. The results of the land-cover classification experiment using the reconstructed HSI further demonstrated the performance of the STANet algorithm.

Multispectral (MS)-hyperspectral (HS) image fusion, which aims to enhance the spatial resolution of low spatial resolution HS images with a high spatial resolution MS has provided a wide range of applications in remote sensing. However, relatively long revisit cycles of HS satellites and irresistible weather factors cause the acquisition of HS and MS images at the same time difficult. Most of the existing approaches neglect the temporal difference between MS and HS images, and perform weakness in the challenging case with diverse temporal difference spans. In this paper, we propose a novel image fusion strategy with embedding a stage of feature matching before interaction. On the one hand, we explore the role of spectral correlation modeling between HS and MS images, which accounts for the utilization of available spatial information from MS images. On the other hand, we design a feature aggregation module to fully exploit the nonlinear gaps and dependencies of heterogeneous data and utilize adaptive gains to realize complementary information projection and fusion. We build Dongying (DY) and Yellow River Estuary (YRE) remote sensing datasets based on Sentinel-2 and ZiYuan(ZY)-1 02D satellites with diverse temporal difference spans. The extensive experiments demonstrate that our method is robust to the span of temporal difference and shows superior performance over the existing methods visually and quantitatively.

Deep learning has emerged as a powerful method for hyperspectral image (HSI) classification. However, a significant prerequisite for HSI classification using deep learning is enough labeled samples, which is both time-consuming and labor-intensive. Yet, labeled samples are essential for training deep learning models. This paper proposes an HSI classification method based on the self-supervised learning of spectral masking (SSLSM). The method mainly includes two steps: self-supervised pre-training and fine-tuning. First, considering the rich spectral information of HSI, we propose masked spectral reconstruction as the pretext task. The unmasked data is input into the encoder and decoder sequentially, which are composed of a multi-layer transformer, for feature learning for masked spectral reconstruction. Second, we use reference samples to fine-tune the network, and the encoder and decoder are innovatively cascaded for deep semantic feature extraction, which can further improve the ability of feature extraction in the downstream classification tasks. Experiment results show that, compared with other methods, the SSLSM obtains the highest classification accuracy of 96.52%, 97.03%, and 96.70% on the Indian Pines dataset, Pavia University dataset, and Yancheng Wetlands dataset, respectively. Our method can also be applied to other HSI datasets, and the codes will be available from https://github.com/CIRSM-GRoup/2023-TGRS-SSLSM.

Image registration aims to eliminate the geometric deviation between multi-source data with the same range, and to promote the collaborative application of data. In recent years, spaceborne hyperspectral (HS) and multispectral (MS) data have been widely used in Earth observation. However, the difference in the number of bands, spatial resolution, and spectral resolution puts forward higher requirements on the registration algorithm. The key to HS and MS image registration is to extract more common key points, weaken and eliminate the difference of radiation and spatial texture information to build superior descriptors, and achieve high-precision matching of key points. This paper introduces a new robust HS and MS registration method based on common deep feature subspaces. We first construct the common deep feature subspaces extraction network to extract consistent edge features and common subspace images of the image pair. Then, Harris algorithm is used to extract key points from consistent edge features between images, which reduces the impact of spatial resolution differences between images. Besides, the SIFT descriptor and subspace images are used to describe key points, which reduces the impact of radiation differences between images. Finally, Euclidean distance is used for the initial matching of key points, and the affine matrix is calculated after the outliers are eliminated, and image registration is performed. We perform experiments on spaceborne HS and MS datasets of different spatial resolutions and comparisons with state-of-the-art methods. Experimental results show that our method can obtain satisfactory registration results and is robust.

Due to sensor design limitations and the influence of weather factors, it is currently challenging to obtain remote sensing images with high temporal, spatial, and spectral resolution. Spatial-temporal-spectral fusion aims to integrate the temporal, spatial, and spectral information from multiple sources of remote sensing images to reconstruct a remote sensing image with high temporal, spatial, and spectral resolution. Existing methods typically require at least three types of data to achieve spatial-temporal-spectral fusion. However, acquiring remote sensing data observed at the same time poses significant difficulties. The major challenge lies in effectively utilizing hyperspectral images with low spatial and temporal resolution and multispectral images with high temporal and spatial resolution to reconstruct remote sensing images with high temporal, spatial, and spectral resolution. To address the aforementioned issues, we propose a novel unsupervised 3D tensor subspace decomposition network. Our method incorporates the theory of 3D tensor subspace decomposition, utilizing a 3D hyperspectral/multispectral tensor subspace extraction network to predict the hyperspectral tensor subspace features with low spatial resolution missing at other times (To better understand, the missing moment is defined as time 2). Subsequently, the 3D hyperspectral tensor subspace reconstruction network is employed along with the time 2 hyperspectral tensor subspace features with low spatial resolution and the time 2 multispectral image to reconstruct the time 2 hyperspectral image with high spatial resolution. In the experiment, we utilize three simulated datasets and two real datasets to evaluate the fusion performance of our proposed method. The results demonstrate that our method achieves high-quality fusion results and exhibits comparable performance, and has robustness and practicality.

Spatial-temporal-spectral fusion (STSF) of remote sensing imagery can produce data with the highest spatial and spectral resolution, only as well as fine temporal resolution, by integrating images with complementary information in both the temporal and spectral domains. Accuracy of temporal variation is an important guarantee for achieving fidelity fusion in STSF. However, current STSF methods estimate the temporal variation only by utilizing the temporal variation between observed multispectral image (MSI) and the relationship between MSI and hyperspectral image (HSI), which is difficult to obtain accurate temporal variation. To address this problem, this paper proposes a coupled temporal variation information estimation and resolution enhancement for remote sensing image spatial-temporal-spectral fusion (CTVRE-STSF). The temporal variation information estimation model estimates the temporal variation of the target image, while the resolution enhancement model provides additional constraints for estimating the temporal variation. For the temporal variation information reconstruction model, we build a temporal variation information estimation based on a generalized linear mixed model and use the temporal variation between MSIs. In addition, a resolution enhancement model is constructed to estimate the temporal variation of the target image by incorporating relevant prior knowledge. The introduction of the resolution enhancement model in the prior provides additional constraints on the estimation of the temporal variation high-dimensional information, thus facilitating the resolution improvement. Experimental results on two real datasets demonstrate the effectiveness and superiority of our proposed method over current state-of-the-art methods, especially in terms of spectral fidelity.

The pursuit of super-resolution (SR) with large upscaling factors such as 8×, for enhancing the spatial resolution of low-resolution (LR) remote sensing images is a persistent and challenging problem. To address this issue, we propose the Progressive Feature Enhancement SR (PFESR) network with an 8× upscaling factor. Given the limited high-frequency information provided by a single LR image, we propose an improved style transfer technology to generate auxiliary details that aid in the recovery of high-resolution (HR) images. Additionally, multi-scale texture features are extracted through the Visual Geometry Group (VGG) feature extraction (VFE) block. To efficiently fuse various features, we combine hard and soft attention mechanisms. Finally, we use a hierarchical fusion block to address the progressive fusion problem of multiple scale features. Experiments on three datasets demonstrate that our method achieves state-of-the-art performance and exhibits good robustness in 8× and higher scale SR tasks.

The improvement of temporal resolution of hyperspectral (HS) data is a fundamental and challenging problem. In this paper, we propose a Temporal-Spectral fusion method based on Generative Adversarial Network (TSF-GAN). First, the generator is used to train the nonlinear relationship between multispectral (MS) and HS data pairs at time T1 and T3, and we map the relationship to the MS data at T2 to obtain the HS data. Second, the discriminator is used to identify whether the differential image of HS data at different times is consistent with that of MS data, and whether the HS data at time T2 after spectral down-sampling is consistent with that of MS data at time T2. Preliminary experimental results demonstrate that the proposed TSF-GAN achieves comparative fidelity and has strong practicability.

Zi Yuan (ZY)-1 02D satellite simultaneously provides the low spatial resolution (LR) and narrow swath-width hyperspectral (HS) image, the moderate spatial resolution (MR) multispectral (MS) image with a wider swath width, and the high spatial resolution (HR) panchromatic (PAN) image with the same wide swath width to the MR MS. How to comprehensively integrate their complementary advantages to obtain the wide swath-width and high-fidelity HR HS image is interesting but challenging. In this paper, we propose an integrated fusion method for the HR PAN, MR MS, and LR HS images with different swath widths, to generate the optimal wide swath-width HR HS image. The proposed method is based on the encoder-decoder learning framework. In the proposed fusion framework, a novel multi-branch encoder structure with an enhanced HS-encoder module and the multilevel spatial-spectral aggregation block is designed, by considering the difference in the spatial and spectral resolution among the multi-sensor images. The experiments on synthetic and real datasets from both qualitative and quantitative aspects demonstrated the competitive performance of the proposed method.

Pansharpening aims to sharpen a low spatial resolution (LR) multispectral (MS) image using a high spatial resolution (HR) panchromatic (PAN) image to obtain the HR MS image. Though large numbers of pansharpening methods have been proposed, and many advanced methods have shown high quantitative results, few of them are widely used in real applications. This may be attributed to their instability for different images with different ground surface features, or the complexity to be implemented and the time-consuming process for some state-of-the-art methods. In this letter, we proposed a simple, adjustable, and robust fusion (SARF) method. In the proposed method, a spatial-spectral coenhanced strategy was proposed, and several details of the proposed fusion model were specifically designed for the “simple, adjustable, robust” features. It was tested and verified by four-band and eight-band MS images based on reduced resolution (RR) and full resolution (FR) experiments. The experimental results demonstrated the promising spatial visuality of the proposed method, and the spectral fidelity was more robust than most of component substitution (CS)-based and multiresolution analysis (MRA)-based methods.

This letter presents a multiscale low-rank decomposition (MSLRD) method to extract multiscale spatial structures from hyperspectral images. The MSLRD assumes that ground objects have divergent characteristics in changing spatial scales. It decomposes each band image into a series of block-wise matrices, where these low-rank blocks take detailed spatial structures at multiple scales. It formulates the low-rank matrix decomposition problem into minimizing the ranks of all block matrices and adopts the alternative direction of the multiplier method to optimize it. Experiments on Indian Pines and Pavia University data sets show that the MSLRD can greatly improve the classification performance of regular classification on spectral features (i.e., all bands) and perform better than five state-of-the-art spatial feature extraction methods.

The maintenance of spectral variability between subclass objects and the relationship between hyperspectral (HS) bands have been a fundamental but challenging problem for fusing low spatial resolution (LR) HS and high spatial resolution (HR) multispectral (MS) images. This article presents a locally optimized image segmentation fusion (LOISF) framework for HS super-resolution reconstruction. First, LR HS and HR MS are clustered and segmented, and the label attributes of the segmented objects are identified by the prior information. Then, a novel joint fusion model for different typical ground objects is constructed based on spectral unmixing. The fusion problem is formulated mathematically as a convex optimization of a Frobenius norm, which includes spatial, spectral, and index constraints, with an alternating-directions’ optimization featuring linearization providing the solution. Experimental results demonstrate that the proposed LOISF preserves both spatial details and texture, achieving high spectral fidelity, and yielding significantly improved image quality compared to other state-of-the-art fusion methods.

This article proposes a dual global–local attention network (DGLAnet), which is an end-to-end unsupervised band selection (UBS) method that fully utilizes spatial and spectral information in both global and local aspects. The DGLAnet assumes that BS can be realized using the hyperspectral image (HSI) reconstruction process. First, the DGLAnet implements a dual attention module to obtain spatial–spectral and global–local features to reweight the HSI data. It adopts bi-directional relations to grasp spatial and spectral features from a global perspective. Meanwhile, the DGLAnet extracts local features through max-pooling and mean-pooling and then merges them via the convolution operation. Global–local features are utilized to learn attention to recalibrate the original data, and the reconstruction module is adopted to restore the original image from the reweighted HSI data. Finally, a proper band subset is selected by the constructed band evaluation index. Experiments on three hyperspectral data show that the DGLAnet outperforms other state-of-the-art methods and uses all bands with a lower computational cost.

The nonoverlapped spectrum range between low spatial resolution (LR) hyperspectral (HS) and high spatial resolution (HR) multispectral (MS) images has been a fundamental but challenging problem for MS/HS fusion. The spectrum of HS data is generally 400–2500 nm, and the spectrum of MS data is generally 400–900 nm; how to obtain the high-fidelity HR HS fused image within the whole spectrum of 400–2500 nm? In this article, we proposed a band divide-and-conquer framework (BDCF) to solve the problem, by comprehensively considering spectral fidelity, spatial enhancement, and computational efficiency. First, the spectral bands of HS were divided into overlapped and nonoverlapped bands according to the spectral response between HS and MS. Then, a novel improved component substitution (CS)-based method by combing neural network was proposed to fuse the overlapped bands of LR HS. Then, a mapping-based method with the neural network was presented to construct the complicated nonlinear relationship between overlapped and nonoverlapped bands of the original LR HS data. The trained network was mapped to the fused overlapped HR HS bands to estimate the nonoverlapped HR HS bands. Experimental results on two simulated data sets and two realistic data sets of Gaofen (GF)-5 LR HS, GF-1 MS, and Sentinel-2A MS show that the proposed BDCF has superior performance in both high spectral fidelity and sharp spatial details, and it obtained competitive fusion behaviors compared with other state-of-the-art methods. Moreover, BDCF has relatively higher computational efficiency than optimal solution-based methods and deep learning-based fusion methods.

Fusing low spatial resolution (LR) hyperspectral (HS) data and high spatial resolution (HR) multispectral (MS) data aims to obtain HR HS data. However, due to bad weather and the aging of sensor equipment, HS images usually contain a lot of noise, e.g., Gaussian noise, strip noise, and mixed noise, which would make the fused image have low quality. To solve this problem, we propose the multiscale low-rank deep back projection fusion network (MLR-DBPFN). First, HS and MS are superimposed, and multiscale spectral features of the stacked image are extracted through multiscale low-rank decomposition and convolution operation, which effectively removes noisy spectral features. Second, the upsampling and downsampling network mechanisms are used to extract the multiscale spatial features from each layer of spectral features. Finally, the multiscale spectral features and multiscale spatial features are combined for network training, and the weight of the noisy spectrum features is reduced through the network feedback mechanism, which suppresses the noisy spectrum and improves the noisy HS fusion performance. Experimental results on datasets of different noise demonstrate that MLR-DBPFN has superior spatial and spectral fidelity, comparative fusion quality, and robust antinoise performance compared with state-of-the-art methods.

This article proposes a multiscale spectral features graph fusion (MSFGF) method for selecting proper hyperspectral bands. The MSFGF regards that the selected bands should reflect diagnostic spectral information of ground objects at different scales, and it explores band selection from the aspect of multiple spatial scales. First, it adopts the multiscale low-rank decomposition (MSLRD) model to find multiscale spectral features of different ground objects. The model considers divergent spatial structures or spatial correlations of ground objects at different scales, and factorizes the hyperspectral data cube into a series of low-rank block-wise data cubes, where the blocks take spatial structures of different ground objects at increasing scales. Second, the MSFGF presents the multiscale sparse spectral clustering (MSSC) model to fuse the separate connected graphs of multiscale spectral features into a consensus graph. The consensus graph combines the complementary information of multiscale spectral features and helps to reveal the intrinsic clustering structure of all spectral bands. Finally, the MSFGF utilizes spectral clustering to find clusters from the consensus graph and selects representative bands. Experimental results on three widely used hyperspectral data prove the superiority of MSFGF in selecting bands, where it outperforms other seven state-of-the-art methods in classification with an acceptable computational cost.

Image fusion effectively solves the trade-off between spatial resolution, temporal resolution, and spectral resolution of remote sensing sensors. However, most of existing methods focus on the fusion of two of the spatial, temporal, and spectral metrics of remote sensing images. The few spatial-temporal-spectral fusion (STSF) methods available are mainly for fusing MODIS and Landsat images, which are not suitable for the characteristics of the spaceborne hyperspectral images with low temporal resolution, such as Hyperion, ZY-1 02D, and PRISMA. For this purpose, we proposed a novel generalized linear spectral mixing model for spatial-temporal-spectral fusion (GLMM-STSF). In the method, the GLMM is introduced into the STSF problem, and the temporal variations of images at different times are transferred to the endmember and abundance matrix variations of images for estimation. To the best of our knowledge, for the first time, the STSF task of remote sensing images is handled from the perspective of spectral unmixing. Compared with existing STSF fusion methods, our method targets the task of fusing spaceborne HSI with low temporal and spatial resolutions with multispectral image featured by high temporal and spatial resolutions. Taking the STSF of ZY-1 02D hyperspectral and Sentinel-2 multispectral real datasets as an example, comparisons with related state-of-the-art methods demonstrate that our proposed method achieves superior fusion performance.

This letter presents a correntropy-based sparse spectral clustering (CSSC) method to select proper bands of a hyperspectral image. The CSSC first constructs an affinity matrix with the correntropy measure which considers the nonlinear characteristics of hyperspectral bands and can suppress effects from noise or outliers in measuring band similarity. The CSSC imposes the sparsity and block diagonal constraint on spectral clustering, which can further improve band clustering performance. Bands are finally selected from each cluster on the connected graph. Experimental results on two widely used hyperspectral images show that the CSSC behaves better than spectral clustering and other several state-of-the-art methods in band selection.

This letter proposes a lateral-slice sparse tensor robust principal component analysis (LSSTRPCA) method to remove gross errors or outliers from hyperspectral images so as to promote the performance of subsequent classification. The LSSTRPCA assumes that a three-order hyperspectral tensor has a low-rank structure, and gross errors or outliers are sparsely scattered in a 2-D space (i.e., lateral-slice) of the tensor. It formulates a low-rank and sparse tensor decomposition problem into a convex problem and then implements the inexact augmented Lagrange multiplier method to solve it. The experiments on two hyperspectral data sets show that the LSSTRPCA can successfully remove outliers or gross errors and achieve higher accuracies than both the original robust principal component analysis (RPCA) and tensor robust principal component analysis (TRPCA).

This article presents a fast and latent low-rank subspace clustering (FLLRSC) method to select hyperspectral bands. The FLLRSC assumes that all the bands are sampled from a union of latent low-rank independent subspaces and formulates the self-representation property of all bands into a latent low-rank representation (LLRR) model. The assumption ensures sufficient sampling bands in representing low-rank subspaces of all bands and improves robustness to noise. The FLLRSC first implements the Hadamard random projections to reduce spatial dimensionality and lower the computational cost. It then adopts the inexact augmented Lagrange multiplier algorithm to optimize the LLRR program and estimates sparse coefficients of all the projected bands. After that, it employs a correntropy metric to measure the similarity between pairwise bands and constructs an affinity matrix based on sparse representation. The correntropy metric could better describe the nonlinear characteristics of hyperspectral bands and enhance the block-diagonal structure of the similarity matrix for correctly clustering all subspaces. The FLLRSC conducts spectral clustering on the connected graph denoted by the affinity matrix. The bands that are closest to their separate cluster centroids form the final band subset. Experimental results on three widely used hyperspectral data sets show that the FLLRSC performs better than the classical low-rank representation methods with higher classification accuracy at a low computational cost.

This paper presents a robust multi-feature spectral clustering (RMSC) method for hyperspectral band selection. The RMSC combines four features of hyperspectral bands and formulates a low-rank and sparse decomposition program to estimate the integrated similarity matrix for spectral clustering. The integrated similarity matrix is assumed to represent the total similarity information of band information entropy, band correlation and band divergence and alleviate negative effects from noise or outliers in hyperspectral images. After that, spectral clustering is implemented on the integrated similarity matrix to select appropriate bands. Experimental results on the Indian Pines datasets show that the RMSC could greatly improve the classification accuracy of spectral clustering and meanwhile outperform state-of-the-art band selection methods.

This paper aims to compare the performance of GaoFen-1 (GF-1), GaoFen-5 (GF-5), Landsat-8 data in fine classification. An optimized sample selection method (OSSM) is developed to ensure the high quality of samples. This method adopts different band combination strategies to realize optimal selection of training samples under the aid of normalized vegetation index (NDVI), normalized water index (NDWI) and the components of Kauth-Thomas (KT) Transformation. After that, support vector machine (SVM) is implemented on these three types of data. Experimental results on China Dunhuang calibration field, Gansu Ying-mao-tuo exploration area and Gan River lower reaches datasets show that GF-5 data performs best in both qualitative and quantitative evaluation of fine classification thanks to its hyperspectral properties.

In this letter, we propose an unsupervised change detection method for synthetic aperture radar (SAR) images based on variational multivariate Gaussian mixture model (MGMM) and Shannon entropy. First, the difference features are generated from the Gabor wavelet transform of two SAR images. In variational inference framework, the variational MGMM is first introduced to implement accurate modeling for the data distribution of difference features and to output responsibilities. Subsequently, spatial information is explored on the responsibilities to yield thecontextual responsibilitiesfor improving the accuracy and reliability of change detection. Then,a posterioriprobabilities of the changed and unchanged classes are derived from thecontextual responsibilities, and Shannon entropy, being directly related to the classification error rate, is proposed to determine the optimal index integer. Finally, the binary change mask is achieved by separating the pixels into the changed and unchanged classes. The experiments on three pairs of SAR images for describing urban sprawl and water bodies demonstrate the effectiveness of the proposed method.

In this paper, we propose a novel unsupervised change detection method of remote sensing (RS) images based on a unified framework for weighted collaborative representation (WCR) with robust deep dictionary learning (RDDL) and fuzzy clustering. Specifically, WCR is employed to collaboratively represent neighborhood features with lower computational complexity, for which the RDDL model is built to learn more effective and representative overcomplete dictionary and enhance the robustness against the noise and outliers. Meanwhile, in order to make the resulting collaborative coefficients more beneficial for clustering, the unified framework for WCR with RDDL and fuzzy clustering is designed. By doing so, our framework not only precludes the utilization of third-party clustering algorithm, but also achieves better detection performance. Subsequently, the spatial constraint is enforced on the membership matrix to yield the updated one for further improving the accuracy of change detection. Finally, a binary change mask (CM) is achieved by assigning the pixels into the changed and unchanged classes. Experiments are performed on five pairs of RS images, and experimental results demonstrate the effectiveness of the proposed method.

This paper proposes a compressed columnwise robust principal component analysis (CCRPCA) method for hyperspectral anomaly detection. The CCRPCA improves the regular RPCA by using the Hadamard random projection and constraining the columnwise structure of sparse anomaly matrix. The Hadamard random projection reduces the computational cost of the hyperspectral data, and the columnwise sparse structure alleviates negative effects from the anomalies on the columns of the background. The sparse anomaly matrix and the background matrix are estimated by optimizing a convex program, and the anomalies are estimated from nonzero columns of the compressed sparse matrix. Preliminary experiment result from the San Diego dataset shows that the CCRPCA outperforms four state-of-the-art detection methods in both the receiver operating characteristic curve and the area under curve.

In this paper, we propose two novel classification frameworks for hyperspectral image (HSI) based on capsule network (CapsNet), which could address the drawbacks of convolutional neural network (CNN) and problem of limited training samples by introducing affine transformation matrix. Specifically, the proposed framework first performs the classification of HSI based on spectral information. Second, considering the importance of spatial information for HSI processing, we integrate the spatial and spectral information into the proposed framework to further improve the classification performance. Experimental results on real HSI data demonstrate the effectiveness of the proposed framework.

In the paper, an unsupervised change detection method for remote sensing (RS) images based on deep semi-nonnegative matrix factorization (semi-NMF) is proposed. Firstly, the difference image is generated in different ways, depending on the types of input images. Then principal component analysis (PCA) is applied on the difference image to form the feature matrix X for improving the capability against various noise. In order to exploit more useful information from the resulting feature matrix, deep semi-NMF is introduced to factorize X into L+1 factors consisting of L nonrestricted matrices {Fl}l=1Land nonnegative cluster indicator matrix GL. Finally, the binary change mask (CM) is generated by assigning the pixels into changed and unchanged classes according to maximum criterion. The experimental results on two pairs of multitemporal RS images demonstrate the effectiveness of the proposed method.

The land surface reflectance data are indispensable to generate many other land products. Global land surface reflectance data have been routinely produced from remote sensing sensors aboard different satellite platforms. However, the original data, especially the daily data, suffer from a large number of spatial gaps, which result from atmospheric contamination and instrument deficiencies. This seriously limits their further applications. Many composite products with less spatial gaps have been generated to solve the above problem, but they easily sacrifice their temporal resolutions of original data. Even worse, they cannot be directly implemented in realistic applications because of the noise and composite seams. This paper proposes a temporal-spatial reconstruction method (TSRM) to generate daily gapless and seamless land surface reflectance data. The TSRM integrates both temporal and spatial information for recovering different land cover types using three processing steps. First, spatial gaps are coarsely filled with multiyear weighted average (Step1). After that, all the gaps that are not filled in the first step are interpolated by using harmonic analysis of time series with true value constraint (Step2). Finally, the reconstructed results in the last step are seamlessly processed using the Poisson image editing method, and the seamless daily reflectance data set is generated (Step3). The Moderate Resolution Imaging Spectroradiometer reflectance data set (MOD09GA and MYD09GA) on two testing areas is selected to verify the performance of the proposed TSRM. Experimental results show that the TSRM has good performance with regard to maintaining the temporal and spatial integrity of the daily land surface reflectance data. Results on different testing sites also demonstrate that the TSRM preserves spectral integrity with clear seasonal trends for each spectral band.

A sparse and low-rank near-isometric linear embedding (SLRNILE) method has been proposed to make dimensionality reduction and extract proper features for hyperspectral imagery (HSI) classification. The SLRNILE stands on the theory of the John-Lindenstrauss lemma, and tries to estimate a sparse and low-rank projection matrix that satisfies the restricted isometric property (RIP) condition on all secants of the HSI data. The RIP condition guarantees that the desired linear mapping near-isometrically preserves nearest neighbor points of all HSI pixels. Seeking the desired mapping is then modeled into minimizing a Lagrange multipliers formulation. The alternating direction method of multipliers framework is utilized to solve the above convex program, and column generation techniques are adopted to alleviate the computation memory burden during the optimization procedure. Five experiments on three widely used HSI data sets are designed to completely test the performance of SLRNILE, and experimental results are compared against those of six state-of-the-art feature extraction methods, including principal component analysis, Laplacian eigenmaps, locality preserving projections, neighborhood preserving embedding, sparse nonnegative matrix underapproximation, and random projections. The results show that SLRNILE performs best among all the seven methods, and its computational time is longest of all but still bearable for regular users. Therefore, the SLRNILE can be a good choice for feature extraction in HSI classification.

Global or regional environmental change is of wide concern. Extensive studies have indicated that long-term vegetation cover change is one of the most important factors reflecting environmental change, and normalized difference vegetation index (NDVI) time-series data sets have been widely used in vegetation dynamic change monitoring. However, the significant residual effects and noise levels impede the application of NDVI time-series data in environmental change research. This study develops a novel and robust filter method, i.e., the moving weighted harmonic analysis (MWHA) method, which incorporates a moving support domain to assign the weights for all the points, making the determination of the frequency number much easier. Additionally, a four-step process flow is designed to make the data approach the upper NDVI envelope, so that the actual change in the vegetation can be detected. A total of 487 test pixels selected from SPOT VEGETATION 10-day MVC NDVI time-series data from January 1999 to December 2001 were used to illustrate the effectiveness of the new method by comparing the MWHA results with the results of another four existing methods. Finally, the long-term SPOT VEGETATION 10-day maximum-value compositing (MVC) NDVI time series for China from April 1998 to May 2014 was reconstructed by the use of the proposed method, and a test region in China was utilized to validate the effectiveness of the proposed MWHA method. All the results indicate that the reconstructed high-quality NDVI time series fits the actual growth profile of the vegetation and is suitable for use in further remote sensing applications.

With regard to quantitative remote sensing products in the visible and infrared ranges, thick clouds and accompanying shadows are an inevitable source of noise. Due to the absence of adequate supporting information from the data themselves, it is a formidable challenge to accurately restore the surficial information underlying large-scale clouds. In this paper, dictionary learning is expanded into the multitemporal recovery of quantitative data contaminated by thick clouds and shadows. This paper proposes two multitemporal dictionary learning algorithms, expanding on their KSVD and Bayesian counterparts. In order to make better use of the temporal correlations, the expanded KSVD algorithm seeks an optimized temporal path, and the expanded Bayesian method adaptively weights the temporal correlations. In the experiments, the proposed algorithms are applied to a reflectance product and a land surface temperature product, and the respective advantages of the two algorithms are investigated. The results show that, from both the qualitative visual effect and the quantitative objective evaluation, the proposed methods are effective.