High-resolution satellite imagery with dense temporal series is crucial for long-term surface change monitoring. Spatiotemporal fusion seeks to reconstruct remote sensing image sequences with both high spatial and temporal resolutions by leveraging prior information from multiple satellite platforms. However, significant radiometric discrepancies and large spatial resolution variations between images acquired from different satellite sensors, coupled with the limited availability of prior data, present major challenges to accurately reconstructing missing data using existing methods. To address these challenges, this paper introduces GCM-PDA, a novel generative compensation model with progressive difference attenuation for spatiotemporal fusion of remote sensing images. The proposed model integrates multi-scale image decomposition within a progressive fusion framework, enabling the efficient extraction and integration of information across scales. Additionally, GCM-PDA employs domain adaptation techniques to mitigate radiometric inconsistencies between heterogeneous images. Notably, this study pioneers the use of style transformation in spatiotemporal fusion to achieve spatial-spectral compensation, effectively overcoming the constraints of limited prior image information. Experimental results demonstrate that GCM-PDA not only achieves competitive fusion performance but also exhibits strong robustness across diverse conditions.

Hyperspectral image (HSI) has fine spectral resolution and abundant spatial information to detect subtle differences between targets. However, it is heavily contaminated with noise due to sensor design and atmospheric radiative transfer, resulting in spectral shifts and spatial discontinuities. Current denoising methods usually establish constraints directly on the ground truth and denoised image, lacking supervision of intermediate parameters of the network, resulting in insufficient model constraints and poor convergence. In addition, existing methods do not consider spatial-spectral compensation, so the denoising results have obvious spatial-spectral distortion. To this end, we propose a novel multiscale spatial-spectral invertible compensation network (MSIC-Net) for HSI denoising. The method constructs an invertible spatial-spectral compensation (ISSC) module, which supervises intermediate features through inverse constraints, realizes the circulation of multiscale information, and improves the stability of the model. At the same time, we also introduce style transfer for spatial-spectral compensation, which uses its superior fine feature control ability to precisely compensate for the lost spatial and spectral detail features. The method is extensively validated experimentally and categorically on simulated and real datasets. The experimental results show that MSIC-Net outperforms other state-of-the-art denoising methods in quantitative and qualitative evaluations.

The hyperspectral imaging instrument makes a trade-off by sacrificing spatial resolution to achieve high spectral resolution. This compromise leads to a low signal-to-noise ratio, and hyperspectral images (HSIs) are often heavily contaminated with mixed noise, which is an inherent challenge. Previous research has achieved satisfactory results for natural image denoising; hyperspectral denoising has remained a formidable task. In this article, we introduce an innovative method called the multistage hybrid-denoising network for satellite hyperspectral images (SUC-MSDN). SUC-MSDN initially decomposes the noisy HSI into multiple scales and constructs a multistage denoising network by analyzing the spatial spectrum texture distribution characteristics of noise signals. Instead of simply stacking the output results from each scale, SUC-MSDN uses the denoising results from the low-scale network as prior knowledge for the high-scale denoising network to more accurately remove the final noise components. Extensive experimental datasets are used to validate the performance of SUC-MSDN. Experimental results show that SUC-MSDN outperforms benchmark methods and significantly enhances the accuracy of land cover mapping.

This paper proposes a novel deep learning-based framework for hyperspectral band selection, named Cross-Scene Self-Representative Network (CSSRnet). The proposed method leverages the rich labels of the source domain (SD) to guide the band selection in the target domain (TD). To our knowledge, CSSRnet is the first deep learning-based solution for cross-scene hyperspectral band selection. First, the CSSRnet employs contextual attention mechanism to capture the latent features of SD and TD. It combines the self-attention mechanism with convolutional operations to capture static and dynamic contextual information. Then, the self-representative layer provides the self-representative coefficient of SD and TD. Subsequently, the maximum mean difference is utilized to align the self-representative coefficients of both SD and TD. To enhance the representativeness and precision of these coefficients, we introduce different tasks for the SD and TD branches. Finally, a suitable band subset is selected based on a ranking method that evaluates each band’s importance by considering its self-representative coefficient matrix. Experiments are carried out to assess the efficacy of CSSRnet. These experiments focus on evaluating classification accuracy across various cross-scene datasets, the utility of cross-scene concepts, and the practical application in coastal wetland. Experimental results confirm the effectiveness of CSSRnet.

While fusion of hyperspectral images (HSIs) with low spatial resolution and multispectral images (MSIs) with high spatial resolution has achieved significant success, high-quality fusion between noisy images has always been challenging. In this article, we propose a domain transform model driven by deep learning for anti-noise hyperspectral and multispectral image fusion (DTAFN). This marks the first time that wavelet decomposition theory is combined with deep learning for noise reduction in hyperspectral and MSI fusion. DTAFN initially decomposes hyperspectral and MSIs into frequency components and constructs a novel feature interaction fusion module (FIFM). This module, while using MSIs to guide the removal of noise from HSIs, also achieves the fusion of spatial and spectral information. Furthermore, it maps the fused features to a lower dimensional subspace to enhance computational efficiency. Additionally, we introduce a spatial-spectral self-attention mechanism to optimize the reconstructed frequency components using the subspace features. In the end, the wavelet inverse transform is used to reconstruct the clean fused image. It is worth noting that the extraction of the subspace is considered a process of nonlinear low-rank component extraction, which, to a certain extent, suppresses noise signals. Numerous experiments of mixed noise image fusion are carried out, and the experimental results show that DTAFN can obtain high-quality fusion results, is robust, and superior to the state-of-the-art methods.

Spectral super-resolution (SSR) aims to improve the spectral resolution of images from multispectral imagery or even red, green, blue (RGB) images. However, the majority of existing SSR methods do not fully exploit the spatial and texture features in RGB images, which would lead to the image unreal and distort of the high-frequency details in the reconstructed SSR images. In this study, a hybrid spectral and texture attention pyramid network (STANet) is proposed to reconstruct hyperspectral images (HSIs) with RGB bands of remote sensing images as input. More specifically, a learnable texture feature extraction module is proposed, aiming to make full use of the texture features in the RGB images, which are important in the subsequent spectral reconstruction. Furthermore, to better reconstruct the correlations between various spectral channels, a spatial-spectral-constrained cross-attention module is introduced. Finally, a novel spectral-texture fusion method is proposed, which successfully alleviates the problem of insufficient deep interaction among multiple deep features. On three remote sensing datasets, STANet demonstrates state-of-the-art performance, with its peak signal-to-noise ratio (PSNR) exceeding the suboptimal methods by 0.7266, 0.6724, and 0.6 dB, respectively. The results of the land-cover classification experiment using the reconstructed HSI further demonstrated the performance of the STANet algorithm.

Deep learning methods face significant challenges in practical cross-scene classification tasks of hyperspectral images, primarily due to the difficulty of acquiring labels and the issue of inconsistent distribution caused by spectral drift. To tackle the above issues, we propose a deep dynamic adaptation network based on joint correlation alignment (DDAN-JCA) for cross-scene hyperspectral image classification. First, the dual-channel residual network (DCRN) and the attention mechanism module (AMM) are employed to extract spatial-spectral joint features from both source domain and target domain. Then, the method of correlation alignment (CORAL) is employed to minimize the marginal distribution discrepancy between two domains and further reduce the conditional distribution discrepancy of each class. Finally, a dynamic distribution adaptation strategy is used to dynamically adjust the importance of marginal distribution and conditional distribution by using a balance factor. DDAN-JCA can achieve unsupervised classification without using target labels. The performance of DDAN-JCA has been validated using three hyperspectral datasets, and the experimental results demonstrate that DDAN-JCA significantly enhances classification accuracy and exhibits greater robustness compared to state-of-the-art methods.

Deep learning has emerged as a powerful method for hyperspectral image (HSI) classification. However, a significant prerequisite for HSI classification using deep learning is enough labeled samples, which is both time-consuming and labor-intensive. Yet, labeled samples are essential for training deep learning models. This paper proposes an HSI classification method based on the self-supervised learning of spectral masking (SSLSM). The method mainly includes two steps: self-supervised pre-training and fine-tuning. First, considering the rich spectral information of HSI, we propose masked spectral reconstruction as the pretext task. The unmasked data is input into the encoder and decoder sequentially, which are composed of a multi-layer transformer, for feature learning for masked spectral reconstruction. Second, we use reference samples to fine-tune the network, and the encoder and decoder are innovatively cascaded for deep semantic feature extraction, which can further improve the ability of feature extraction in the downstream classification tasks. Experiment results show that, compared with other methods, the SSLSM obtains the highest classification accuracy of 96.52%, 97.03%, and 96.70% on the Indian Pines dataset, Pavia University dataset, and Yancheng Wetlands dataset, respectively. Our method can also be applied to other HSI datasets, and the codes will be available from https://github.com/CIRSM-GRoup/2023-TGRS-SSLSM.

Change detection on heterogeneous optical and synthetic aperture radar (SAR) images is soaring and plays a crucial role in monitoring land cover changes, such as disaster emergencies and natural resource monitoring. This is commonly recognized as a promising but challenging work due to the intrinsic differences in imaging mechanisms between the optical and SAR images. Recently, deep learning-based change detection methods based on two-step processing have attracted attention, i.e., first image translation between optical and SAR images to alleviate their modality differences and then change detection based on the translated images. However, image translation itself is a trouble task for the heterogeneous optical and SAR images. The unreliable image translation results further limit the accuracy of change detection. In this paper, to mitigate this problem, we propose a change detection model on domain adaptation by novelty integrating change detection and image reconstruction into a unified framework. Specifically, we first transform the optical and SAR images into an intermediate common domain for comparison. Moreover, cross reconstruction for optical and SAR images is designed to maintain the characteristics of the images and improve the performance of domain adaptation. In addition, a feedback guidance mechanism is circumspectly designed to co-optimize change detection and image reconstruction tasks. Extensive experiments were conducted on four publicly available datasets, the results demonstrate the effectiveness of our proposed method.

Image registration aims to eliminate the geometric deviation between multi-source data with the same range, and to promote the collaborative application of data. In recent years, spaceborne hyperspectral (HS) and multispectral (MS) data have been widely used in Earth observation. However, the difference in the number of bands, spatial resolution, and spectral resolution puts forward higher requirements on the registration algorithm. The key to HS and MS image registration is to extract more common key points, weaken and eliminate the difference of radiation and spatial texture information to build superior descriptors, and achieve high-precision matching of key points. This paper introduces a new robust HS and MS registration method based on common deep feature subspaces. We first construct the common deep feature subspaces extraction network to extract consistent edge features and common subspace images of the image pair. Then, Harris algorithm is used to extract key points from consistent edge features between images, which reduces the impact of spatial resolution differences between images. Besides, the SIFT descriptor and subspace images are used to describe key points, which reduces the impact of radiation differences between images. Finally, Euclidean distance is used for the initial matching of key points, and the affine matrix is calculated after the outliers are eliminated, and image registration is performed. We perform experiments on spaceborne HS and MS datasets of different spatial resolutions and comparisons with state-of-the-art methods. Experimental results show that our method can obtain satisfactory registration results and is robust.

Due to sensor design limitations and the influence of weather factors, it is currently challenging to obtain remote sensing images with high temporal, spatial, and spectral resolution. Spatial-temporal-spectral fusion aims to integrate the temporal, spatial, and spectral information from multiple sources of remote sensing images to reconstruct a remote sensing image with high temporal, spatial, and spectral resolution. Existing methods typically require at least three types of data to achieve spatial-temporal-spectral fusion. However, acquiring remote sensing data observed at the same time poses significant difficulties. The major challenge lies in effectively utilizing hyperspectral images with low spatial and temporal resolution and multispectral images with high temporal and spatial resolution to reconstruct remote sensing images with high temporal, spatial, and spectral resolution. To address the aforementioned issues, we propose a novel unsupervised 3D tensor subspace decomposition network. Our method incorporates the theory of 3D tensor subspace decomposition, utilizing a 3D hyperspectral/multispectral tensor subspace extraction network to predict the hyperspectral tensor subspace features with low spatial resolution missing at other times (To better understand, the missing moment is defined as time 2). Subsequently, the 3D hyperspectral tensor subspace reconstruction network is employed along with the time 2 hyperspectral tensor subspace features with low spatial resolution and the time 2 multispectral image to reconstruct the time 2 hyperspectral image with high spatial resolution. In the experiment, we utilize three simulated datasets and two real datasets to evaluate the fusion performance of our proposed method. The results demonstrate that our method achieves high-quality fusion results and exhibits comparable performance, and has robustness and practicality.

Spatial-temporal-spectral fusion (STSF) of remote sensing imagery can produce data with the highest spatial and spectral resolution, only as well as fine temporal resolution, by integrating images with complementary information in both the temporal and spectral domains. Accuracy of temporal variation is an important guarantee for achieving fidelity fusion in STSF. However, current STSF methods estimate the temporal variation only by utilizing the temporal variation between observed multispectral image (MSI) and the relationship between MSI and hyperspectral image (HSI), which is difficult to obtain accurate temporal variation. To address this problem, this paper proposes a coupled temporal variation information estimation and resolution enhancement for remote sensing image spatial-temporal-spectral fusion (CTVRE-STSF). The temporal variation information estimation model estimates the temporal variation of the target image, while the resolution enhancement model provides additional constraints for estimating the temporal variation. For the temporal variation information reconstruction model, we build a temporal variation information estimation based on a generalized linear mixed model and use the temporal variation between MSIs. In addition, a resolution enhancement model is constructed to estimate the temporal variation of the target image by incorporating relevant prior knowledge. The introduction of the resolution enhancement model in the prior provides additional constraints on the estimation of the temporal variation high-dimensional information, thus facilitating the resolution improvement. Experimental results on two real datasets demonstrate the effectiveness and superiority of our proposed method over current state-of-the-art methods, especially in terms of spectral fidelity.

The pursuit of super-resolution (SR) with large upscaling factors such as 8×, for enhancing the spatial resolution of low-resolution (LR) remote sensing images is a persistent and challenging problem. To address this issue, we propose the Progressive Feature Enhancement SR (PFESR) network with an 8× upscaling factor. Given the limited high-frequency information provided by a single LR image, we propose an improved style transfer technology to generate auxiliary details that aid in the recovery of high-resolution (HR) images. Additionally, multi-scale texture features are extracted through the Visual Geometry Group (VGG) feature extraction (VFE) block. To efficiently fuse various features, we combine hard and soft attention mechanisms. Finally, we use a hierarchical fusion block to address the progressive fusion problem of multiple scale features. Experiments on three datasets demonstrate that our method achieves state-of-the-art performance and exhibits good robustness in 8× and higher scale SR tasks.

The improvement of temporal resolution of hyperspectral (HS) data is a fundamental and challenging problem. In this paper, we propose a Temporal-Spectral fusion method based on Generative Adversarial Network (TSF-GAN). First, the generator is used to train the nonlinear relationship between multispectral (MS) and HS data pairs at time T1 and T3, and we map the relationship to the MS data at T2 to obtain the HS data. Second, the discriminator is used to identify whether the differential image of HS data at different times is consistent with that of MS data, and whether the HS data at time T2 after spectral down-sampling is consistent with that of MS data at time T2. Preliminary experimental results demonstrate that the proposed TSF-GAN achieves comparative fidelity and has strong practicability.

The maintenance of spectral variability between subclass objects and the relationship between hyperspectral (HS) bands have been a fundamental but challenging problem for fusing low spatial resolution (LR) HS and high spatial resolution (HR) multispectral (MS) images. This article presents a locally optimized image segmentation fusion (LOISF) framework for HS super-resolution reconstruction. First, LR HS and HR MS are clustered and segmented, and the label attributes of the segmented objects are identified by the prior information. Then, a novel joint fusion model for different typical ground objects is constructed based on spectral unmixing. The fusion problem is formulated mathematically as a convex optimization of a Frobenius norm, which includes spatial, spectral, and index constraints, with an alternating-directions’ optimization featuring linearization providing the solution. Experimental results demonstrate that the proposed LOISF preserves both spatial details and texture, achieving high spectral fidelity, and yielding significantly improved image quality compared to other state-of-the-art fusion methods.

This article proposes a dual global–local attention network (DGLAnet), which is an end-to-end unsupervised band selection (UBS) method that fully utilizes spatial and spectral information in both global and local aspects. The DGLAnet assumes that BS can be realized using the hyperspectral image (HSI) reconstruction process. First, the DGLAnet implements a dual attention module to obtain spatial–spectral and global–local features to reweight the HSI data. It adopts bi-directional relations to grasp spatial and spectral features from a global perspective. Meanwhile, the DGLAnet extracts local features through max-pooling and mean-pooling and then merges them via the convolution operation. Global–local features are utilized to learn attention to recalibrate the original data, and the reconstruction module is adopted to restore the original image from the reweighted HSI data. Finally, a proper band subset is selected by the constructed band evaluation index. Experiments on three hyperspectral data show that the DGLAnet outperforms other state-of-the-art methods and uses all bands with a lower computational cost.

The nonoverlapped spectrum range between low spatial resolution (LR) hyperspectral (HS) and high spatial resolution (HR) multispectral (MS) images has been a fundamental but challenging problem for MS/HS fusion. The spectrum of HS data is generally 400–2500 nm, and the spectrum of MS data is generally 400–900 nm; how to obtain the high-fidelity HR HS fused image within the whole spectrum of 400–2500 nm? In this article, we proposed a band divide-and-conquer framework (BDCF) to solve the problem, by comprehensively considering spectral fidelity, spatial enhancement, and computational efficiency. First, the spectral bands of HS were divided into overlapped and nonoverlapped bands according to the spectral response between HS and MS. Then, a novel improved component substitution (CS)-based method by combing neural network was proposed to fuse the overlapped bands of LR HS. Then, a mapping-based method with the neural network was presented to construct the complicated nonlinear relationship between overlapped and nonoverlapped bands of the original LR HS data. The trained network was mapped to the fused overlapped HR HS bands to estimate the nonoverlapped HR HS bands. Experimental results on two simulated data sets and two realistic data sets of Gaofen (GF)-5 LR HS, GF-1 MS, and Sentinel-2A MS show that the proposed BDCF has superior performance in both high spectral fidelity and sharp spatial details, and it obtained competitive fusion behaviors compared with other state-of-the-art methods. Moreover, BDCF has relatively higher computational efficiency than optimal solution-based methods and deep learning-based fusion methods.

Fusing low spatial resolution (LR) hyperspectral (HS) data and high spatial resolution (HR) multispectral (MS) data aims to obtain HR HS data. However, due to bad weather and the aging of sensor equipment, HS images usually contain a lot of noise, e.g., Gaussian noise, strip noise, and mixed noise, which would make the fused image have low quality. To solve this problem, we propose the multiscale low-rank deep back projection fusion network (MLR-DBPFN). First, HS and MS are superimposed, and multiscale spectral features of the stacked image are extracted through multiscale low-rank decomposition and convolution operation, which effectively removes noisy spectral features. Second, the upsampling and downsampling network mechanisms are used to extract the multiscale spatial features from each layer of spectral features. Finally, the multiscale spectral features and multiscale spatial features are combined for network training, and the weight of the noisy spectrum features is reduced through the network feedback mechanism, which suppresses the noisy spectrum and improves the noisy HS fusion performance. Experimental results on datasets of different noise demonstrate that MLR-DBPFN has superior spatial and spectral fidelity, comparative fusion quality, and robust antinoise performance compared with state-of-the-art methods.

Image fusion effectively solves the trade-off between spatial resolution, temporal resolution, and spectral resolution of remote sensing sensors. However, most of existing methods focus on the fusion of two of the spatial, temporal, and spectral metrics of remote sensing images. The few spatial-temporal-spectral fusion (STSF) methods available are mainly for fusing MODIS and Landsat images, which are not suitable for the characteristics of the spaceborne hyperspectral images with low temporal resolution, such as Hyperion, ZY-1 02D, and PRISMA. For this purpose, we proposed a novel generalized linear spectral mixing model for spatial-temporal-spectral fusion (GLMM-STSF). In the method, the GLMM is introduced into the STSF problem, and the temporal variations of images at different times are transferred to the endmember and abundance matrix variations of images for estimation. To the best of our knowledge, for the first time, the STSF task of remote sensing images is handled from the perspective of spectral unmixing. Compared with existing STSF fusion methods, our method targets the task of fusing spaceborne HSI with low temporal and spatial resolutions with multispectral image featured by high temporal and spatial resolutions. Taking the STSF of ZY-1 02D hyperspectral and Sentinel-2 multispectral real datasets as an example, comparisons with related state-of-the-art methods demonstrate that our proposed method achieves superior fusion performance.

The Physical Unclonable Function (PUF) circuit generates a random, unclonable key by extracting random deviations of the manufacturing processes. In this paper, a delay ternary PUF (DT-PUF) circuit scheme based on Carbon Nanotube Field Effect Transistor (CNFET) is proposed. In this scheme, the two inputs and two outputs (2I2O) multi-value delay circuit and the multi-valued arbiter circuit is designed by the threshold controllable CNFET. Then, the ternary signals logic 0, logic 1 and logic 2 transmission delay path is realized by cascading multi-bit 2I2O multi-value delay circuits. Secondly, the delay deviation of the two identical transmission paths is set as the random source. With the help of ternary arbiter, the random delay competes to produce the unclonable ternary PUF output data. Finally, the proposed DT-PUF circuit is designed under 32nm CNFET standard model library, and the circuit is simulated and analyzed by HSPICE. Experiment results show that DT-PUF circuit has the correct logic function, and the distribution is 30.3%@logic 0, 36%@logic 1, 33.7%@logic 2, near ideal about 33.3%.