Recent advancements in text-to-image (T2I) generation have led to the emergence of highly expressive models such as diffusion transformers (DiTs), exemplified by FLUX. However, their massive parameter sizes lead to slow inference, high memory usage, and poor deployability. Existing acceleration methods (e.g., single-step distillation and attention pruning) often suffer from significant performance degradation and incur substantial training costs. To address these limitations, we propose FastFLUX, an architecture-level pruning framework designed to enhance the inference efficiency of FLUX. At its core is the Block-wise Replacement with Linear Layers (BRLL) method, which replaces structurally complex residual branches in ResBlocks with lightweight linear layers while preserving the original shortcut connections for stability. Furthermore, we introduce Sandwich Training (ST), a localized fine-tuning strategy that leverages LoRA to supervise neighboring blocks, mitigating performance drops caused by structural replacement. Experiments show that our FastFLUX maintains high image quality under both qualitative and quantitative evaluations, while significantly improving inference speed, even with 20% of the hierarchy pruned.

Hyperspectral images (HSIs) often suffer from various noises, such as Gaussian noise, stripe noise, impulse noise, and deadlines due to the influence of sensors and external environments. These noises significantly degrade the quality of HSI and hinder subsequent applications. While most current transformer-based methods can effectively remove certain types of noise, they struggle with wide stripe noise. In addition, transformers are typically applied within local windows due to the limitation of computational complexity. Although windowshifting operations enhance the interaction between windows to a certain extent, this interaction remains insufficient for comprehensive global modeling. In light of these limitations, we propose a Bidirectional-aware network combining Transformer and Mamba (BTMnet), which consists of Bidirectional Long-Short Distance Attention (BLSDA) and Channel-Split Mamba (CSM). To better remove wide stripe noise, BLSDA is designed with two rectangular windows adapted to wide stripes in both vertical and horizontal directions, utilizing transformers to compute attention relationships within windows and across different windows. To further integrate global information and enhance the interaction of features between adjacent windows, CSM extracts global features by scanning in four directions across different feature channels. In BLSDA, we applied bidirectional windows in vertical and horizontal directions, and in CSM, we conducted bidirectional scanning in vertical and horizontal directions. The combination of these techniques allows for the simultaneous extraction of bidirectional features from HSI. By evaluating the metrics and visualization, the experimental results on simulated and real experiments prove that our method can achieve better results.

Polarimetric synthetic aperture radar (SAR) can capture rich polarization information of targets, but it is inherently affected by speckle. Learning-based methods have demonstrated superior speckle suppression potential. Most existing methods use optical images to simulate SAR noise for model training. Because of the significant differences in the imaging mechanisms between optical and SAR images, the data characteristics of these two types differ significantly, resulting in poor generalization performance. To this end, an Information Flow Switching-based Despeckling Network (IFSDN) is proposed for dual-polarization SAR image. By using the long time series data, the first dual-polarization SAR real dataset is constructed. The hybrid feature extraction module (HFEM) is constructed to independently extract and integrate features from both the diagonal and nondiagonal elements of the covariance matrix. Additionally, the multihierarchical residual attention despeckling (MRAD) module performs despeckling on feature maps from low to high levels. On this basis, the information flow switching mechanism facilitates the interaction of dominant features before and after despeckling, injecting spatial details into the despeckled results, reducing speckle noise, and preserving polarization information. By considering temporal changes, an adaptive joint loss function is, furthermore, constructed to guide the network training process, achieving high-fidelity despeckling while maintaining spatial-polarization information. Experiments show that IFSDN outperforms existing state-of-the-art methods in the speckle removal task for real dual-polarization SAR images, which can effectively preserve spatial and polarization information while suppressing speckles. Besides, generalization experiments demonstrate that the proposed model can be effectively applied to diverse datasets across various climate zones, showcasing its strong robustness.

As typical structural noise, stripes commonly occur in remote sensing images captured by linear array sensors, which seriously lowers the image quality and hinders the downstream applications. Differing from the conventional methods, this article explores the ability of the ordering domain in separable stripe representation and provides a new perspective for destriping. To enhance the model flexibility to adapt to different types of stripes, the additive and multiplicative stripe components are fully considered and creatively incorporated into the observation model. Based on the additive-multiplicative observation model and the ordering domain transformation, we propose a novel destriping model, called ordering domain destriping (ODD), which constrains the additive and multiplicative stripe components in line with their statistical distribution characteristics. The results obtained on simulated and real striped images show that the proposed method can successfully estimate the latent clean images without losing stripe-like object details in challenging test scenarios, such as mixed additive-multiplicative stripes, wide stripes, and deadlines. The qualitative and quantitative comparisons with six other destriping methods verify the effectiveness and stability of the proposed model.

Synthetic aperture radar (SAR) images are inherently affected by speckle noise due to their imaging principles, significantly impacting downstream research. Denoising methods based on deep learning have garnered attention and are gradually maturing, yet they face certain challenges. Denoising single SAR images lacks temporal information, often resulting in structural fitting that introduces artifacts. The existing denoising techniques do not adequately account for the differences between SAR and optical images, despite their distinct structural characteristics. In addition, mainstream deep learning approaches heavily rely on data-driven methods, with limited consideration for statistical properties. In response to these challenges, we propose a novel network for denoising SAR images fusing multitemporal interaction with coherence prior, termed MTCSAR. The proposed network leverages multitemporal interaction (MTI) to gather information from different moments in various regions. The redundancy across these temporal dimensions helps mitigate artifacts and edge blurring. To address SAR image characteristics, we design a dual-branch network combining global and local information to handle large-scale regions and fine structures. In addition, we incorporate prior coherence information from SAR images into the network, utilizing statistical properties to enhance transparency during training. Experimental results on simulated and real datasets demonstrate that injecting MTIs and coherence improves our method’s qualitative and quantitative performance, surpassing current state-of-the-art algorithms. This validates the effectiveness of the proposed MTCSAR for multitemporal SAR image denoising.

Semi-empirical models and machine learning models have been widely used in remote sensing studies. In order to explore the feasible joint mode integrating semi-empirical model and machine learning algorithm, two distinct joint modes, cycle and series mode and deep and parallel mode, were designed and evaluated to synthesize the respective advantages of the two models to complete high-resolution soil moisture retrieval. Cycle and series mode improved the generalization ability of the retrieval models in the case of less labeled data, and enhanced its physical interpretability. The deep and parallel mode improved the accuracy of the retrieval models in a wider range of application with estimation accuracy of 0.754 and 0.071 m3·m-3in terms of coefficient of determination and unbiased root mean square error in site-based validation. The joint modes constructed in this study provide ideas for subsequent studies on model fusion and improving the physical interpretability of machine learning models.

Hyperspectral (HS) video captures continuous spectral information of objects, enhancing material identification in tracking tasks. It is expected to overcome the inherent limitations of red-green–blue (RGB) and multimodal tracking, such as finite spectral cues and cumbersome modality alignment. However, HS tracking faces challenges such as data anxiety, bandgaps, and huge volumes. In this study, inspired by prompt learning in language models, we propose the prompting for hyperspectral video tracking (PHTrack) framework. PHTrack learns prompts to adapt foundation models, mitigating data anxiety and enhancing performance and efficiency. First, the modality prompter (MOP) is proposed to capture rich spectral cues and bridge bandgaps for improved model adaptation and knowledge enhancement. In addition, the distillation prompter (DIP) is developed to refine cross-modal features. PHTrack follows feature-level fusion, effectively managing huge volumes compared to traditional decision-level fusion fashions. Extensive experiments validate the proposed framework, offering valuable insights for future research. The code and data will be available athttps://github.com/YZCU/PHTrack

Supervised deep learning despeckling methods usually use optical images to simulate multiplicative noise for training. However, due to the different imaging mechanisms of optical images and SAR images, the data characteristics of the two are significantly different, resulting in poor generalization performance of the model trained through the above form. Besides, the existing deep learning models do not fully consider the physical scattering mechanism, which causes the loss of polarization information. To solve those problems, an unsupervised deep learning method is proposed for dual-polarization SAR image despeckling. Under this framework, we combine the dual-polarization SAR covariance matrix and polarization decomposition information to construct a Dual-branch SAR image Despeckling Network (DSDN). The residual channel and the spatial attention mechanism are embedded to calibrate the polarization and spatial feature maps. The cross-attention mechanism is designed to mine the association of feature maps before and after denoising. Besides, the dual-branch joint loss function is proposed to constrain the training process. Spatial information experiments and polarization information experiments indicate that, compared with the existing state-of-the-art SAR despeckling methods, the proposed method can effectively remove the coherent speckle noise of dual-polarization SAR images, and can better preserve the polarization information. Codes are available at https://github.com/LiupengLin/DSDN.

Deep learning has achieved significant success in pan sharpening, but there are still two nonnegligible challenges. First, most of the existing methods rely on reduced-resolution training samples, limiting their performance when migrating from simulated to real-world scenes. Second, they pay insufficient attention to the imaging mechanism and the complexity and heterogeneity of remote sensing information, which leads to unclear representation of the relationships between images and the underutilization of the full-resolution features. In response to the abovementioned issues, this article presents an unsupervised pan-sharpening network incorporating imaging spectral prior and spatial-spectral compensation, named USCPNet. First, a structure-guided cross-attention (CA) residual (SCAR) block is constructed, deriving the desired high-resolution multispectral (HRMS) image guided by the panchromatic texture-structure features. A spectrally adaptive degradation network (SNet) coupling imaging spectral prior is then introduced, which characterizes the pixel-by-pixel spectral mapping between the HRMS image and the high-resolution panchromatic (HRPAN) image, to implement a precise spatial constraint driven by the imaging mechanism. In addition, given the difficulty of comprehensively extracting and integrating complementary features within an unsupervised framework through single-stream fusion, spatial-spectral joint progressive compensated (SSPC) stages are employed to achieve refined enhancement of the effective information in the predicted HRMS image through iterative rounds of residual fusion. Experiments conducted on Gaofen-1 (GF-1), Gaofen-2 (GF-2), and WorldView-2 (WV-2) satellite images reveal that the proposed USCPNet excels in spatial enhancement (SE) while preserving spectral fidelity. USCPNet also demonstrates advantages in specific applications, such as large-scale image fusion and vegetation index generation, compared with state-of-the-art methods.

Hyperspectral images (HSIs) with rich spectral information have been widely used in surface classification, object detection, and other real application problems. However, due to the hardware limitations, the low spatial resolution HSIs hinder the exploration of their application potential. Deep learning-based methods are currently the most common solutions for single HSI super-resolution (HSI SR) tasks. However, such methods often overlook the degradation principle from high-resolution HSI to low-resolution HSI. In this article, we propose a CNN-transformer embedded unfolding network (CTUNet), in which an unfolding framework with an effective spatial-spectral prior network is designed for HSI SR by incorporating the degradation principle of HSIs. Specifically, a maximum posterior-based energy model is employed, enabling alternate optimization to seek the optimal solution in an iterative mechanism. To effectively utilize the structure prior of HSI, multiscale self-calibrated convolution (MSSC) and edge-guided transformer module are combined to learn latent spatial-spectral priors. Additionally, hidden feature connections between adjacent iterations enhance the representation of the image features. Extensive experiments conducted on three available HSI datasets demonstrate that our method outperforms several state-of-the-art HSI SR methods. The code will be available athttps://github.com/YoeTon/CTUNet.

Soil moisture (SM) is a key state variable in hydrology, climatology and water resource management. Microwave remote sensing can retrieve soil moisture at regional or global scales, but limited by spatial resolution, it is difficult to accurately reflect the details of soil moisture. To overcome this limitation, in this paper, we propose a Soil Moisture Downscaling residual dense Network (SMDN) based on spatiotemporal information. We model the relationship between low-resolution geoscience parameters and soil moisture, and then downscale soil moisture products by applying the model to high-resolution geoscience parameters. On this basis, the residual correction is performed on the soil moisture downscaling results to generate high-precision soil moisture products. Visual evaluation and quantitative experiments show that the proposed network can effectively improve the spatial detail information, maintain the spatial distribution pattern, and have strong stability.

Due to the limitations of sensor hardware devices, the hyperspectral image (HSI) often suffers from various types of noise, such as Gaussian noise, impulse noise, stripe noise, and deadlines, which can significantly degrade their quality. Although many data-driven methods have been proposed to deal with complex noise, few of them consider the structural characteristics of noise. This not only leads to a lack of interpretability but also results in poor performance when dealing with structural noise in practical applications. To address this issue, this article proposes KPInet, a convolutional neural network (CNN) driven by the structural knowledge of noise for HSI denoising. First and foremost, the knowledge optimization-driven module (KODM) utilizes the deep unrolling method to unfold a total variation (TV) algorithm that considers the structural characteristics of noise. This approach improves the network’s interpretability and results in better performance on structural noise, while maintaining the effect of removing Gaussian noise. Second, the statistical feature injection module (SFIM) extracts more features by utilizing spectral gradients, medians, and means of the HSI. Third, the multiscale degradation guidance module (MDGM) utilizes a dual-stream decoder with a low-resolution upsampling guidance branch to better distinguish the real structure and noise structure in the HSI. Experimental results on simulated and real datasets indicate that the approach achieves favorable denoising performance, as evidenced by both quantitative evaluation metrics and visual results. Furthermore, it also demonstrates the robustness and generalization capacity of the proposed KPInet.

Soil moisture (SM) downscaling aims to solve the coarse resolution problem of passive microwave SM products. On the basis of SMAP SM products and related MODIS products, this study develops a deep residual cycle generative adversarial network (GAN) based heterogeneous spatial-spectral fusion method to downscale SMAP SM from 36km to 9km. On the one hand, the proposed method creatively regards the MODIS products that can reflect the SM state as the spectral features of SM in a broad sense and performs the heterogeneous spatial-spectral fusion between the low-resolution (LR) SM product and high-resolution (HR) MODIS products. On the other hand, considering the spatial correlation of SM, the proposed method utilizes a deep residual cycle generative adversarial network (GAN) to extract and fuse features of heterogeneous images through convolutions. Both qualitative and quantitative evaluation of experimental results shows that the proposed method can generate high accuracy SM products.

Hyperspectral images(HSIs) captured from actual observations are inevitably interfered by different types of degradation, but existing methods are difficult to recover from corruptions with extensive coverage. To surmount these challenges, a hyperspectral image restoration method based on tensor av-erage rank and spatial-spectral total variation is proposed in this paper, which can be solved by the alternating direction method of multipliers (ADMM) algorithm. Extensive experi-ments on both synthetic and real-world data demonstrate that the model has a better ability to restore hyperspectral images from corruptions, such as deadlines and stripe noises.

In the field of remote sensing image pansharpening, deep learning-based methods have shown impressive performances recently. However, most deep learning-based pansharpening methods are based on supervised learning, which requires a large number of training images. In addition, obtaining large amounts of images with a high spatial and spectral resolution for training may be difficult in practice. In this letter, a novel self-supervised learning method based on a cycle-consistent generative adversarial network (CycleGAN) is proposed for remote sensing image pansharpening, without requiring large volumes of data for training. The framework contains two generators and two discriminators, and applies a residual neural network to the first generator. The panchromatic (PAN) image and multispectral (MS) image are input into the first generator to obtain the fused image, and then the fused image is input into the second generator to obtain a PAN image, which should be consistent with the input PAN image. The experimental results show that the proposed method performs better than the state-of-the-art unsupervised pansharpening method, and also achieves a competitive performance when compared with a supervised method.

Deep learning shows potential superiority in the image fusion field. To solve the problem of the spatial resolution degradation of polarimetric synthetic aperture radar (PolSAR) images caused by system limitation, we propose a fully PolSAR images and DualSAR images fusion network (FDFNet). We use low resolution (LR)-PolSAR super-resolution (LPSR) and modified cross attention mechanism (MCroAM) to perform data fusion on LR-PolSAR and high resolution (HR)-dual-polarization synthetic aperture radar (DualSAR) and design a polarimetric decomposition attention module to introduce the polarimetric parameters of LR-PolSAR images to maintain polarimetric information. Besides, we use the differential information between LR-PolSAR and HR-DualSAR to guide spatial resolution reconstruction. The loss function based on the$L_{1} $norm is used to constrain the network training process. The experimental results show the superiority of the proposed method over the existing methods in visual and quantitative evaluation. In addition, polarimetric decomposition experiments verify the effectiveness of the proposed method to maintain polarimetric information.

Multispectral images play a crucial role in environmental monitoring or ecological analysis for their large scope, quick acquisition, and big data. With the rapid development of technology and increasing demand, very high-resolution multispectral images have attracted a lot of attention these days. However, due to sensor equipment and the imaging environment, the spatial resolution of multispectral images is always restricted. With the help of panchromatic images, pan-sharpening is a very important technique to enhance the spatial details of multispectral images. In this study, we proposed a knowledge optimization-driven pan-sharpening network with normalizer-free group ResNet prior, called PNXnet, which is unfolded from a physical knowledge optimization-driven variational model. We solved the memory overhead brought by the traditional ResNet relying on batch normalization. Results on four sensors show that high quantitative indexes and natural visual effects have verified the reliability of PNXnet. Focusing on the NIR band where spatial details are hard to be injected, we compared the Normalized Difference Vegetation Index (NDVI) generated from the fused results, the estimated NDVI shows a high consistency to the ground truth with R2above 0.91. Besides, we also compared the model generation. Furthermore, low model complexity and quicker computational speed make the daily application of PNXnet possible.

It is a challenging task to integrate the spatial, temporal, and spectral information of multi-source remote sensing images, especially in the case of heterogeneous images. To this end, for the first time, this paper proposes a heterogeneous integrated framework based on a novel deep residual cycle generative adversarial network (GAN). The proposed network consists of a forward fusion part and a backward degeneration feedback part. The forward part generates the desired fusion result from the various observations; the backward degeneration feedback part considers the imaging degradation process and regenerates the observations inversely from the fusion result. The heterogeneous integrated fusion framework supported by the proposed network can simultaneously merge the complementary spatial, temporal, and spectral information of multi-source heterogeneous observations to achieve heterogeneous spatio-spectral fusion, spatio-temporal fusion, and heterogeneous spatio-temporal-spectral fusion. Furthermore, the proposed heterogeneous integrated fusion framework can be leveraged to relieve the two bottlenecks of land-cover change and thick cloud cover. Thus, the inapparent and unobserved variation trends of surface features, which are caused by the low-resolution imaging and cloud contamination, can be detected and reconstructed well. Images from many different remote sensing satellites, i.e., Moderate Resolution Imaging Spectroradiometer (MODIS), Landsat 8, Sentinel-1, and Sentinel-2, were utilized in the experiments conducted in this study, and both the qualitative and quantitative evaluations confirmed the effectiveness of the proposed image fusion method.

The data fusion technology aims to aggregate the characteristics of different data and to obtain products with multiple data advantages. To solve the problem of reduced resolution of polarimetric synthetic aperture radar (PolSAR) images due to system limitations, we propose a fully PolSAR images and single-polarization synthetic aperture radar (SinSAR) images fusion network to generate high-resolution PolSAR (HR-PolSAR) images. To take advantage of the polarimetric information of the low-resolution PolSAR (LR-PolSAR) images and the spatial information of the high-resolution single-polarization SAR (HR-SinSAR) images, we propose a fusion framework for joint LR-PolSAR images and HR-SinSAR images and design a cross-attention mechanism to extract features from the joint input data. Besides, based on the physical imaging mechanism, we designed the PolSAR polarimetric loss functions for constrained network training. The experimental results confirm the superiority of the fusion network over traditional algorithms. The average peak signal-to-noise ratio (PSNR) is increased by more than 3.6 dB, and the average mean absolute error (MAE) is reduced to less than 0.07. Experiments on polarimetric decomposition and polarimetric signature show that it maintains polarimetric information well.

Hyperspectral images (HSIs) are generally distorted by various types of damage and degradation due to limited imaging conditions. Hence, noise reduction is an essential process before HSI interpretations and applications. In this paper, a novel local-global feature-aware transformer based residual network (FATR) is proposed for hyperspectral image denoising. First, a spatial-spectral feature extraction module is built to extract spatial and spectral shallow features simultaneously. Second, these spatial-spectral features are forwarded to the deep feature extraction module, which contains several local-global feature-aware transformer blocks, where contextual information as well as local and global information can be further aggregated by multiscale windows transformer layers. Finally, in the reconstruction module, different hierarchical features from branches of two modules are merged into the final restoration to recover clean HSIs. Extensive experiments on both synthetic and real-world data demonstrate that the model has a better ability to restore HSIs in terms of evaluation metrics and visual assessments.

Enhancement of hyperspectral image (HSI) resolution is significant for better application in practice. In this article, a dual U-Net (D-UNet) is proposed to improve the spatial resolution of HSI. The whole network contains two parts. One is the detail extraction network, whose network architecture is encoder–decoder and mainly extracts various spatial features from multispectral images (MSIs). Another is the spatio-spectral fusion network (SSFN), which aims at injecting the features from the detail extraction network into HSI for better reconstruction. Furthermore, in the primary stage of the whole network, a novel multiscale spatio-spectral attention module (MSSAM) is utilized to pay more attention to important features at different scales. Considering the complex ground scenes, the features of different scale and depth are continually extracted and fused in the whole network. The experimental results show that the proposed method is more effective compared with the state-of-the-art methods.

Hyperspectral imagery (HSI) is one of the emerging tools to explore the physical properties and chemical composition of the lunar surface. Moon mineralogy mapper (M3) is the most widely used lunar HSI data set with the widest coverage and the excellent resolution; however, dense and nonperiodic stripes distributed across all bands in M3images hinder visual interpretation as well as their use in subsequent applications. In this article, a fast destriping approach for M3is proposed using the Hodrick–Prescott decomposition embedded in the low-rank framework (LRHP) to overcome this limitation. The integration of a statistical filter and variational model tackles the problem stemming from a lack of the correct residual information when certain pixels are corrupted in every band, thereby restoring severely degraded hyperspectral images (HSIs). Simulated and real experiments conducted on typical regions on the Moon with various levels of corruption demonstrate that the proposed LRHP rapidly achieves favorable performance against state-of-the-art approaches. Also, expanding tests on interference imaging spectrometer (IIM) data of Chang’E-1 and commonly used terrestrial remote sensing images show that LRHP has good generalization capability. Moreover, the integrated band depth (IBD) maps further verify the necessity of destriping and the high spectral fidelity of LRHP that benefits further applications.

Hyperspectral images (HSIs) are crucial for many research works. Spectral super-resolution (SSR) is a method used to obtain high-spatial-resolution (HR) HSIs from HR multispectral images. Traditional SSR methods include model-driven algorithms and deep learning. By unfolding a variational method, this article proposes an optimization-driven convolutional neural network (CNN) with a deep spatial-spectral prior, resulting in physically interpretable networks. Unlike the fully data-driven CNN, auxiliary spectral response function (SRF) is utilized to guide CNNs to group the bands with spectral relevance. In addition, the channel attention module (CAM) and the reformulated spectral angle mapper loss function are applied to achieve an effective reconstruction model. Finally, experiments on two types of data sets, including natural and remote sensing images, demonstrate the spectral enhancement effect of the proposed method, and also, the classification results on the remote sensing data set verified the validity of the information enhanced by the proposed method.

Image fusion is a powerful means to integrate complementary spatio-temporal-spectral information among multi-source remote sensing images. The existing remote sensing image fusion is mostly limited to the fusion between optical images, and most of them are limited to the fusion between two sensors. Based on this, this paper proposes a heterogeneous spatio-temporal-spectral fusion method based on deep learning. Specifically, it combines the low-spatial-resolution (LR) cloudy image with the high-spatial-resolution (HR) SAR images and the HR cloud-free optical image to remove the clouds and improve the spatial resolution of the LR cloudy image. The SAR image is acquired at the same date as the LR cloudy image, while the HR cloud-free image is acquired at another date. Experiments are performed on the images of Landsat 8, Sentinel-1, and Sentinel-2. The experimental results show that the proposed method can effectively achieve the joint goal of spatial resolution improvement and cloud removal of the Landsat image.

Three-dimensional (3D) convolution is well-suited for volumetric data exploration, and therefore it has great potential in spatial-spectral feature learning to promote hyperspectral image super-resolution (HSI SR). However, 3D convolution is computationally expensive, and this is especially true when it operates on the high spectral dimensionality. In this paper, we design the enhanced 3D (E3D) convolution, an efficient form of spatial-spectral convolution. The standard 3D convolution is factorized into sequential spatial and spectral components. And the novel lightweight spatial and spectral squeeze-and-excitation modules are incorporated to corresponding components, respectively. As such, E3D convolution can largely reduce the computational complexity and extract effective spatial-spectral features with the holistic information. We further construct a fully 3D convolutional network (E3DN) with the proposed E3D convolution. The additional global residual learning and share-source skip connections can achieve spectral mapping and facilitate feature propagation. The simulated and real experiments demonstrate the accuracy and performance advantages of E3DN.

It is very significant to enhance the spatial resolution of hyperspectral images for more accurate image interpretation. In this paper, we propose an innovative fusion method by the enhanced residual dense network to better extract the spatio-spectral features joint with GRU. The enhanced residual dense blocks (ERDB) contains a modified spatial and spectral attention module. All attention coefficients are calculated based on original images. Then, the idea of gate recurrent unit (GRU) is applied to integrate the useful information from results of all ERDBs followed by a convolution for reconstruction. Finally, a skip connection is added to further maintain the spectrum. The effectiveness of this method can be seen from the experimental results of simulation experiments.

Although unprecedented success has been achieved in convolutional neural network (CNN)-based super-resolution (SR) for natural images, hyperspectral image (HSI) SR without auxiliary high-resolution images remains a challenging task due to the high spectral dimensionality, where learning effective spatial and spectral representations is of great importance. In this article, we introduce a novel CNN-based HSI SR method, termed spectral grouping and attention-driven residual dense network (SGARDN) to facilitate the modeling of all spectral bands and focus on the exploration of spatial-spectral features. Considering the block characteristic of HSI, we employ group convolutions in and between groups composed of highly similar spectral bands at early stages to extract informative spatial features and avoid spectral disorder caused by normal convolution. To exploit spectral prior, a new spectral attention mechanism constructed by covariance statistics of features is designed to adaptively recalibrate features. We adapt the spectral attention for group convolutions to rescale grouping features with holistic spectral information. These two sequential operations called spectral grouping and integration module aim to extract effective shallow spatial-spectral features that are reused in the following layers. On the other hand, the residual dense block can better deal with spatial-spectral features by experimental comparison and hence is combined with the spectral attention to form a new basic building block for powerful feature expression and spectral correlation learning. The experimental results on synthesized and real-scenario HSIs demonstrate the feasibility and superiority of the proposed method over other state-of-the-art methods.

Synthetic aperture radar (SAR) images are inherently affected by speckle noise, for which deep learning-based methods have shown good potential. However, the deep learning-based methods proposed until now directly map low-quality images to high-quality images, and they are unable to characterize the priors for all the kinds of speckle images. The variational method is a classic model optimization approach that establishes the relationship between the clean and noisy images from the perspective of a probability distribution. Therefore, in this article, we propose the recursive deep convolutional neural network (CNN) prior model for SAR image despeckling (SAR-RDCP). First, the data-fitting term and regularization term of the SAR variational model are decoupled into two subproblems, i.e., a data-fitting block and a deep CNN prior block. The gradient descent algorithm is then used to solve the data-fitting block, and a predenoising residual channel attention network based on dilated convolution is used for the deep CNN prior block, which combines an end-to-end iterative optimization training. In the experiments undertaken in this study, the proposed model was compared with several state-of-the-art despeckling methods, obtaining better results in both the quantitative and qualitative evaluations.

In this paper, five spectral superresolution (SSR) algorithms are compared to verify the availability of SSR results as input data in classification. To enhance the spectral resolution, SSR algorithms are proposed to increase the channel number of multispectral images, which can be divided into model-driven and data-driven methods. To combine the advantage of these two types of algorithms, we proposed an optimization-inspired convolutional neural network (OCNN) by unfolding a traditional variational model. The proposed method combines data-driven training with model-driven optimization together to enhance the spectral resolution of high-resolution (HR) multispectral images (MSIs) to obtain HR hyperspectral images (HSIs). Experiments in both SSR and classification are made to show the proposed method is of efficiency and superiority.

In this paper, we present a novel hyperspectral image (HSI) denoising method especially for mixed noise removal. The proposed method combines both data-driven with model-driven strategy via a deep spatio-spectral variational structure. The mixed noise estimation and removal are collaboratively derived through fusing the Bayesian spatio-spectral posterior and deep learning model. The framework can both utilize the logicality of traditional model-driven methods, and the high efficiency of data-driven methods for parameters optimizing. Simulated and actual experiments demonstrate that the presented method outperforms other existing methods for HSI mixed noise removal, on both reconstructing effects and time-consuming.

Compared with traditional remote sensing images, satellite remote sensing video contains more useful information and can capture continuous dynamic video. Recently, many deep-learning based methods have been proposed for video super resolution. However, these methods tend to ignore the structural information and characteristics for video satellite imagery such as small ground targets, a wide range of scales and weak textures. To this end, this paper proposes a single-and-multi-frame ensembled framework called SMFE for remote sensing videos super-resolution. The SMFE framework combines a non-local based single image super resolution (SISR) network and a state-of-the-arts multi-frame super resolution (MFSR) network EDVR. Experiments have been performed to demonstrate the effectiveness of the proposed method on Jilin-1.

The lunar hyperspectral remote sensing is one of the most important means to understand the physical properties and chemical constituents of lunar surface materials. Moon Mineral Mapper (M3) is currently the only hyperspectral image (HSI) data of Moon. However, due to the limitations of sensor manufacture and the impact of complex extraterrestrial environment, there are serious stripes on the M3 images, which do harm to subsequent identifications and analysis. In this paper, an effective destriping algorithm for lunar HSIs based on the intrinsic characteristics of the stripes on M3 is proposed. Experimental results demonstrate that our method shows an improvement in terms of visual perception and spectral fidelity.

Remote sensing image super-resolution has always been a major research focus, and many deep-learning-based algorithms have been proposed in recent years. However, since the structure of remote sensing images tends to be much more complex than that of natural images, several difficulties still remain for remote sensing images super-resolution. First, it is difficult to depict the nonlinear mapping between high-resolution (HR) and low-resolution (LR) images of different scenes with the same model. Second, the wide range of scales within the ground objects in remote sensing images makes it difficult for single-scale convolution to effectively extract features of various scales. To address the above-mentioned issues, we propose a multiscale attention network (MSAN) to extract the multilevel features of remote sensing images. The basic component of MSAN is the multiscale activation feature fusion block (MAFB). In addition, a scene-adaptive super-resolution strategy for remote sensing images is employed to more accurately describe the structural characteristics of different scenes. The experiments undertaken on several data sets confirm that the proposed algorithm outperforms the other state-of-the-art algorithms, in both evaluation indices and visual results.

In this work, a progressively spatio-temporal patch group learning framework for cloud and shadow removal in Sentinel-2 data is proposed. Through sorting the spatial and corresponding multi-temporal patches with masks as the patch group fashion, a spatiotemporal patch group recovering model is developed using a global-local deep CNN. Finally, all the ergodic patches are weighted aggregated with integrity measure, then updated spatial data and its mask are regenerated through progressive iteration. Two experiments have been performed to demonstrate the effectiveness of the proposed method on Sentienl-2 MSI data, with single/multiple temporal imageries in small and largescale scenarios.

In this paper, a new pansharpening method with residual convolutional neural network (RCNN)) is proposed. The proposed method utilizes a novel end-to-end CNN, which maps the differential information between the high spatial resolution panchromatic image (HR-PAN) and the low spatial resolution multispectral image (LR-MS) to the differential information between the HR-PAN image and the high spatial resolution multispectral image (HR-MS). Unlike the CNN-based pansharpening methods in other literatures, the proposed method makes full use of the spatial information in the HR-PAN image, and simultaneously preserve the spectral information of the MS image. Experimental results at both reduced resolution and full resolution demonstrate the superior performance of the proposed method comparing to state-of-the-art pansharpening methods in both quantitative and visual assessments.

In order to solve the problem of full-polarimetric SAR image degradation, this paper proposes a full-polarimetric SAR image super-resolution reconstruction method combined with a convolutional neural network and residual compensation. Through the advantages of the deep convolutional neural network for nonlinear model fitting, this paper performs super-resolution reconstruction on low-resolution full-polarimetric SAR images, and then applies residual compensation to network reconstruction results, using low-resolution image information to the network. The super-resolution reconstruction results are corrected to obtain a high-resolution full-polarimetric SAR image. Compared with the traditional full-polarimetric SAR image super-resolution reconstruction method, the proposed method shows excellent results in both visual and quantitative evaluation indicators, especially the reconstruction of detailed information.

Enhancing the spatial resolution of hyperspectral (HS) images by fusing with higher spatial resolution multispectral (MS) data is of significance for applications. However, due to the narrow bandwidth, HS images (HSIs) are vulnerable to various types of noise, such as Gaussian noise and stripes, which can severely affect the fusion performance. This paper focuses on antinoise HS and MS image fusion to enhance the spatial details and suppress the noise. By analysis of the intrinsic structure and noise properties, we formulate this problem as the minimization of an objective function. Under the optimization framework, small multilinear ranks in tensor are first used to identify the intrinsic structures of the clean HSI part. Then, considering the high spectral correlation, it is assumed that any bands can be represented by the combination of certain adjacent bands. The difference between one band and its corresponding combination can be used to preserve the spatio-spectral consistency and characterize the distribution of sparse noise (such as stripe noise), based on the variational properties along two directions. The alternating direction method of multipliers (ADMM) is applied to solve and accelerate the model optimization. Experiments with both simulated- and real-data demonstrate the effectiveness of the proposed model and its robustness to the noise, in terms of both qualitative and quantitative perspectives.

In the field of spatial–spectral fusion, the variational model-based methods and the deep learning (DL)-based methods are state-of-the-art approaches. This paper presents a fusion method that combines the deep neural network with a variational model for the most common case of spatial–spectral fusion: panchromatic (PAN)/multispectral (MS) fusion. Specifically, a deep residual convolutional neural network (CNN) is first trained to learn the gradient features of the high spatial resolution multispectral image (HR-MS). The image observation variational models are then formulated to describe the relationships of the ideal fused image, the observed low spatial resolution multispectral image (LR-MS) image, and the gradient priors learned before. Then, fusion result can then be obtained by solving the fusion variational model. Both quantitative and visual assessments on high-quality images from various sources demonstrate that the proposed fusion method is superior to all the mainstream algorithms included in the comparison, in terms of overall fusion accuracy.

Hyperspectral image (HSI) denoising is a crucial preprocessing procedure to improve the performance of the subsequent HSI interpretation and applications. In this paper, a novel deep learning-based method for this task is proposed, by learning a nonlinear end-to-end mapping between the noisy and clean HSIs with a combined spatial-spectral deep convolutional neural network (HSID-CNN). Both the spatial and spectral information are simultaneously assigned to the proposed network. In addition, multiscale feature extraction and multilevel feature representation are, respectively, employed to capture both the multiscale spatial-spectral feature and fuse different feature representations for the final restoration. The simulated and real-data experiments demonstrate that the proposed HSID-CNN outperforms many of the mainstream methods in both the quantitative evaluation indexes, visual effects, and HSI classification accuracy.

The existence of hybrid noise in hyperspectral images (HSIs) severely degrades the data quality, reduces the interpretation accuracy of HSIs, and restricts the subsequent HSI applications. In this paper, the spatial-spectral gradient network (SSGN) is presented for mixed noise removal in HSIs. The proposed method employs a spatial-spectral gradient learning strategy, in consideration of the unique spatial structure directionality of sparse noise and spectral differences with additional complementary information for effectively extracting intrinsic and deep features of HSIs. Based on a fully cascaded multiscale convolutional network, SSGN can simultaneously deal with different types of noise in different HSIs or spectra by the use of the same model. The simulated and real-data experiments undertaken in this study confirmed that the proposed SSGN outperforms at mixed noise removal compared with the other state-of-the-art HSI denoising algorithms, in evaluation indices, visual assessments, and time consumption.

Hyperspectral remote sensing could acquire hundreds of bands to cover a complete spectral interval, which deliver more information and allow a whole range of new and more precise applications. But vast data volume can cause trouble in computer processing and data transmission. Too many bands may cause interference for image processing and endmember variability is inevitable in hyperspectral data, which will affect the accuracy of interpretation. Band selection for hyperspectral image data is an effective way to mitigate the curse of dimensionality. In this paper, one hyperspectral band selection method based on endmember dissimilarity is proposed. This method used Mahalanobis distance as class separability criterion, and the spectral signature for each class is proposed by endmember extraction method automatically. Experiments on both synthetic and real hyperspectral data sets indicate that the proposed method outperformed the Minimum Estimated Abundance Covariance (MEAC) and Uniform Spectral Spacing (USS) method.

Missing data reconstruction (e.g., dead pixel repair and cloud removing) in remote sensing images is a very important problem for the subsequent image analysis. It is well-known that missing data reconstruction is an ill-posed problem. In remote sensing images, there is a strong correlation in spectral frequencies or in temporal frames, and also there are a lot of self-similarity patterns in spatial domain. We can make use of these properties to derive low rank matrices according to their spectral, temporal and spatial dimensions. In this paper, we propose a tensor completion model based on these low rank matrices to deal with missing data reconstruction problem. We also present a weighting method for spectral, temporal and spatial dimensions and for their distribution of singular values. Our experimental results demonstrate that the weighting method can recover remote images very well. In particular, we show the effectiveness of the proposed method for both simulated and real data sets, and the performance of the proposed in terms of visual and quantitative measures is better than those of the other testing methods.

Remote sensing images are widely used in various fields. However, they are usually degraded by haze. In this paper, a spatial-spectral adaptive haze removal method for remote sensing images is proposed. The proposed method is based on haze imaging model and the dark channel prior. Our method is able to diminish the phenomenon of color distortion and haze removed not completely in different bands. The experimental results verify that the proposed method can remove the haze completely and yield visually haze free images, even the haze distribution is uneven.

Hyperspectral image super resolution (SR) reconstruction has been studied widely and many algorithms have been proposed. In this paper, a novel super resolution reconstruction method was designed by employing a joint spectral-spatial sub-pixel mapping model which aims to obtain the probabilities of sub-pixels to belong to different land cover classes by dividing mixed pixels into several sub-pixels. Given these sub-pixel probabilities, the resolution enhanced image can be further generated. The proposed approach has been evaluated using both synthetic and real hyperspectral images and compared with other well-known methods. The visual and quantitative comparisons confirm the effectiveness of the proposed method.

Due to the limitation of hyperspectral sensors and optical imaging systems, there are several irreconcilable conflicts between high spatial resolution and high spectral resolution of hyperspectral images (HSIs). Therefore, HSI super-resolution (SR) is regarded as an important preprocessing task for subsequent applications. In this letter, we use sparse representation to analyze the spectral and spatial feature of HSIs. Considering the sparse characteristic of spectral unmixing and high pattern repeatability of spatial-spectral blocks, we proposed a novel HSI SR framework utilizing spectral mixture analysis and spatial-spectral group sparsity. By simultaneously combining the sparsity and the nonlocal self-similarity of the images in the spatial and spectral domains, the method not only maintains the spectral consistency but also produces plenty of image details. Experiments on three hyperspectral data sets confirm that the proposed method is robust to noise and achieves better results than traditional methods.

Hyperspectral image (HSI) denoising is a crucial preprocessing task that is used to improve the quality of images for object detection, classification, and other subsequent applications. It has been reported that noise can be effectively removed using the sparsity in the nonnoise part of the image. With the appreciable redundancy and correlation in HSIs, the denoising performance can be greatly improved if this redundancy and correlation is utilized efficiently in the denoising process. Inspired by this observation, a noise reduction method based on joint spectral-spatial distributed sparse representation is proposed for HSIs, which exploits the intraband structure and the interband correlation in the process of joint sparse representation and joint dictionary learning. In joint spectral-spatial sparse coding, the interband correlation is exploited to capture the similar structure and maintain the spectral continuity. The intraband structure is utilized to adaptively code the spatial structure differences of the different bands. Furthermore, using a joint dictionary learning algorithm, we obtain a dictionary that simultaneously describes the content of the different bands. Experiments on both synthetic and real hyperspectral data show that the proposed method can obtain better results than the other classic methods.

Hyperspectral images (HSIs) provide abundant information to solve various kinds of problems like object identification and classification. However, HSIs often inevitably suffer many factors from various resources [1], such as imperfect imaging optics, sensor noise, and atmospheric effects, which degrade the acquired image quality [2]. Thus, HSI image super resolution reconstruction, used to achieve sub-pixel mapping, is an active research topic due to its effectiveness in improving the resolution of hyperspectral image. In the paper, a HSI super-resolution method, in which the different dictionaries are learnt for different bands and sparse structure from wavelength range with high correlation is regarded with similar sparse coefficients , is proposed to obtain the high-resolution image.

The sub-pixel mapping technique, which can provide a fine-resolution map of class labels, has attracted more and more attention in recent years. Generally speaking, there are two kinds of methods used to realize the sub-pixel labeling. The first kind are image reconstruction based methods, which first improve the spatial resolution of an image by the super-resolution technique, and then perform a hard classification on the super-resolved image. The second kind are pixel unmixing based methods, where the sub-pixel mapping is implemented based on the results of image unmixing. In this paper, we present a sparse representation method and a back-propagation (BP) neural network method for image reconstruction based and pixel unmixing based mapping, respectively. The advantages and disadvantages of both kinds of methods are analyzed and discussed.