Recently, tensor network (TN) decompositions have gained prominence in computer vision and contributed promising results to tensor recovery for their capability of compactly and efficiently representing high-order tensors. However, current TN topologies are rather being developed towards more intricate structures to pursue incremental improvements, resulting in a drastically increased number of TN ranks, which requires laborious hyper-parameter selection, especially for higher-order cases. In this paper, we propose a novel TN decomposition, dubbed tensor wheel (TW) decomposition, in which a high-order tensor is represented by a set of latent factors mapped into a specific wheel topology. Such a decomposition is constructed starting from analyzing the graph structure, aiming to more accurately characterize the complex interactions inside objectives while maintaining a lower hyper-parameter scale, theoretically alleviating the above deficiencies. The comprehensive analysis of the mathematical properties fully demonstrates that TW decomposition can be more potential in representation capabilities and more flexible in controlling both parameter storage and computational costs. To compute the TW-format decomposition, the sequential singular value decomposition (SVD)-based and the alternating least squares (ALS)-based learning algorithms are developed. Furthermore, to investigate the validity of TW decomposition, we provide its one numerical application, i.e., tensor completion (TC), yet develop an efficient proximal alternating minimization-based solving algorithm with guaranteed convergence. Experimental results on both synthetic and real-world data reveal that TW decomposition significantly outperforms other state-of-the-art tensor decompositions for incomplete-tensor inference, especially under solely few observations, thus substantiating the superiority and reliability of TW decomposition.

Pansharpening is an image fusion technique that fuses the high spatial resolution of panchromatic images (PAN) and the rich spectral information of multispectral images (MS) to produce high resolution multispectral images (HRMS). The preservation of spatial details is crucial for enhancing the quality of the final results. However, existing detail extraction methods often fail to capture spatial information effectively. Most approaches rely only on internal details from the PAN image while overlooking external information, such as the deep-driven prior. Additionally, they struggle to establish an accurate relationship between the HRMS and PAN images, leading to spatial distortions. To address these issues, in this article, we propose a novel variational model based on double detail injection. Specifically, it integrates internal details from an adaptive spatial fidelity term and external details from a deep-driven injection term. Furthermore, an alternating direction method of multipliers (ADMM)-based algorithm is developed to efficiently solve the proposed model. The effectiveness of the proposed method is demonstrated through extensive experiments, showing superior performance compared to some existing pansharpening techniques.

Recently, implicit neural representations (INR) have made significant strides in various vision-related domains, providing a novel solution for Multispectral and Hyperspectral Image Fusion (MHIF) tasks. However, INR is prone to losing high-frequency information and is confined to the lack of global perceptual capabilities. To address these issues, this paper introduces a Fourier-enhanced Implicit Neural Fusion Network (FeINFN) specifically designed for MHIF task, targeting the following phenomena: The Fourier amplitudes of the HR-HSI latent code and LR-HSI are remarkably similar; however, their phases exhibit different patterns. In FeINFN, we innovatively propose a spatial and frequency implicit fusion function (Spa-Fre IFF), helping INR capture high-frequency information and expanding the receptive field. Besides, a new decoder employing a complex Gabor wavelet activation function, called Spatial-Frequency Interactive Decoder (SFID), is invented to enhance the interaction of INR features. Especially, we further theoretically prove that the Gabor wavelet activation possesses a time-frequency tightness property that favors learning the optimal bandwidths in the decoder. Experiments on two benchmark MHIF datasets verify the state-of-the-art (SOTA) performance of the proposed method, both visually and quantitatively. Also, ablation studies demonstrate the mentioned contributions. The code can be available at https://github.com/294coder/Efficient-MIF.

Pansharpening is a significant image fusion technique that merges the spatial content and spectral characteristics of remote sensing images to generate high-resolution multispectral images. Recently, denoising diffusion probabilistic models have been gradually applied to visual tasks, enhancing controllable image generation through low-rank adaptation (LoRA). In this paper, we introduce a spatial-spectral integrated diffusion model for the remote sensing pansharpening task, called SSDiff, which considers the pansharpening process as the fusion process of spatial and spectral components from the perspective of subspace decomposition. Specifically, SSDiff utilizes spatial and spectral branches to learn spatial details and spectral features separately, then employs a designed alternating projection fusion module (APFM) to accomplish the fusion. Furthermore, we propose a frequency modulation inter-branch module (FMIM) to modulate the frequency distribution between branches. The two components of SSDiff can perform favorably against the APFM when utilizing a LoRA-like branch-wise alternative fine-tuning method. It refines SSDiff to capture component-discriminating features more sufficiently. Finally, extensive experiments on four commonly used datasets, i.e., WorldView-3, WorldView-2, GaoFen-2, and QuickBird, demonstrate the superiority of SSDiff both visually and quantitatively. The code is available at https://github.com/Z-ypnos/SSdiff_main.

In this paper, we propose an ℓ0-based nonconvex optimization model with overlapping group sparse hyper-Laplacian prior (ℓ0-OGSHL) to remove stripes from remote sensing images (RSIs) effectively. Specifically, we utilize the hyper-Laplacian prior with overlapping group sparsity (OGSHL) to characterize the properties of the underlying image. Additionally, the related ℓ0-quasi equivalent is transformed into an easily solvable form by employing a mathematical program with equilibrium constraints (MPEC). Furthermore, the alternating direction method of multipliers (ADMM) algorithm is employed for resolving the equivalent nonconvex optimization model, and the complex OGSHL subproblem is addressed through the majorization-minimization (MM) method. Finally, the experimental results on the simulated datasets conclusively demonstrate the superior performance of the proposed method over the compared methods (with 1 3dB higher MPSNR), both quantitatively and visually. The code will be available after possible acceptance.

Pansharpening aims to enhance remote sensing image (RSI) quality by merging high-resolution panchromatic (PAN) with multispectral (MS) images. However, prior techniques struggled to optimally fuse PAN and MS images for enhanced spatial and spectral information, due to a lack of a systematic framework capable of effectively coordinating their individual strengths. In response, we present the cross modulation Transformer (CMT), a pioneering method that modifies the attention mechanism. This approach utilizes a robust modulation technique from signal processing, integrating it into the attention mechanism’s calculations. It dynamically tunes the weights of the carrier’s value (V) matrix according to the modulator’s features, thus resolving historical challenges and achieving a seamless integration of spatial and spectral attributes. Furthermore, considering that RSI exhibit large-scale features and edge details along with local textures, we crafted a hybrid loss function that combines Fourier and wavelet transforms to effectively capture these characteristics, thereby enhancing both spatial and spectral accuracy in pansharpening. Extensive experiments demonstrate our framework’s superior performance over existing state-of-the-art methods. The source code is publicly available athttps://github.com/WenjieShu/CMT.

In this paper, we propose a novel dynamic fusion model (DFM) with joint variational and deep priors for the task of hyperspectral image super-resolution (HISR). The given model can benefit from both the advantages of traditional modeling and deep learning methods, thus achieving significant improvements based on existing deep pre-trained models. Specifically, the given model mainly contains two new designed terms, i.e., the weighted spatial fidelity (WSF) term and the deep fusion (DF) term. The WSF term focuses on the spatial recovery of the low-resolution hyperspectral image through the high-resolution multispectral image without the knowledge of the spectral response matrix, thus the proposed DFM can be viewed as a semi-blind model for HISR. Moreover, the DF term relied upon deep fusion with a designed adaptive weight matrix, which can effectively inject the deep priors into the traditional minimization model. Besides, the proposed DFM can be quickly and effectively solved using the alternating direction method of multipliers. Experimental results on widely used datasets demonstrate the superiority of our approach compared with state-of-the-art HISR methods.

Recently, tensor network (TN) decompositions have gained prominence in computer vision and contributed promising results to high-order data recovery tasks. However, current TN models are rather being developed towards more intricate structures to pursue incremental improvements, which instead leads to a dramatic increase in rank numbers, thus encountering laborious hyper-parameter selection, especially for higher-order cases. In this paper, we propose a novel TN decomposition, dubbed tensor wheel (TW) decomposition, in which a high-order tensor is represented by a set of latent factors mapped into a specific wheel topology. Such decomposition is constructed starting from analyzing the graph structure, aiming to more accurately characterize the complex interactions inside objectives while maintaining a lower hyper-parameter scale, theoretically alleviating the above deficiencies. Furthermore, to investigate the potentiality of TW decomposition, we provide its one numerical application, i.e., tensor completion (TC), yet develop an efficient proximal alternating minimization-based solving algorithm with guaranteed convergence. Experimental results elaborate that the proposed method is significantly superior to other tensor decomposition-based state-of-the-art methods on synthetic and real-world data, implying the merits of TW decomposition. The code is available at: https://github.com/zhongchengwu/code_TWDec.

Hyperspectral image super-resolution (HISR) is to fuse a low-resolution hyperspectral image (LR-HSI) and a high-resolution multispectral image (HR-MSI), aiming to obtain a high-resolution hyperspectral image (HR-HSI). Recently, various convolution neural network (CNN) based techniques have been successfully applied to address the HISR problem. However, these methods generally only consider the relation of a local neighborhood by convolution kernels with a limited receptive field, thus ignoring the global relationship in a feature map. In this paper, we design a transformer-based architecture (called Fusformer) for the HISR problem, which is the first attempt to apply the transformer architecture to this task to the best of our knowledge. Thanks to the excellent ability of feature representations, especially by the self-attention in the transformer, our approach can globally explore the intrinsic relationship within features. Considering the specific HISR problem, since the LR-HSI holds the primary spectral information, our method estimates the spatial residual between the upsampled LR-MSI and the desired HR-HSI, reducing the burden of training the whole data in a smaller mapping space. Various experiments show that our approach outperforms current state-of-the-art HISR methods. The code is available at https://github.com/J-FHu/Fusformer.

In this paper, we mainly focus on a quite challenging denoising problem in remote sensing images, which is to simultaneously remove Gaussian noise and sparse noise that mainly include stripes and salt-pepper noise. We propose a convex unidirectional sparse model based on mode-3 tensor modeling to remove the mixture noise. A proximal alternating direction method of multipliers (ADMM) based algorithm is designed to effectively solve the given minimization model. Comparing with some recent state-of-the-art denoising methods, the proposed method shows the best performance from visual and quantitative aspects.

In this paper, a pan-sharpening approach based on RoG filtering is proposed. This approach follows the framework of classic methods of pan-sharpening, i.e., component substitution and multi-resolution analysis. The filtering technique based on Relativity-of-Gaussian (RoG) regularization is first used in the process of upsampling the original multi-spectral image, and then in the detail extraction phase to obtain spatial details from the panchromatic image. Experiments on datasets acquired by Quickbird and IKONOS demonstrate that the proposed approach obtains competitive performance comparing with several popular pan-sharpening methods.

This paper commits to remove the stripe noise to enhance the visual quality of remote sensing images, in the meanwhile preserves image details of stripe-free regions. Instead of solving the underlying image as most of researches, we propose a non-convex l0model for remote sensing image destriping by taking full consideration of the intrinsically directional and structural priors of stripe noise. Moreover, the proposed non-convex model can be solved by the proximal alternating direction method of multipliers (PADMM) method which theoretically guarantees converging to a KKT point. Extensively experimental results on simulated and real data demonstrate that the proposed method outperforms recent state-of-the-art destriping methods, both visually and quantitatively.

In this paper, we develop a sparsity based model for the fusion of a high spatial-resolution image and a multispectral image. The given model is based on the combination of a dynamic gradient sparsity (DGS) and an anisotropic spectral-spatial total variation (ASSTV). We design an alternating direction method of multipliers (ADMM) based algorithm to solve the proposed model. In contrast to existing approaches, the proposed method can generate more spatial details as well as preserve favorable spectral information. Experimental results demonstrate that the proposed approach outperforms several state-of-the-art image fusion methods both quantitatively and visually, in terms of both pansharpening application of remote sensing images and fusion application of natural color images.