Deep learning methods for pansharpening have advanced rapidly, yet models pretrained on data from a specific sensor often generalize poorly to data from other sensors. Existing methods to tackle such cross-sensor degradation include retraining model or zero-shot methods, but they are highly time-consuming or even need extra training data. To address these challenges, our method first performs modular decomposition on deep learning-based pansharpening models, revealing a general yet critical interface where high-dimensional fused features begin mapping to the channel space of the final image. % may need revisement A Feature Tailor is then integrated at this interface to address cross-sensor degradation at the feature level, and is trained efficiently with physics-aware unsupervised losses. Moreover, our method operates in a patch-wise manner, training on partial patches and performing parallel inference on all patches to boost efficiency. Our method offer two key advantages: (1) Improved Generalization Ability: it significantly enhance performance in cross-sensor cases. (2) Low Generalization Cost: it achieves sub-second training and inference, requiring only partial test inputs and no external data, whereas prior methods often take minutes or even hours. Experiments on the real-world data from multiple datasets demonstrate that our method achieves state-of-the-art quality and efficiency in tackling cross-sensor degradation. For example, training and inference of 512 times 512 times 8 image within 0.2 seconds and 4000 times 4000 times 8 image within 3 seconds at the fastest setting on a commonly used RTX 3090 GPU, which is over 100 times faster than zero-shot methods.

Hyperspectral pansharpening refers to fusing a panchromatic image (PAN) and a low-resolution hyperspectral image (LR-HSI) to obtain a high-resolution hyperspectral image (HR-HSI). Recently, guiding pre-trained diffusion models (DMs) has demonstrated significant potential in this area, leveraging their powerful representational abilities while avoiding complex training processes. However, these DMs are often trained on RGB images, not well-suited for pansharpening tasks, limited in adapting to the hyperspectral images. In this work, we propose a novel guided diffusion scheme with zero-shot guidance and neural spatialspectral decomposition (NSSD) to iteratively generate the RGB detail image and map the RGB detail image to target HR-HSI. Specifically, zero-shot guidance employs an auxiliary neural network that trained only with a PAN and LR-HSI to guide pre-trained DMs in generating the RGB detail image, informed by specific prior knowledge. Then, NSSD establishes a spectral mapping from the generated RGB detail image to the final HR-HSI. Extensive experiments are conducted on Pavia, Washington DC, Chukusei, and FR1 datasets to demonstrate that the proposed method significantly enhances the performance of DMs for hyperspectral pansharpening tasks, outperforming existing methods across multiple metrics and achieving improvements in visualization results. The code is available at https://github.com/Jin-liangXiao/DM-zs.

Pansharpening is an image fusion technique that fuses the high spatial resolution of panchromatic images (PAN) and the rich spectral information of multispectral images (MS) to produce high resolution multispectral images (HRMS). The preservation of spatial details is crucial for enhancing the quality of the final results. However, existing detail extraction methods often fail to capture spatial information effectively. Most approaches rely only on internal details from the PAN image while overlooking external information, such as the deep-driven prior. Additionally, they struggle to establish an accurate relationship between the HRMS and PAN images, leading to spatial distortions. To address these issues, in this article, we propose a novel variational model based on double detail injection. Specifically, it integrates internal details from an adaptive spatial fidelity term and external details from a deep-driven injection term. Furthermore, an alternating direction method of multipliers (ADMM)-based algorithm is developed to efficiently solve the proposed model. The effectiveness of the proposed method is demonstrated through extensive experiments, showing superior performance compared to some existing pansharpening techniques.

Hyperspectral unmixing has become a valuable research area in recent years. As significant characteristics of hyperspectral images (HSIs), low-rankness and sparsity have been widely studied to improve the accuracy of abundance estimation for spectral unmixing. However, most of the existing models perform the low-rank and sparse constraints on the entire abundance matrix at the same time, ignoring that low-rankness is often only caused by a few active endmembers. In this paper, we propose a simple but effective method to separate endmembers that contribute more to low-rank property from the given spectral dictionary, and then exploit the weighted nuclear norm on their corresponding abundance maps to enhance the low-rankness. In addition, to make full use of sparsity, both spectral and spatial weighted factors are considered in the ℓ1-norm to constrain abundances of all endmembers. The proposed algorithm is based on the alternating direction method of multipliers (ADMM) framework. Simulated and real-data experiments demonstrate the effectiveness of the resulting unmixing algorithm.

Pansharpening aims to obtain the high resolution multispectral image (HRMS) using the panchromatic image (PAN) and low spatial resolution multispectral image (LRMS). The similarity between PAN and HRMS has shown powerful performance for spatial feature extraction. The prevailing methods usually describe the similarity on a fixed transformed domain. However, such domain, e.g., gradient domain, usually limits the preservation of spatial details and neglects flexibility. To overcome these challenges, we propose an adaptive transformed domain-based spatial fidelity to depict the similarity accurately and flexibly. Based on the proposed spatial fidelity, we build a novel variational pansharpening model that consists of spectral and spatial fidelity terms. We design an algorithm based on the alternating direction method of multiplier (ADMM) framework to solve the model. Experimental results on reduced- and full-resolution data verify the effectiveness of the proposed method.

Hyperspectral and multispectral image fusion, denoted as HSI-MSI fusion, involves merging a pair of hyperspectral (HSI) and multispectral (MSI) images to generate a high spatial resolution hyperspectral image (HR-HSI). The primary challenge in HSI-MSI fusion is to find the best way to extract one-dimensional spectral features and two-dimensional (2-D) spatial features from HSI and MSI and harmoniously combine them. In recent times, coupled tensor decomposition (CTD)-based methods have shown promising performance in the fusion task. However, the tensor decompositions (TDs) used by these CTD-based methods face difficulties in extracting complex features and capturing 2-D spatial features, resulting in suboptimal fusion results. To address these issues, we introduce a novel method called Coupled Tensor Double-Factor Decomposition (CTDF). Specifically, we propose a Tensor Double-Factor (TDF) decomposition, representing a 3rd-order HR-HSI as a 4th-order spatial factor and a 3rd-order spectral factor, connected through tensor contraction. Compared to other TDs, the TDF has better feature extraction capability since it has a higher order factor than that of HR-HSI, whereas the other TDs only have the same order factor as the HR-HSI. Moreover, the TDF can extract 2-D spatial features using the 4th-order spatial factor. We apply the TDF to the HSI-MSI fusion problem and formulate the CTDF model. Furthermore, we design an algorithm based on proximal alternating minimization to solve this model and provide insights into its computational complexity and convergence analysis. The simulated and real experiments validate the effectiveness and efficiency of the proposed CTDF method. The code is available at https://github.com/tingxu113/CTDF.

Pansharpening (which stands for panchromatic sharpening) involves the fusion between a multispectral (MS) image with a higher spectral content than a fine spatial resolution panchromatic (PAN) image to generate a high spatial resolution multispectral (HRMS) image. A widely-used concept is the construction of the relationship between PAN and HRMS images by designing pixel-based coefficients. Previous pixel-based methods compute the coefficients pixel-by-pixel while suffering from inaccuracies in some areas leading to spatial distortion. However, we found that the coefficients inherit the spatial properties of the HRMS image, e.g., the local smoothness and nonlocal self-similarity, and the spatial correlation between the coefficients and the HRMS image can increase the accuracy of the estimation process. In this article, we propose a novel spatial fidelity with nonlocal regression (SFNLR) to describe the relationship between PAN and HRMS images. Unlike from the pixel-based perspective, the SFNLR can jointly utilize the local smoothness and nonlocal self-similarity of the coefficients for preserving spatial information. Besides, the SFNLR is integrated with a widely-used spectral fidelity to formulate a new variational model for the pansharpening problem. An effective algorithm based on the alternating direction method of multiplier (ADMM) framework is designed to solve the proposed model. Qualitative and quantitative assessments on reduced and full resolution datasets from different satellites demonstrate that the proposed approach outperforms several state-of-the-art methods. The code is available at: https://github.com/Jin-liangXiao/SFNLR.

Pansharpening is related to the fusion of a low spatial resolution multispectral (MS) image retaining an abundant spectral content and a high spatial resolution panchromatic (PAN) image to obtain a product with both the abundant spectral content of the former and the high spatial resolution of the latter. Many previous studies are only focused on the global or local relationship between the PAN image and the corresponding high-resolution multispectral (HRMS) image. However, we found that the relationship between PAN and HRMS images in the gradient domain can be better explored through the image context. In this article, we propose context-aware details injection fidelity (CDIF) with adaptive coefficients estimation, which can fully explore the complicated relationship between the PAN image and the HRMS image in the gradient domain. More specifically, we apply a clustering method to divide the pixels of an image into different context-based regions. Afterward, the adaptive coefficients are estimated by using a regression-based method for each region. The CDIF is effective in extracting the main features from the two inputs to be fused. In addition, we integrate the CDIF with a conventional fidelity term and a total variation regularization to formulate a novel variational pansharpening model that is solved by designing an algorithm based on the alternating direction method of multiplier (ADMM) framework. Qualitative and quantitative assessments on different datasets support the effectiveness and robustness of the proposed method. The code is available athttps://github.com/liangjiandeng/CDIF.