Pansharpening aims to combine a high-resolution panchromatic (PAN) image with a low-resolution multispectral (LRMS) image to produce a high-resolution multispectral (HRMS) image. Although pansharpening in the frequency domain offers clear advantages, most existing methods either continue to operate solely in the spatial domain or fail to fully exploit the benefits of the frequency domain. To address this issue, we innovatively propose Multi-Frequency Fusion Attention (MFFA), which leverages wavelet transforms to cleanly separate frequencies and enable lossless reconstruction across different frequency domains. Then, we generate Frequency-Query, Spatial-Key, and Fusion-Value based on the physical meanings represented by different features, which enables a more effective capture of specific information in the frequency domain. Additionally, we focus on the preservation of frequency features across different operations. On a broader level, our network employs a wavelet pyramid to progressively fuse information across multiple scales. Compared to previous frequency domain approaches, our network better prevents confusion and loss of different frequency features during the fusion process. Quantitative and qualitative experiments on multiple datasets demonstrate that our method outperforms existing approaches and shows significant generalization capabilities for real-world scenarios.

Currently, deep learning-based methods for remote sensing pansharpening have advanced rapidly. However, many existing methods struggle to fully leverage feature heterogeneity and redundancy, thereby limiting their effectiveness. We use the covariance matrix to model the feature heterogeneity and redundancy and propose Correlation-Aware Covariance Weighting (CACW) to adjust them. CACW captures these correlations through the covariance matrix, which is then processed by a nonlinear function to generate weights for adjustment. Building upon CACW, we introduce a general adaptive dual-level weighting mechanism (ADWM) to address these challenges from two key perspectives, enhancing a wide range of existing deep-learning methods. First, Intra-Feature Weighting (IFW) evaluates correlations among channels within each feature to reduce redundancy and enhance unique information. Second, Cross-Feature Weighting (CFW) adjusts contributions across layers based on inter-layer correlations, refining the final output. Extensive experiments demonstrate the superior performance of ADWM compared to recent state-of-the-art (SOTA) methods. Furthermore, we validate the effectiveness of our approach through generality experiments, redundancy visualization, comparison experiments, key variables and complexity analysis, and ablation studies. Our code is available at https://github.com/Jie-1203/ADWM.

Hyperspectral unmixing has become a valuable research area in recent years. As significant characteristics of hyperspectral images (HSIs), low-rankness and sparsity have been widely studied to improve the accuracy of abundance estimation for spectral unmixing. However, most of the existing models perform the low-rank and sparse constraints on the entire abundance matrix at the same time, ignoring that low-rankness is often only caused by a few active endmembers. In this paper, we propose a simple but effective method to separate endmembers that contribute more to low-rank property from the given spectral dictionary, and then exploit the weighted nuclear norm on their corresponding abundance maps to enhance the low-rankness. In addition, to make full use of sparsity, both spectral and spatial weighted factors are considered in the ℓ1-norm to constrain abundances of all endmembers. The proposed algorithm is based on the alternating direction method of multipliers (ADMM) framework. Simulated and real-data experiments demonstrate the effectiveness of the resulting unmixing algorithm.

Recently, deep learning has been widely applied in the field of blind hyperspectral unmixing (HU), which aims to simultaneously estimate constitutive endmembers and their abundances in hyperspectral images (HSIs). Generally, the HU process based on deep-learning methods consists of two parts: an encoder and a decoder. In many networks, the decoder stage uses the extracted semantic information of the HSI by the encoder, without direct access to the manifold structure of the HSI. To address this limitation and simultaneously capture both the semantic information and manifold structure of the HSI, in this letter, we propose a dual-channel enhanced decoder network (DED-Net) for the HU problem. Specifically, DED-Net redesigns a decoder by adding a dual-channel graph regularizer that establishes a physically meaningful immediate connection between the abundance and the HSI, effectively integrating both the information from the encoder and the original HSI to enhance endmembers and abundance estimation. Experimental results demonstrate the superiority of our proposed method, which leads to a more accurate unmixing performance.

Hyperspectral unmixing is aimed at identifying pure materials in hyperspectral images as well as their relative proportions within each pixel. In light of the high similarity of spectral signatures among neighboring pixels, a low-rank property is proposed as a prior to enhance the abundance estimation results. In the previous studies, however, the low-rank prior is only reflected in the low-rank constraint on the abundance matrix. In this letter, we present a multidimensional low-rank model for the hyperspectral unmixing problem. We first reshape the abundance matrix to a 3-D abundance tensor. Then we simultaneously impose low-rank constraints on different modes of the abundance tensor to maximize the use of latent spatial information. Moreover, we incorporate the bilateral joint-sparse structure and derive a new algorithm, named asmultidimensional low-rank representation based sparse unmixing. Experiments on both synthetic and real data demonstrate the effectiveness of the proposed algorithm.

Pansharpening refers to the fusion of a low spatial-resolution multispectral image with a high spatial-resolution panchromatic image. In this paper, we propose a novel low-rank tensor completion (LRTC)-based framework with some regularizers for multispectral image pansharpening, called LRTCFPan. The tensor completion technique is commonly used for image recovery, but it cannot directly perform the pansharpening or, more generally, the super-resolution problem because of the formulation gap. Different from previous variational methods, we first formulate a pioneering image super-resolution (ISR) degradation model, which equivalently removes the downsampling operator and transforms the tensor completion framework. Under such a framework, the original pansharpening problem is realized by the LRTC-based technique with some deblurring regularizers. From the perspective of regularizer, we further explore a local-similarity-based dynamic detail mapping (DDM) term to more accurately capture the spatial content of the panchromatic image. Moreover, the low-tubal-rank property of multispectral images is investigated, and the low-tubal-rank prior is introduced for better completion and global characterization. To solve the proposed LRTCFPan model, we develop an alternating direction method of multipliers (ADMM)-based algorithm. Comprehensive experiments at reduced-resolution (i.e., simulated) and full-resolution (i.e., real) data exhibit that the LRTCFPan method significantly outperforms other state-of-the-art pansharpening methods. The code is publicly available at: https://github.com/zhongchengwu/code_LRTCFPan.

Sparse hyperspectral unmixing has attracted much attention in recent decades. Recently, the low-rank representation provides a new perspective for spatial correlation and the weighted nuclear norm regularization has been well studied to enhance the low-rankness of the abundance matrix. However, the commonly used weights only depend on respective singular values, ignoring other singular values' information. In this paper, we propose a new weighting scheme for the weighed nuclear norm to further enhance the sparsity of the singular values of the abundance matrix. The proposed weight for each singular value considers information of all singular values, instead of particular singular value only. Then we refine two sparsity and low-rankness based unmixing algorithms. Simulated and real-data experiments demonstrate the effectiveness of the resulting unmixing algorithms.

Hyperspectral unmixing (HU), estimating endmembers and the corresponding abundances, is crucial for the development of hyperspectral images (HSIs). To improve the unmixing performance, various spatial regularizers are imposed on the abundance matrix. Note that endmember information is also important for HU, especially when the spectral signature in HSIs are highly correlated. In this paper, we investigate information from both endmembers and abundances and propose an endmember constraint non-negative tensor factorization via total variation (EC-NTF-TV) for HU. For estimating end-members, we introduce an endmember constraint to alleviate the spectral signatures' high correlation. In addition, we adopt the TV regularization to exploit the spatial correlation in abundance maps. Finally, we solve the proposed model under the augmented multiplicative update framework. Both synthetic and real hyperspectral data experiments demonstrate the effectiveness of the proposed algorithm.

The aim of hyperspectral image super-resolution (HSI-SR) is to produce high spatial resolution hyperspectral image (HR - HSI) by exploiting the available high spatial resolution multispectral image (HR-MSI) and low spatial resolution hyperspectral image (LR-HSI). In this work, we develop a novel matrix factorization (MF)-based HSI -SR way, which formulates the HSI -SR problem as estimating the spectral dictionary from the observed LR - HSI and the coefficient matrix from both the observed HR-MSI and LR-HSI. Specifically, we first estimate the spectral dictionary from the observed LR - HSI by the dictionary learning algorithm with redundancy assumption. Moreover, based on the superpixel segmentation technology used in the observed HR-MSI, the coefficient vectors are grouped. By concatenating the joint-sparse, nonlocallow-rank, and nonnegative priors of the grouped coefficient vectors, we develop a novel coefficient matrix estimation variational model, which fully explores the nonlocal self-similarity of the desired HR-HSI. The proposed coefficient matrix estimation model is solved under the alternating direction method of multipliers (ADMM) framework. Experimental results prove the superiority of the proposed way from the quantitative and qualitative analysis.

Sparse unmixing is an important technique for analyzing and processing hyperspectral images (HSIs). Simultaneously exploiting spatial correlation and sparsity improves substantially abundance estimation accuracy. In this article, we propose to exploit nonlocal spatial information in the HSI for the sparse unmixing problem. Specifically, we first group similar patches in the HSI, and then unmix each group by imposing simultaneous a low-rank constraint and joint sparsity in the corresponding third-order abundance tensor. To this end, we build an unmixing model with a mixed regularization term consisting of the sum of the weighted tensor trace norm and the weighted tensor$\ell _{2,1}$-norm of the abundance tensor. The proposed model is solved under the alternating direction method of multipliers framework. We term the developed algorithm as the nonlocal tensor-based sparse unmixing algorithm. The effectiveness of the proposed algorithm is illustrated in experiments with both simulated and real hyperspectral data sets.

In this paper, a pan-sharpening approach based on RoG filtering is proposed. This approach follows the framework of classic methods of pan-sharpening, i.e., component substitution and multi-resolution analysis. The filtering technique based on Relativity-of-Gaussian (RoG) regularization is first used in the process of upsampling the original multi-spectral image, and then in the detail extraction phase to obtain spatial details from the panchromatic image. Experiments on datasets acquired by Quickbird and IKONOS demonstrate that the proposed approach obtains competitive performance comparing with several popular pan-sharpening methods.

In this paper, we propose a new tensor rank, named fibered rank, by generalizing the tensor singular value decomposition (t-SVD) to the mode-k t-SVD. It factorizes a three-way tensor into two mode-k orthogonal tensors and a mode-k diagonal tensor. To efficiently remove mixed noise and finely preserve the structure information, we propose a novel hyperspectral image denoising model based on the three-directional tensor nuclear norm (3DTNN), which is a convex relaxation of the fibered rank. An efficient alternating direction method of multipliers (ADMM)-based algorithm is developed to solve the proposed model. Experimental results demonstrate the superiority of the proposed method over the compared ones.

Hyperspectral unmixing has attracted much attention in recent years. Single sparse unmixing assumes that a pixel in a hyperspectral image consists of a relatively small number of spectral signatures from large, ever-growing, and available spectral libraries. Joint-sparsity (or row-sparsity) model typically enforces all pixels in a neighborhood to share the same set of spectral signatures. The two sparse models are widely used in the literature. In this paper, we propose a joint-sparsity-blocks model for abundance estimation problem. Namely, the abundance matrix of size m × n is partitioned to have one row block and s column blocks and each column block itself is joint-sparse. It generalizes both the single (i.e., s = n) and the joint (i.e., s = 1) sparsities. Moreover, concatenating the proposed joint-sparsity-blocks structure and low rankness assumption on the abundance coefficients, we develop a new algorithm called joint-sparseblocks and low-rank unmixing. In particular, for the joint-sparseblocks regression problem, we develop a two-level reweighting strategy to enhance the sparsity along the rows within each block. Simulated and real-data experiments demonstrate the effectiveness of the proposed algorithm.