In this letter, based on tensor modeling, we propose a novel panchromatic (Pan)-guided tensor low-rank (PGTLR) model for multispectral image (MSI) sharpening, which aims to fuse the low resolution (LR) MSI and Pan image to output the high resolution (HR) MSI. On one hand, we novelly exploit the tensor low-fibered-rank prior of HR MSI to model its global three-dimensional spatial-spectral correlations, which is constructed as the tensor nuclear norm (TNN) prior term. On the other hand, we further novelly exploit the Pan-guided tensor low-fibered-rank prior to model the spatial link between HR MSI and Pan, which is constructed as the novel Pan-guided TNN prior term. Furthermore, the proposed PGTLR model is optimized by an efficient alternative algorithm. Moreover, the experimental results on reduced-scale and full-scale datasets quantitatively and visually validate the superiority of PGTLR.

Convolutional neural networks (CNNs) and transformers have achieved excellent classification performances in hyperspectral imagery (HSI) and light detection and ranging (LiDAR) land cover classification. However, for complex land covers, effectively characterizing the contextual information and spectral-spatial interaction features of HSI and LiDAR is crucial for improving classification accuracy. Motivated by this, this letter is dedicated to selective convolutional kernel mechanisms and spectral-spatial interactive transformer feature learning style, proposing a selective spectral-spatial aggregation transformer network, named S2ATNet. A convolution feature selected module (CFSM), which can dynamically capture the contextual features of various land covers, is first utilized in both of HSI and LiDAR branches. Afterward, a cascaded spatial-spectral learning and interactive fusion (CSLIF) block is designed for acquiring the nonlocal spatial-spectral characteristics in an interactive feature learning style. The learned features are fed into the max-average classification head (MACH) to obtain the final classification results. The effectiveness of the proposed S2ATNet is validated on two publicly available datasets. Codes are available athttps://github.com/RSIP-NJUPT/S2ATNet.git.

In this article, we propose a novel, unified, and effective hyperspectral image (HSI) destriping and denoising method with gradient subspace-regularized HSI and stripe-coupled nonconvex tensor low-rank priors (GSHSNTLRs). First, by exploiting the mode-3 low-rank properties of the gradients of HSI (i.e., the spatial horizontal gradient, spatial vertical gradient, and spectral gradient of HSI) along the spectral dimension, we apply the mode-3 low-rank decomposition of the gradients of HSI to obtain their representation coefficient tensors (RCTs), and further study the tensor low-tubal-rank properties of the RCTs in the gradient subspace. Thus, we propose the unified gradient subspace-regularized log tensor nuclear norm (LogTNN)-based nonconvex tensor low-rank prior term of the RCTs. Moreover, by fully considering the structural speciality of stripe noise, which has strong tensor low-tubal-rank property, we particularly study the HSI-guided tensor low-rank modeling for the stripe noise by exploring the tensor low-tubal-rank property of HSI plus stripe and propose the unified HSI and stripe-coupled LogTNN-based nonconvex tensor low-rank prior term of HSI and stripe simultaneously. Subsequently, the proposed GSHSNTLR model is solved by using the alternating direction method of multipliers (ADMMs). Finally, lots of experimental results and analysis fully demonstrate the destriping and denoising performance and superiority of GSHSNTLR.

This paper proposes a new hyperspectral image (HSI) denoising and destriping method via gradient tensor subspace low-rank learning and along-across stripe directional constraints (GTSL2A2SDC) under the unified framework of tensor representation modeling. On one hand, for the modeling of stripe noise, based on the inherently directional and structural attributes of the stripe noise, we mainly investigate the mode-1 gradient tensor of stripe along the stripe direction which holds the preferable “zero plane” constraint as well as the mode-2 gradient tensor of stripe across the stripe direction which holds the preferable tensor low-fibered-rank attribute along the mode-3 spectral dimension. Therefore, we novelly propose the unified along-across stripe directional gradient tensor constraints for the stripe noise, which can simultaneously characterize the directional and structural attributes of the stripe noise. On the other hand, for the modeling of HSI, based on the preferably spectral low-rankness attributes of the multi-mode gradient tensors of HSI, namely, mode-1 gradient tensor, mode-2 gradient tensor and mode-3 gradient tensor, we further utilize the spectral low-rank factorization of the gradient tensors of HSI to get the corresponding representation tensors, and particularly investigate the nonlocal self-similarities-based low-rankness of the representation tensors under the gradient tensor-based subspace low-rank learning framework. Moreover, we optimize the proposed GTSL2A2SDC model via an efficiently alternative and iterative algorithm. Lastly, extensive experiments comprehensively validate the denoising and destriping capacity and superiority of GTSL2A2SDC.

The fusion of hyperspectral and light detection and ranging (LiDAR) data could significantly improve land-cover classification performance. Most existing feature fusion methods focus on “late fusion” or “halfway feature interaction fusion” methods, which treat LiDAR and hyperspectral data as model inputs while overlooking the redundancy between hyperspectral imagery (HSI) and LiDAR features. In addition, the unique characteristics of HSI and LiDAR data, combined with the complexity of land-cover backgrounds, make it challenging to accurately describe their properties. High-order deep features, as a deep statistical representation, could effectively capture the statistical characteristics of these land covers. Based on this, this article focuses on the unique characteristics of HSI and LiDAR data, as well as the distinguishability of features, and designs a progressive hyperspectral and LiDAR collaborative classification method, named coarse-to-fine high-order network (CHNet). In the coarse stage, HSI data redundancy reduction and LiDAR feature reconstruction are performed in either the frequency domain or the spatial domain to ensure the effectiveness of subsequent feature fusion. The fine stage focuses primarily on selective feature fusion and discriminability enhancement, introducing gating mechanisms, deep expert systems, and high-order feature statistics. This approach enhances the discriminability of the fused features while simultaneously reducing their dimensionality. The proposed method, based on a “redundancy removal-feature learning” mechanism, captures more effective deep features by accounting for the different imaging mechanisms of multisource data and the complex background of land covers, ultimately improving the effectiveness of land-cover classification. Experimental results on three public datasets and one self-constructed dataset demonstrate that CHNet achieves superior performance. The code is available athttps://github.com/RSIP-NJUPT/CHNet.

In this paper, we propose an effective multi-mode structural nonconvex tensor low rank (M2SNTLR) regularized hyperspectral image (HSI) destriping and denoising method in a unified framework of tensor representation. Firstly, we exploit and model the tensor low-tubal-rank properties of the HSI and its spectral gradient along spectral mode simultaneously via the tensor tubal rank functions. Secondly, based on the speciality and directionality of stripe noise, we particularly exploit and model the tensor low-tubal-rank properties of stripe noise and its unidirectional vertical gradient along spatial vertical mode simultaneously via also the tensor tubal rank functions. Thirdly, by using the log tensor nuclear norm (logTNN)-based nonconvex surrogate on those tensor tubal rank functions for a closer approximation, we thus propose the logTNN-based multi-mode structural nonconvex tensor low rank priors of HSI and stripe noise. Furthermore, we solve the proposed M2SNTLR model via the alternating direction method of multipliers. At last, extensive experiments validate that the proposed M2SNTLR method performs better denoising results than various low rank-based HSI denoising methods.

In this letter, based on the tensor representation modeling, we propose a novel and strict tensor-based pansharpening model via double nonconvex tensor low-tubal-rank (DNTLTR) priors for the fusion of low resolution multispectral (LRMS) and panchromatic (Pan) images to produce the high resolution MS (HRMS) images. By modeling the MS image as a third-order tensor for better modeling its spatial-spectral structural correlations, we particularly exploit the tensor low-tubal-rank properties of HRMS as well as the difference of HRMS and Pan at the same time, and then propose a novel unified log tensor nuclear norm-based double nonconvex tensor low-tubal-rank prior term. Moreover, for the spectral preservation of LRMS image, we also impose the spatial degradation-based spectral fidelity constraint between HRMS and LRMS. Then, we apply the alternating direction method of multiplier to optimize the proposed DNTLTR model. Finally, we show both the reduced-scale and full-scale fusion experiments to validate the effectiveness of DNTLTR visually and quantitatively.

In this article, we propose a novel adaptive spatial structure-aware and spectral gradient structure tensor-guided model (AS3GSTM) for pansharpening, which realizes the process of fusing the low-resolution multispectral (LRMS) image and the paired panchromatic (Pan) image to output the high-resolution multispectral (HRMS) image. Specifically, based on the basic spectral fidelity term between HRMS and LRMS obtained from the spatial degradation model for spectral fidelity, we also enforce the radiometric ratio-guided high-frequency detail fidelity term between HRMS, LRMS, and Pan for high-frequency detail fidelity. Moreover, considering that the HRMS image and the Pan image actually not only have strong spatial structure similarities, but also differ from each other, we further propose a novel Pan-guided adaptive spatial structure-aware prior term for the HRMS image to guide the fusion process. Besides, we particularly exploit the structure tensor of the spectral gradient of HRMS for simultaneously spectral-spatial prior modeling, and propose a novel spectral gradient-guided structure tensor total variation prior term for the HRMS image. Subsequently, we design an efficiently alternating algorithm to optimize the proposed AS3GSTM model. Finally, lots of fusion experiments comprehensively validate the superiority of AS3GSTM.

The vision transformer has been widely applied in remote sensing image scene classification due to its excellent ability to capture global features. However, remote sensing scene images involve challenges such as scene complexity and small inter-class differences. Directly utilizing the global tokens of transformer for feature learning may increase computational complexity. Therefore, constructing a distinguishable transformer network which adaptively selects tokens can effectively improve the classification performance of remote sensing scene images while considering computational complexity. Based on this, a second-order differentiable token transformer network (SDT2Net) is proposed for considering the efficacy of distinguishable statistical features and non-redundant learnable tokens of remote sensing scene images. A novel transformer block, including an efficient attention block (EAB) and differentiable token compression (DTC) mechanism, is inserted into SDT2Net for acquiring selectable token features of each scene image guided by sparse shift local features and token compression rate learning style. Furthermore, a fast token fusion (FTF) module is developed for acquiring more distinguishable token feature representations. This module utilizes the fast global covariance pooling algorithm to acquire high-order visual tokens and validates the effectiveness of classification tokens and high-order visual tokens for scene classification. Compared with other recent methods, SDT2Net achieves the most advanced performance with comparable FLOP-s (Floating Point Operations Per Second). The code will be available at https://github.com/RSIP-NJUPT/SDT2Net.

Local-global feature learning method based on deep learning has significantly improved the collaborative classification of hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data. However, HSI encompasses numerous bands with significant interband correlations. Addressing how to efficiently capture spatial, spectral, and elevation information from hyperspectral and LiDAR data while considering data redundancy and enhancing feature representation of land cover will contribute to enhancing classification effectiveness. Combining frequency feature learning methods with convolutional neural networks (CNNs), transformers, and other architectures to construct an end-to-end feature learning network framework is an effective method. Therefore, this article proposes a frequency domain-based network (FDNet) for the classification of HSI and LiDAR data using a frequency local feature learning framework and a self-attention mechanism based on fast Fourier transform (FFT). FDNet could effectively capture local efficient frequency features of spatial, spectral, and elevation information in HSI and LiDAR data in an adaptive feature learning style, and embedding convolutional offsets into the frequency domain-based transformer network not only enhances local features but also effectively captures global semantic characteristics of land covers while reducing computational complexity. We validated the efficacy of FDNet across three publicly available datasets and a particularly challenging self-constructed dataset, denoted as the Yancheng dataset. The source codes will be available athttps://github.com/RSIP-NJUPT/FDNet.

Pansharpening, which is also known as the fusion of low resolution multispectral (LRMS) and panchromatic (PAN) images, refers to producing a high resolution multispectral (HRMS) image by preserving the spectral detail from the LRMS image while extracting the spatial detail from the PAN image. In this article, we revisit and novelly reinterpret the multi-resolution analysis (MRA)-based pansharpening framework as the fusion framework of “spectral detail + spatial detail" and can obtain two alternative formulations of spectral detail and spatial detail respectively, and hence propose a novel variational pansharpening method with MRA-inspired spatial and spectral details preserved model. Firstly, the spatial degradation relationship between HRMS and LRMS is imposed as the spectral fidelity term. Secondly, based on the new reinterpretation of “spectral detail + spatial detail" of MRA fusion framework, we propose to use the structure tensor to model the spatial detail image, and propose a new structure tensor total variation (STV)-guided spatial detail preserved prior term. Moreover, to model the spectral detail image, we propose to impose the spectral detail preserved constraint between the two alternative formulations of spectral detail as the MRA-inspired spectral detail preserved prior term. Then, we optimize the proposed model via the alternating direction method of multipliers (ADMM). Furthermore, variously experimental results on the reduced-scale and full-scale datasets validate the superiority of proposed method.

The dictionary-aided sparse regression (SR) approach has been developed in hyperspectral unmixing (HU) in remote sensing. By using an available spectral library as a dictionary, the SR approaches unmix the spectral image by selecting the endmember matrix from the library which best represent the image and its fractional abundance. In this paper, we proposed a simultaneous dictionary refining and enhanced sparse regression method for hyperspectral unmixing(DRESR). The proposed method not only enhances the sparsity of fractional abundance through double weighted sparse regularization, and also improves the spectral signature mismatches between an actual spectra and its corresponding endmember in the spectral library by using dictionary sparse refining. Experimental results on both synthetic and real hyperspectral data sets demonstrate better performance compared with several state-of-art algorithms.