Hyperspectral image fusion (HIF) is pivotal in scientific research and engineering. With the fast development of deep learning techniques, numerous HIF algorithms have been proposed. However, traditional HIF algorithms, which are designed specifically for different input conditions, such as hyperspectral-multispectral image fusion (HMF), hyperspectral-panchromatic image fusion (HPF), and hyperspectral-multispectral-panchromatic image fusion (HMPF), cause significant inconvenience in the practical application of HIF. To overcome this issue, we propose a unified network (UniNet) that integrates HMF, HPF, and HMPF in an “all in one” manner. During testing, UniNet can perform HIF under different input conditions, enhancing its practicality. Specifically, the UniNet consists of the HMF, the HPF, and the HMPF sub-networks. Each sub-network is designed to reconstruct a high-resolution (HR) hyperspectral image (HSI) under a specific input condition. A mutual learning strategy is proposed to establish the relationships among sub-networks, enabling the transfer of spectral-spatial knowledge across different input conditions. In addition, a multiscale gradient spatial attention module (MGSAM) is designed in the network to better extract the spatial edge details of HSIs at multiple scales. Once trained, UniNet is capable of working with its sub-networks independently. Extensive experiments show that UniNet outperforms 18 state-of-the-art algorithms with five HSI datasets. Furthermore, the effectiveness of our algorithm is validated through ablation experiments.

Multispectral image (MSI) reconstruction via remote sensing image fusion (RSIF), involving the fusion of MSIs with panchromatic (PAN) images, has attracted considerable attention. However, the main limitation of RSIF lies in the fact that the resolution of the fused image is limited by the PAN image. Although single image super-resolution (SISR) methods have the potential to improve image resolution, the larger scale factors often result in the loss of edge sharpness. To tackle these challenges, this article proposes a novel end-to-end framework for MSI reconstruction, termed FSRNet, aimed at generating higher resolution MSI while preserving the fine edge structural details. The FSRNet seamlessly integrates RSIF and SISR into a unified framework, thereby mitigating the cumulative errors that typically arise in two-stage cascaded methods. Moreover, an innovative implicit edge calibration (IEC) scheme is proposed that leverages gradient prior knowledge implicitly to calibrate and sharpen the MSI structure effectively. Notably, IEC is exclusively utilized during the training phase and is omitted during testing. Thus, it does not increase the complexity of the network in the actual application (i.e., testing phase). Additionally, we introduce the progress feature distillation module to improve feature representation through the feature distillation structure, facilitating the learning of more distinctive hierarchical features. Experimental results on the QuickBird, Gaofen-2, and WorldView-3 datasets demonstrate that the proposed method exhibits advantages compared to other state-of-the-art SISR and two-stage methods in terms of both quantitative and qualitative comparisons.

Hyperspectral images (HSIs) contain rich spectral information and have great application value. However, due to various hardware limitations, the spatial resolution of HSIs acquired by the sensor is low. HSI super-resolution (SR) attracts much attention to improve spatial quality. In this letter, a single HSI SR method based on network fusion is proposed. Our method includes the SR network part and fusion part. In the SR network part, we construct 3-D multiscale mixed attention networks (3-D-MSMANs) by cascading 3-D multiscale mixed attention block (3-D-MSMAB) to restore high-resolution HSIs. 3-D-MSMAB consists of the 3-D Res2net module and the mixed attention module. 3-D Res2net module is a simple and effective multiscale method. The mixed attention module is proposed by combining the first- and second-order statistics of features. In addition, we use the mutual learning loss between 3-D-MSMAN so that they can learn from each other. In the fusion part, the fusion module is designed to merge the output of each 3-D-MSMAN. Our method can achieve good results in both simulated and real SR experiments. Code is available athttps://github.com/LYT-max/Mixed-Attention-for-HSI-SR.

Deep convolutional neural networks (CNNs) have made great progress in the super-resolution (SR) of hyperspectral images (HSIs). However, most methods utilize convolution to explore local features, and global features are ignored. It is expected that combining non-local mechanism with CNN will improve the performance of HSI SR. This paper presents a multi-level progressive HSI SR network. The dense non-local and local block (DNLB) is constructed to combine local and global features, which are used to reconstruct super-resolution images at each level. Due to the high dimension of HSI, original non-local methods produce memory-expensive attention maps. We develop a non-local channel attention block to extract the global features of HSIs efficiently. Spatial-spectral gradient is injected in the non-local attention block to obtain better details. Furthermore, the progressive learning mode based multi-level network is proposed to reconstruct HSI with fine details. A number of experiments demonstrate that our method can reconstruct hyperspectral images more accurately than existing methods.

Due to rich spectral information, hyperspectral images (HSIs) have been widely used in various fields. However, limited by imaging systems, the low spatial resolution of HSIs has become an important problem. In this article, for enhancing the spatial resolution, Interactformer is proposed to interact with global and local features extracted by Transformer and 3D convolutional neural network (CNN) branches. Within the Transformer branch, a separable self-attention module with linear complexity is designed to solve the problem that traditional self-attention mechanisms suffer from large memory costs due to quadratic complexity. In the 3D CNN branch, the spectral attention module and 3D convolution are applied jointly to better protect the spectral correlation among spectral bands and facilitate local feature extraction of HSIs. The interactive attention unit between the two parallel branches is designed to interact with local and global feature information adaptively. Compared with state-of-the-art super-resolution (SR) methods, the proposed method reconstructs better HSI in simulated SR experiments, real SR experiments, and classification experiments, which prove that Interactformer can effectively improve the spatial resolution while preserving the spectral information.