Pixel-level fusion of visible and infrared images has demonstrated promise in enhancing information representation. However, nighttime image fusion remains challenging due to low and uneven lighting. Existing fusion methods neglect the preservation of color-related information at night, resulting in unsatisfactory outcomes with insufficient brightness. This paper presents a novel color image fusion framework to prevent color distortion, thus generating results more aligned with human perception. Firstly, we design an image fusion network to retain color information from visible images under low-light conditions. Secondly, we incorporate mature low-light enhancement technology into the network as a flexible component to produce fusion results under normal illumination. The training process is carefully designed to address potential issues of overexposure or noise amplification. Finally, we utilize knowledge distillation to create a lightweight end-to-end network that directly generates fusion results under normal lighting conditions from pairs of low-light images. Experimental results demonstrate that our proposed framework outperforms existing methods in nighttime scenarios.

Unsupervised image restoration methods relying on a single data source often face challenges in achieving high-quality visual data completion due to the absence of additional supplementary information. This paper presents a novel optimization framework to address this limitation and further enhance the performance of image restoration. The framework generates pseudo side information (PSI) and utilizes it to guide the process of visual data completion. We introduce a pseudo side information regularizer (PSIR) tailored specifically for visual data completion tasks. The PSIR comprises two components: the PSI generator and updater, responsible for generating and refining the PSI, and the neural self-expressive prior (NSEP), which identifies a prior matching the desired result and PSI during optimization. Notably, our method achieves comprehensive visual data completion across various data types without the need for additional reference side information or training data. Extensive experimental evaluations conducted on spectral data (including color images, multispectral images, and hyperspectral images), video data (including gray video, color video, and hyperspectral video), magnetic resonance image, and real cloud data demonstrate the superiority of our approach over other state-of-the-art completion methods under different missing rate scenarios.

Hyperspectral image (HSI) denoising algorithms often face challenges with non-uniform, band-dependent noise and fail to fully utilize high signal-to-noise ratio (SNR) spectral bands. To address these issues, this paper presents a zero-shot framework that leverages high-SNR bands as scene-specific references to guide the restoration of degraded bands. The framework features a dual-branch network architecture with the guide branch extracting high-fidelity spatial structures from high-SNR bands and the denoising branch restoring degraded bands through dynamic feature fusion. Fusion modules, enhanced with spectral-spatial attention, enable adaptive and spatially aware refinement between branches. Unlike conventional methods relying on external training datasets or fixed priors, the proposed approach adapts to the scene-specific quality disparities within each HSI. Extensive experiments on synthetic and real-world datasets demonstrate its superior ability in preserving spatial-spectral fidelity and handling complex noise scenarios, outperforming state-of-the-art methods.

In real hyperspectral scenes, heterogeneous spatial details and noises make a single subspace assumptions unrealistic. In this letter, a novel transferable multiple tensor subspace learning scheme is proposed for super-resolution enhancement of hyperspectral image (HSI). The intrinsic assumption is that the nonlocal patch tensors extracted from HSIs are derived from multiple tensor low-rank subspaces, which is compatible with practical data distribution and may better characterize the complex structures underlying HSIs. The transferable subspace structures are embedded into both nonblind and semi-blind HSI super-resolution. The alternating direction method of multipliers (ADMMs) algorithm is derived for model learning. The superiority of our method is demonstrated by comprehensive experiments on both synthetic and real datasets.

This paper presents a deep learning-based spectral demosaicing technique trained in an unsupervised manner. Many existing deep learning-based techniques relying on supervised learning with synthetic images, often underperform on real-world images, especially as the number of spectral bands increases. This paper presents a comprehensive unsupervised spectral demosaicing (USD) framework based on the characteristics of spectral mosaic images. This framework encompasses a training method, model structure, transformation strategy, and a well-fitted model selection strategy. To enable the network to dynamically model spectral correlation while maintaining a compact parameter space, we reduce the complexity and parameters of the spectral attention module. This is achieved by dividing the spectral attention tensor into spectral attention matrices in the spatial dimension and spectral attention vector in the channel dimension. This paper also presents Mosaic 25 , a real 25-band hyperspectral mosaic image dataset featuring various objects, illuminations, and materials for benchmarking purposes. Extensive experiments on both synthetic and real-world datasets demonstrate that the proposed method outperforms conventional unsupervised methods in terms of spatial distortion suppression, spectral fidelity, robustness, and computational cost. Our code and dataset are publicly available at https://github.com/polwork/Unsupervised-Spectral-Demosaicing.

The fusion of infrared and visible images combines the information from two complementary imaging modalities for various computer vision tasks. Many existing techniques, however, fail to maintain a uniform overall style and keep salient details of individual modalities simultaneously. This paper presents an end-to-end Laplacian Pyramid Fusion Network with hierarchical guidance (HG-LPFN) that takes advantage of pixel-level saliency reservation of Laplacian Pyramid and global optimization capability of deep learning. The proposed scheme generates hierarchical saliency maps through Laplacian Pyramid decomposition and modal difference calculation. In the pyramid fusion mode, all sub-networks are connected in a bottom-up manner. The sub-network for low-frequency fusion focuses on extracting universal features to produce an opposite style while sub-networks for high-frequency fusion determine how much the details of each modality will be retained. Taking the style, details, and background into consideration, we design a set of novel loss functions to supervise both low-frequency images and full-resolution images under the guidance of saliency maps. Experimental results on public datasets demonstrate that the proposed HG-LPFN outperforms the state-of-the-art image fusion techniques.

Recently, tensor sparsity modeling has achieved great success in the tensor completion (TC) problem. In real applications, the sparsity of a tensor can be rationally measured by low-rank tensor decomposition. However, existing methods either suffer from limited modeling power in estimating accurate rank or have difficulty in depicting hierarchical structure underlying such data ensembles. To address these issues, we propose a parametric tensor sparsity measure model, which encodes the sparsity for a general tensor by Laplacian scale mixture (LSM) modeling based on three-layer transform (TLT) for factor subspace prior with Tucker decomposition. Specifically, the sparsity of a tensor is first transformed into factor subspace, and then factor sparsity in the gradient domain is used to express the local similarity in within-mode. To further refine the sparsity, we adopt LSM by the transform learning scheme to self-adaptively depict deeper layer structured sparsity, in which the transformed sparse matrices in the sense of a statistical model can be modeled as the product of a Laplacian vector and a hidden positive scalar multiplier. We call the method as parametric tensor sparsity delivered by LSM-TLT. By a progressive transformation operator, we formulate the LSM-TLT model and use it to address the TC problem, and then the alternating direction method of multipliers-based optimization algorithm is designed to solve the problem. The experimental results on RGB images, hyperspectral images (HSIs), and videos demonstrate the proposed method outperforms state of the arts.

We propose a novel graph Laplacian-guided coupled tensor decomposition (gLGCTD) model for fusion of hyperspectral image (HSI) and multispectral image (MSI) for spatial and spectral resolution enhancements. The coupled Tucker decomposition is employed to capture the global interdependencies across the different modes to fully exploit the intrinsic global spatial-spectral information. To preserve local characteristics, the complementary submanifold structures embedded in high-resolution (HR)-HSI are encoded by the graph Laplacian regularizations. The global spatial-spectral information captured by the coupled Tucker decomposition and the local submanifold structures are incorporated into a unified framework. The gLGCTD fusion framework is solved by a hybrid framework between the proximal alternating optimization (PAO) and the alternating direction method of multipliers (ADMM). Experimental results on both synthetic and real data sets demonstrate that the gLGCTD fusion method is superior to state-of-the-art fusion methods with a more accurate reconstruction of the HR-HSI.

Hyperspectral image super-resolution by fusing high-resolution multispectral image (HR-MSI) and low-resolution hyperspectral image (LR-HSI) aims at reconstructing high resolution spatial-spectral information of the scene. Existing methods mostly based on spectral unmixing and sparse representation are often developed from a low-level vision task perspective, they cannot sufficiently make use of the spatial and spectral priors available from higher-level analysis. To this issue, this paper proposes a novel HSI super-resolution method that fully considers the spatial/spectral subspace low-rank relationships between available HR-MSI/LR-HSI and latent HSI. Specifically, it relies on a new subspace clustering method named "structured sparse low-rank representation" (SSLRR), to represent the data samples as linear combinations of the bases in a given dictionary, where the sparse structure is induced by low-rank factorization for the affinity matrix. Then we exploit the proposed SSLRR model to learn the SSLRR along spatial/spectral domain from the MSI/HSI inputs. By using the learned spatial and spectral low-rank structures, we formulate the proposed HSI super-resolution model as a variational optimization problem, which can be readily solved by the ADMM algorithm. Compared with state-of-the-art hyperspectral super-resolution methods, the proposed method shows better performance on three benchmark datasets in terms of both visual and quantitative evaluation.