Existing image fusion methods focus on containing more complementary information, but source images always suffer from motion blur owing to object motion, which results in distorted details in fused images and further deteriorates performance on high-level tasks. This paper proposes a novel visible and infrared image fusion framework capable for motion deblurring (MDbFusion++), which can simultaneously perform image fusion and deblurring within a mutually reinforcing framework. MDbFusion++ employs a coarse-to-fine image restoration strategy and comprises two key components: a coarse deblurring part (CDP) and a fine deblurring and fusion part (FDFP). Firstly, CDP transfers multi-modal images into features corresponding to spatial locations and creatively leverages infrared features to coarsely compensate motion blurred visible ones through adaptive weights module (AWM). Subsequently, FDFP further restores fine visible features and achieves multi-modal images fusion in spatial and frequency domains with the help of multi-domain enhancement module (MEM). The deblurred visible features provide clear information to improve fusion results, and the improved fused images, in turn, provide gradient feedback to further improve deblurring effects. We evaluate our network in terms of both image deblurring and fusion, and extensive comparative experiments demonstrate the superior performance and distinct advantages of MDbFusion++.

A single-modal infrared or visible image offers limited representation in scenes with lighting degradation or extreme weather. We propose a multi-modal fusion framework, named SDSFusion, for all-day and all-weather infrared and visible image fusion. SDSFusion exploits the commonality in image processing to achieve enhancement, fusion, and semantic task interaction in a unified framework guided by semantic awareness and multi-scale features and losses. To address the disparity between infrared and visible images in degraded scenes, we differentiate modal features in a unified fusion model. Unlike existing joint fusion methods, we propose an adversarial generative network that refines the reconstruction of low-light images by embedding fused features. It provides feature-level brightness supplementation and image reconstruction to refine brightness and contrast. Extensive experiments in degraded scenes confirm that our approach is superior to state-of-the-art approaches in visual quality and performance, demonstrating the effectiveness of interaction improvement. The code will be posted at: https://github.com/Liling-yang/SDSFusion.

Existing image fusion methods focus on containing more complementary information, but source images always suffer from motion blur owing to object motion, which results in distorted details in fused images and further deteriorates performance on high-level tasks. This paper proposes a novel visible and infrared image fusion framework capable for motion deblurring (MDbFusion++), which can simultaneously perform image fusion and deblurring within a mutually reinforcing framework. MDbFusion++ employs a coarse-to-fine image restoration strategy and comprises two key components: a coarse deblurring part (CDP) and a fine deblurring and fusion part (FDFP). Firstly, CDP transfers multi-modal images into features corresponding to spatial locations and creatively leverages infrared features to coarsely compensate motion blurred visible ones through adaptive weights module (AWM). Subsequently, FDFP further restores fine visible features and achieves multi-modal images fusion in spatial and frequency domains with the help of multi-domain enhancement module (MEM). The deblurred visible features provide clear information to improve fusion results, and the improved fused images, in turn, provide gradient feedback to further improve deblurring effects. We evaluate our network in terms of both image deblurring and fusion, and extensive comparative experiments demonstrate the superior performance and distinct advantages of MDbFusion++.

In this paper, we propose a new end-to-end model, called dual-discriminator conditional generative adversarial network (DDcGAN), for fusing infrared and visible images of different resolutions. Unlike the pixel-level methods and existing deep learning-based methods, the fusion task is accomplished through the adversarial process between a generator and two discriminators, in addition to the specially designed content loss. The generator is trained to generate real-like fused images to fool discriminators. The two discriminators are trained to calculate the JS divergence between the probability distribution of downsampled fused images and infrared images, and the JS divergence between the probability distribution of gradients of fused images and gradients of visible images, respectively. Thus, the fused images can compensate for the features that are not constrained by the single content loss. Consequently, the prominence of thermal targets in the infrared image and the texture details in the visible image can be preserved or even enhanced in the fused image simultaneously. Moreover, by constraining and distinguishing between the downsampled fused image and the low-resolution infrared image, DDcGAN can be preferably applied to the fusion of different resolution images. Qualitative and quantitative experiments on publicly available datasets demonstrate the superiority of our method over the state-of-the-art.