Brightness order reversal usually appears when two large-exposure-ratio images of a high dynamic range scene are directly fused together by an existing multi-scale exposure fusion algorithm. To address the problem, a novel neural augmented framework is introduced to interpolate an image with the medium exposure by integrating physics-driven and data-driven approaches. The physics-driven method infers high-frequency information while the data-driven approach learns remaining information for the interpolated image. The interpolated image and two large-exposure-ratio images are fused together. Experimental results show that the proposed framework can indeed solve the brightness order reversal problem for the fusion of of two large-exposure-ratio images.

Paired training is a widely-used approach in low light image enhancement(LLIE). However, normal light image with pixel wise calibration cannot be obtained at night in real scene. Then training with low light images only can be a possible solution. The generalization of low-light-image-training-only methods is also an important task, since the different training data produces different LLIE performance. To solve this problem, a diffuse map which is regarded as inherent consistent texture feature, is provided in our low stream network as guidance. Then an image-to-curve transformation is adopted achieved with our up stream to produce the final result, which is consistent with low light to extreme low light condition. A set of experiments have validated the image restoration and generalization abilities of the our method.

Due to saturated regions of low dynamic range (LDR) images and large intensity changes among them, it is challenging to produce an information-enriched panoramic LDR image without visual artifacts from multiple geometrically synchronized LDR images with different exposures and piecewise overlapping fields of views for a high dynamic range (HDR) scene. Fortunately, the stitching of such images is innately a perfect scenario for the fusion of physics-driven and data-driven methods. Based on the insight, a novel neural augmented HDR panoramic stitching algorithm is proposed in this paper. Differently exposed panoramic LDR images are initialized by using a physics-driven method on top of the piecewise overlapping fields of views. They are then refined by a data-driven one, and finally merged together via a multi-scale exposure fusion algorithm to produce the desired panoramic LDR image. Experimental results validate the proposed algorithm11The source code and trained model will be publicly available upon the acceptance..

Model-based single image dehazing algorithms restore images with sharp edges and rich details at the expense of low PSNR values. Data-driven ones restore images with high PSNR values but with low contrast, and even some remaining haze. In this paper, a novel single image dehazing algorithm is introduced by integrating model-based and data-driven approaches. Both transmission map and atmospheric light are initialized by the model-based methods, and refined by deep learning based approaches which form a neural augmentation. Haze-free images are restored by using the transmission map and atmospheric light. Experimental results indicate that the proposed algorithm can remove haze well from real-world and synthetic hazy images.

Model-based single image dehazing algorithms restore haze-free images with sharp edges and rich details for real-world hazy images at the expense of low PSNR and SSIM values for synthetic hazy images. Data-driven ones restore haze-free images with high PSNR and SSIM values for synthetic hazy images but with low contrast, and even some remaining haze for real-world hazy images. In this paper, a novel single image dehazing algorithm is introduced by combining model-based and data-driven approaches. Both transmission map and atmospheric light are first estimated by the model-based methods, and then refined by dual-scale generative adversarial networks (GANs) based approaches. The resultant algorithm forms a neural augmentation which converges very fast while the corresponding data-driven approach might not converge. Haze-free images are restored by using the estimated transmission map and atmospheric light as well as the Koschmieder's law. Experimental results indicate that the proposed algorithm can remove haze well from real-world and synthetic hazy images.

It is challenging to remove rain steaks from a single rainy image because the rain steaks are spatially varying in the rainy image. On top of a new insight in single image de-raining, a nonlocal de-raining algorithm is proposed in this paper to remove the rain streaks from the rainy image. The rainy image is not decomposed into different layers by the proposed algorithm. Experimental results validate the proposed algorithm.

A small ISO and a small exposure time are usually used to capture an image in back- or low-light condition which results in an image with negligible motion blur and small noise but looks dark. In this paper, a single image brightening algorithm is introduced to brighten such an image. The proposed algorithm includes a unique hybrid learning framework to generate two virtual images with large exposure times. The virtual images are first generated via intensity mapping functions (IMFs) which are computed using camera response functions (CRFs) and this is a model-driven approach. Both the virtual images are then enhanced by using a data-driven approach, i.e. a residual convolutional neural network to approach the ground truth images. The model-driven approach and the data-driven one compensate each other in the proposed hybrid learning framework. The final brightened image is obtained by fusing the original image and two virtual images via a multi-scale exposure fusion algorithm with properly defined weights. Experimental results show that the proposed brightening algorithm outperforms existing algorithms in terms of MEF-SSIM metric.

Model-based single image dehazing was widely studied due to its extensive applications. Ambiguity between object radiance and haze and noise amplification in sky regions are two inherent problems of model-based single image dehazing. In this paper, a dark direct attenuation prior (DDAP) is proposed to address the former problem. A novel haze line averaging is proposed to reduce the morphological artifacts caused by the DDAP which enables a weighted guided image filter with a smaller radius to further reduce the morphological artifacts while preserve the fine structure in the image. A multi-scale dehazing algorithm is then proposed to address the latter problem by adopting Laplacian and Gaussian pyramids to decompose the hazy image into different levels and applying different haze removal and noise reduction approaches to restore the scene radiance at the different levels. The resultant pyramid is collapsed to restore a haze-free image. Experiment results demonstrate that the proposed algorithm outperforms state-of-the-art dehazing algorithms.

Deep learning based methods have become dominant solutions to many image processing problems. A natural question would be "Is there any space for conventional methods on these problems?" In this paper, exposure interpolation is taken as an example to answer this question and the answer is "Yes". A new hybrid learning framework is introduced to interpolate a medium exposure image for two large-exposure-ratio images from an emerging high dynamic range (HDR) video capturing device. The framework is set up by fusing conventional and deep learning methods. Experimental results indicate that the deep learning method can be used to improve the quality of interpolated image via the conventional method significantly. The conventional method can be adopted to increase the convergence speed of the deep learning method and to reduce the number of samples which is required by the deep learning method. They compensate each other.

One key issue in content‐based image retrieval is to extract effective features so as to represent the visual content of an image. In this study, a new non‐negative sparse feature learning approach to produce a holistic image representation based on low‐level local features is presented. Specifically, a modified spectral clustering method is introduced to learn a non‐negative visual dictionary from local features of training images. A non‐negative sparse feature encoding method termed non‐negative locality‐constrained linear coding (NNLLC) is proposed to improve the popular locality‐constrained linear coding method so as to obtain more meaningful and interpretable sparse codes for feature representation. Moreover, a new feature pooling strategy named kMaxSum pooling is proposed to alleviate the information loss of the sum pooling or max pooling strategy, which produces a more effective holistic image representation and can be viewed as a generalisation of the sum and max pooling strategies. The retrieval results carried out on two public image databases demonstrate the effectiveness of the proposed approach.