A new dichromatic illuminant estimation method using a deep neural network is proposed. Previous methods based on the dichromatic reflection model commonly suffer from inaccurate separation of specularity, thus being limited in their use in a real-world. Recent deep neural network-based methods have shown a significant improvement in the estimation of the illuminant color. However, why they succeed or fail is not explainable easily, because most of them estimate the illuminant color at the network output directly. To tackle these problems, the proposed architecture is designed to learn dichromatic planes and their confidences using a deep neural network with novel losses function. The illuminant color is estimated by a weighted least mean square of these planes. The proposed dichromatic guided learning not only achieves compelling results among state-of-the-art color constancy methods in standard real-world benchmark evaluations, but also provides a map to include color and regional contributions for illuminant estimation, which allow for an in-depth analysis of success and failure cases of illuminant estimation.

Multi-exposure image fusion inevitably causes ghost artifacts owing to inaccurate image registration. In this study, we propose a deep learning technique for the seamless fusion of multi-exposed low dynamic range (LDR) images using a focus-pixel sensor. For auto-focusing in mobile cameras, a focus-pixel sensor originally provides left (L) and right (R) luminance images simultaneously with a full-resolution RGB image. These L/R images are less saturated than the RGB images because they are summed up to be a normal pixel value in the RGB image of the focus pixel sensor. These two features of the focus pixel image, namely, relatively short exposure and perfect alignment are utilized in this study to provide fusion cues for high dynamic range (HDR) imaging. To minimize fusion artifacts, luminance and chrominance fusions are performed separately in two sub-nets. In a luminance recovery network, two heterogeneous images, the focus pixel image and the corresponding overexposed LDR image, are first fused by joint learning to produce an HDR luminance image. Subsequently, a chrominance network fuses the color components of the misaligned underexposed LDR input to obtain a 3-channel HDR image. Existing deep-neural-network-based HDR fusion methods fuse misaligned multi-exposed inputs directly. They suffer from visual artifacts that are observed mostly in saturated regions because pixel values are clipped out. Meanwhile, the proposed method reconstructs missing luminance with aligned unsaturated focus pixel image first, and thus, the luma-recovered image provides the cues for accurate color fusion. The experimental results show that the proposed method not only accurately restores fine details in saturated areas, but also produce ghost-free high-quality HDR images without pre-alignment.

We present a physics-based illumination estimation approach explicitly designed to handle natural images under ambient light. Existing physics-based color constancy methods are theoretically perfect but do not handle real-world images well because the majority of these methods assume a single illuminant. Therefore, specular pixels selected using existing methods produce estimated dichromatic lines that are thick or curvilinear in the presence of ambient light, thus generating significant errors. Based on the Phong reflection model, we show that a group of specular pixels on a uniformly colored object, although they are subject to intensity thresholding, produce a unique dichromatic line length depending on the geometry of each image path. Assuming that the longest dichromatic line is the most desirable when estimating the chromaticity of an illuminant, ambient-robust specular pixels are also found on the same path on which the longest dichromatic line segment is generated. Therefore, we propose a method to find the optimal image path in which the specular pixels produce the longest dichromatic line. Even though the number of collected specular pixels is reduced using the proposed method, they are proven to be more accurate when determining the illuminant chromaticity even in the existing methods. Experiments with an established benchmark data set and a self-produced image set find that the proposed method is better able to locate the illuminant chromaticity compared with the state-of-the-art color constancy methods.

This paper proposes a novel two-step multi-illuminant algorithm for color constancy in outdoor scenes. We adopt different color constancy approaches for both primary and secondary illuminations. In the first step, an input image is white-balanced entirely using the existing single-illuminant color constancy method. Next, we extract the secondary illumination region, which typically corresponds to a shaded region in outdoor scenes. Finally, the shaded region is corrected by the guidance of the Planckian locus theory. Experimental results show that the proposed algorithm can achieve much smaller angular error than conventional multi-illuminant methods while color artifact is alleviated.