Abstract The human visual system achieves color constancy, allowing consistent color perception under varying environmental contexts, while also being deceived by color illusions, where contextual information affects our perception. Despite the close relationship between color constancy and color illusions, and their potential benefits to the field, both phenomena are rarely studied together in computer vision. In this study, we present the benefits of considering color illusions in the field of computer vision. Particularly, we introduce a learning-free method, namely multiresolution color constancy , which combines insights from computational neuroscience and computer vision to address both phenomena within a single framework. Our approach performs color constancy in both multi- and single-illuminant scenarios, while it is also deceived by assimilation illusions. Additionally, we extend our method to low-light image enhancement, thus, demonstrate its usability across different computer vision tasks. Through comprehensive experiments on color constancy, we show the effectiveness of our method in multi-illuminant and single-illuminant scenarios. Furthermore, we compare our method with state-of-the-art learning-based models on low-light image enhancement, where it shows competitive performance. This work presents the first method that integrates color constancy, color illusions, and low-light image enhancement in a single and explainable framework.

Low-light image enhancement is a widely studied field of image processing. Over the past decades, numerous successful algorithms have been proposed to address its challenges, such as low contrast, color fading, and noise. One way to enhance dim scenes with vivid colors while suppressing noise is to apply intrinsic image decomposition prior to enhancement. Intrinsic image decomposition separates an image into its fundamental components, such as reflectance and shading. In this study, we extent our recently proposed learning-free intrinsic image decomposition method based on Retinex theory. This approach utilizes scale-space computations and super-pixel segmentation to effectively estimate the reflectance and shading components of a scene. With slight modifications to the original method, we demonstrate its usability in the context of low-light image enhancement. To validate the effectiveness and generalizability of the method, we conduct comprehensive experiments on five benchmarks. Our results show that our method, originally designed for intrinsic image decomposition, can produce effective results for low-light image enhancement and achieve competitive performance compared to learning-based methods specifically developed for this task.

Abstract The human visual system unconsciously determines the color of the objects by “discounting” the effects of the illumination, whereas machine vision systems have difficulty performing this task. Color constancy algorithms assist computer vision pipelines by removing the effects of the illuminant, which in the end enables these pipelines to perform better on high-level vision tasks based on the color features of the scene. Due to its benefits, numerous color constancy algorithms have been developed, and existing techniques have been improved. Combining different strategies and investigating new methods might help us design simple yet effective algorithms. Thereupon, we present a color constancy algorithm based on the outcomes of our previous works. Our algorithm is built upon the biological findings that the human visual system might be discounting the illuminant based on the highest luminance patches and space-average color. We find the illuminant estimate based on the idea that if the world is gray on average, the deviation of the brightest pixels from the achromatic value should be caused by the illuminant. Our approach utilizes multi-scale operations by only considering the salient pixels. It relies on varying surface orientations by adopting a block-based approach. We show that our strategy outperforms learning-free algorithms and provides competitive results compared to the learning-based methods. Moreover, we demonstrate that using parts of our strategy can significantly improve the performance of several learning-free methods. We also briefly present an approach to transform our global color constancy method into a multi-illuminant color constancy approach.

We recently proposed a color constancy method based on the observations that the human visual system might be "discounting the illuminant" by using space-average color and the highest luminance patches. Based on these observations, our algorithm relies on two assumptions: (i) there are several bright pixels in the scene, and (ii) the world is gray, on average. The main idea of the algorithm is to estimate the illuminant by finding the deviation of the brightest pixels from the gray value. During experiments, we observed that some pixels decrease the performance of the method. In this work, the algorithm is modified to eliminate the impact of these pixels. According to the comprehensive experiments, the proposed method surpasses several existing approaches on two color constancy benchmarks. Also, we show that the performance of some existing color constancy algorithms can be increased by using a block-based approach and salient pixels.

High dynamic range (HDR) imaging enables to immortalize natural scenes similar to the way that they are perceived by human observers. With regular low dynamic range (LDR) capture/display devices, significant details may not be preserved in images due to the huge dynamic range of natural scenes. To minimize the information loss and produce high quality HDR-like images for LDR screens, this study proposes an efficient multi-exposure fusion (MEF) approach with a simple yet effective weight extraction method relying on principal component analysis, adaptive well-exposedness and saliency maps. These weight maps are later refined through a guided filter and the fusion is carried out by employing a pyramidal decomposition. Experimental comparisons with existing techniques demonstrate that the proposed method produces very strong statistical and visual results.

The effects of strong color casts in traditional, learning-based and data-driven color constancy algorithms is analyzed. This is the first study investigating the response of color constancy methods to illuminants on the edges and outside the color temperature curve. According to the comprehensive experiments, while traditional studies do not fail to "discount the illuminant" from inputs which have strong color casts, the efficiency of learning-based and data-driven algorithms in obtaining canonical outputs decreases significantly compared to traditional methods. We discuss the reasons behind this performance decay and introduce a traditional color constancy algorithm, which presents competitive results in a challenging dataset.

The goal of intrinsic image decomposition is to recover low level features of images. Most of the studies tend to consider only reflectance and shading, even though it is known that increasing the number of intrinsics is beneficial for many applications. Existing intrinsic image datasets are quite limited. In this study, a dataset is introduced to provide a comprehensive benchmark to the field of intrinsic image decomposition. IID-NORD contains a large number of scenes and for each scene ground truth reflectance, shading, surface normal vectors, light vectors and depth map is provided to allow detailed decomposition. Moreover, diverse illuminants, viewing angles, and dynamic shadows are used to prevent any bias. To the best of available knowledge, IID-NORD is the most comprehensive dataset in the field of intrinsic image decomposition. IID-NORD will be available on the first author’s official webpage.

Tone-mapping is one of the prevailing methods to overcome high dynamic range imaging limitations over low dynamic range display devices, but the tone-mapped output image may suffer from saturated regions with texture and color information loss. In this paper, a novel approach is proposed to solve the so-called clipping problem in tone-mapped high dynamic range images. A successful saturation correction framework, which relies on linear embeddings, difference of pixel intensities and gradient-guided block-search, is developed as a post-processing technique to tone-mapping. Experimental results demonstrate that the proposed method successfully recovers clipped regions for the saturation problem in tone-mapped output images while avoiding artifacts.

This paper proposes an effective technique for multi-exposure image fusion and visible-infrared image fusion problems. Multi-exposure fusion algorithms generally extract faulty weight maps when the input stack contains multiple and/or severely over-exposed images. To overcome this issue, an alternative method is developed for weight map characterization and refinement in addition to the perspectives of linear embeddings of images and adaptive morphological masking. This framework has then been extended to the visible and infrared image fusion problem. The comprehensive experimental comparisons demonstrate that the proposed algorithm significantly enhances the fused image quality both statistically and visually.