The increased interest in consumer-grade high dynamic range (HDR) images and videos in recent years has caused a proliferation of HDR deghosting algorithms. Despite numerous proposals, a fast, memory-efficient, and robust algorithm has been difficult to achieve. This paper addresses this problem by leveraging the power of attention and U-Net-based neural representations and using a conservative deghosting strategy. Given two bracketed exposures of a scene, we produce an HDR image that maximally resembles the high exposure where it is well-exposed and fuses aligned information from both exposures otherwise. We evaluate the performance of our algorithm under several different challenging scenarios, using both visual and quantitative results, and show that it matches the state-of-the-art algorithms despite using only two exposures and has significantly lower computational complexity. Furthermore, the parameters of our algorithm greatly simplify deploying its different versions for devices with a variety of computational constraints, including mobile devices.

An important problem in autonomous driving is to perceive objects even under challenging illumination conditions. Despite this problem, existing solutions use low-dynamic range (LDR) images for object detection for autonomous driving. In this paper, we provide a novel analysis on whether high-dynamic range (HDR) images can provide better performance for object detection for autonomous driving. To this end, we choose a seminal deep object detector and systematically evaluate its performance when trained with (i) LDR images, (ii) HDR images, and (iii) tone-mapped LDR images for scenes with different illuminations. We show that a detector with HDR images pre-processed with normalization and gamma correction can only marginally perform better than a detector with LDR or tone-mapped LDR images. Our analysis of this unexpected finding reveals that a detector with HDR images requires significantly more samples as the space of HDR images is significantly larger than that of LDR images.

With the progress of autonomous vehicles, the sensing of the environment in more detail with higher dynamic ranges has become more important to classify surrounding objects and obstacles. While stereo HDR images for this purpose can provide advantages compared to the conventional LDR images, they suffer from limited dynamic ranges and spike-like noises due to the inaccuracies in disparity estimation. In this paper, we formulate the HDR image construction problem from trifocal multi-exposure images and develop a method which improves disparity estimation for better HDR image construction. Given the symmetric geometry of the trifocal setup, the proposed method uses the equivalence of disparities from middle to left and middle to right images to determine the reliable regions. The HDR radiance for the pixels in these reliable regions are estimated by using the weighted average of the warped images in different exposures and the middle image, whereas the radiance values outside the reliable regions are estimated by using only the middle image. The experiments with different exposure combinations for left, middle and right images reveal better performances of the proposed method compared to the stereo HDR imaging. It is also observed that the improvements are more apparent for larger disparities between the cameras.

Abstract Image editing is a commonly studied problem in computer graphics. Despite the presence of many advanced editing tools, there is no satisfactory solution to controllably update the position of the sun using a single image. This problem is made complicated by the presence of clouds, complex landscapes, and the atmospheric effects that must be accounted for. In this paper, we tackle this problem starting with only a single photograph. With the user clicking on the initial position of the sun, our algorithm performs several estimation and segmentation processes for finding the horizon, scene depth, clouds, and the sky line. After this initial process, the user can make both fine‐ and large‐scale changes on the position of the sun: it can be set beneath the mountains or moved behind the clouds practically turning a midday photograph into a sunset (or vice versa). We leverage a precomputed atmospheric scattering algorithm to make all of these changes not only realistic but also in real‐time. We demonstrate our results using both clear and cloudy skies, showing how to add, remove, and relight clouds, all the while allowing for advanced effects such as scattering, shadows, light shafts, and lens flares.

Just noticeable quantization levels, which are conventionally used in picture coding, have been mainly developed for standard 8-bit images and low dynamic range (LDR) typical screens. The quantization levels however have not been adapted yet for high dynamic range (HDR) imaging and its accompanied HDR displays, which can reach up to a peak luminance of 4000 cd/m2. This study proposes an experimental methodology on HDR displays to determine just noticeable quantization levels for discrete cosine transform (DCT) coefficients on high luminance images. In the first stage of the proposed method, the quantization noise patterns for different DCT frequencies at different mean luminances are rendered by predicting the LED and LCD values of the two layer HDR display. Then, a two alternative forced choice based psychovisual experimental procedure using geometric search and QUEST methodology is realized by randomly presenting the rendered quantization noise at different amplitudes to the subjects in order to determine the just noticeable levels. The experiments are performed over 3 subjects for 30 different frequencies of 8×8 DCT patterns at mean luminances of 100 cd/m2and 1000 cd/m2. The results are interpreted with respect to frequency and luminance changes and from the point of utilized methodology, namely geometric search and QUEST.

HDR images have traditionally been obtained by merging multiple exposures each captured with a different exposure time. However, this approach entails longer capture times and necessitates deghosting if the captured scene contains moving objects. With the advent of modern camera sensors that can perform per-pixel exposure modulation, it is now possible to capture all of the required exposures within a single shot. The new challenge then becomes how to best combine different pixels with different exposure values into a single full-resolution and low-noise HDR image. We propose a joint multi-exposure frame deinterlacing and denoising algorithm powered by deep convolutional neural networks (DCNN). In our algorithm, we first train two DCNNs, with one tuned for reconstructing low exposures and the other for high exposures. Each DCNN takes the same mosaicked dual-ISO input image and outputs either the low exposure or high exposure depending on the type of the network. The resulting exposures can be demosaicked and converted to the desired target color space prior to HDR assembly. Our evaluations indicate that the quality of our results significantly surpasses the state-of-the-art in single-image HDR reconstruction algorithms.

Protection of visual privacy has become an indispensable component of video surveillance systems due to pervasive use of video cameras for surveillance purposes. In this paper, we propose two fully reversible privacy protection schemes implemented within the JPEG architecture. In both schemes, privacy protection is accomplished by using false colors with the first scheme being adaptable to other privacy protection filters while the second is false color-specific. Both schemes support either a lossless mode in which the original unprotected content can be fully extracted or a lossy mode, which limits file size while still maintaining intelligibility. Our method is not region-of-interest (ROI)-based and can be applied on entire frames without compromising intelligibility. This frees the user from having to define ROIs and improves security as tracking ROIs under dynamic content may fail, exposing sensitive information. Our experimental results indicate the favorability of our method over other commonly used solutions to protect visual privacy.

High dynamic range (HDR) imaging has been developed for improved visual representation by capturing a wide range of luminance values. Owing to its properties, HDR content might lead to a larger privacy intrusion, requiring new methods for privacy protection. Previously, false colours were proved to be effective for assuring privacy protection for low dynamic range (LDR) images. In this work, the reliability of false colours when used for privacy protection of HDR images represented by tone‐mapping operators (TMOs) is studied. Two different TMO techniques are tested, a simple TMO based on the Gamma transform and a more complex local TMO. Moreover, two false colour palettes are also tested, and are applied to images that result from both TMOs and also to an LDR image that represents the centre exposure in the image sequence used to create the HDR image. The degree of privacy protection is analysed through both a subjective test using crowdsourcing and an objective test using face recognition algorithms. It is concluded that the application of the two studied false colour palettes reduces the recognition accuracy with respect to both tests.

Abstract Reconstructing high dynamic range (HDR) images of a complex scene involving moving objects and dynamic backgrounds is prone to artifacts. A large number of methods have been proposed that attempt to alleviate these artifacts, known as HDR deghosting algorithms. Currently, the quality of these algorithms are judged by subjective evaluations, which are tedious to conduct and get quickly outdated as new algorithms are proposed on a rapid basis. In this paper, we propose an objective metric which aims to simplify this process. Our metric takes a stack of input exposures and the deghosting result and produces a set of artifact maps for different types of artifacts. These artifact maps can be combined to yield a single quality score. We performed a subjective experiment involving 52 subjects and 16 different scenes to validate the agreement of our quality scores with subjective judgements and observed a concordance of almost 80%. Our metric also enables a novel application that we call as hybrid deghosting, in which the output of different deghosting algorithms are combined to obtain a superior deghosting result.

Color mapping, which involves assigning colors to the individual elements of an underlying data distribution, is a commonly used method for data visualization. Although color maps are used in many disciplines and for a variety of tasks, in this study we focus on its usage for visualizing luminance maps. Specifically, we ask ourselves the question of how to best visualize a luminance distribution encoded in a high-dynamic-range (HDR) image using false colors such that the resulting visualization is the mostdescriptive. To this end, we first propose a definition for descriptiveness. We then propose a methodology to evaluate it subjectively. Then, we propose an objective metric that correlates well with the subjective evaluation results. Using this metric, we evaluate several false coloring strategies using a large number of HDR images. Finally, we conduct a second psychophysical experiment using images representing a diverse set of scenes. Our results indicate that the luminance compression method has a significant effect and the commonly used logarithmic compression is inferior to histogram equalization. Furthermore, we find that the default color scale of the Radiance global illumination software consistently performs well when combined with histogram equalization. On the other hand, the commonly used rainbow color scale was found to be inferior. We believe that the proposed methodology is suitable for evaluating future color mapping strategies as well.

Abstract Obtaining a high quality high dynamic range (HDR) image in the presence of camera and object movement has been a long‐standing challenge. Many methods, known as HDR deghosting algorithms, have been developed over the past ten years to undertake this challenge. Each of these algorithms approaches the deghosting problem from a different perspective, providing solutions with different degrees of complexity, solutions that range from rudimentary heuristics to advanced computer vision techniques. The proposed solutions generally differ in two ways: (1) how to detect ghost regions and (2) what to do to eliminate ghosts. Some algorithms choose to completely discard moving objects giving rise to HDR images which only contain the static regions. Some other algorithms try to find the best image to use for each dynamic region. Yet others try to register moving objects from different images in the spirit of maximizing dynamic range in dynamic regions. Furthermore, each algorithm may introduce different types of artifacts as they aim to eliminate ghosts. These artifacts may come in the form of noise, broken objects, under‐ and over‐exposed regions, and residual ghosting. Given the high volume of studies conducted in this field over the recent years, a comprehensive survey of the state of the art is required. Thus, the first goal of this paper is to provide this survey. Secondly, the large number of algorithms brings about the need to classify them. Thus the second goal of this paper is to propose a taxonomy of deghosting algorithms which can be used to group existing and future algorithms into meaningful classes. Thirdly, the existence of a large number of algorithms brings about the need to evaluate their effectiveness, as each new algorithm claims to outperform its precedents. Therefore, the last goal of this paper is to share the results of a subjective experiment which aims to evaluate various state‐of‐the‐art deghosting algorithms.

When preparing high dynamic range images (HDR) for display on standard monitors, it is often necessary to make a choice between global and local tone mapping. While the former is simple and efficient, it may fail to reproduce details in high contrast image regions. Although, the latter can better reproduce details in such regions, it often comes at the cost of increased complexity and computational time. In this paper, we present an algorithm that combines the best of both approaches. We perform local tone mapping only in high frequency image regions where the visibility of details can be an issue. In low frequency regions, we employ global tone mapping to save computational resources without degrading quality. Our algorithm is most suitable for tone mapping operators (TMOs) that utilize the concept of local adaptation luminances.

High dynamic-range images cannot be directly displayed on conventional display devices, but have to be tone-mapped first. For this purpose, a large set of tone-reproduction operators is currently available. However, it is unclear which operator is most suitable for any given task. In addition, different tasks may place different requirements upon each operator. In this paper we evaluate several tone-reproduction operators using a paradigm that does not require the construction of a real high dynamic-range scene, nor does it require the availability of a high dynamic-range display device. The user study involves a task that relates to the evaluation of contrast, which is an important attribute that needs to be preserved under tone reproduction.

The development of high dynamic range (HDR) imagery has brought us to the verge of arguably the largest change in image display technologies since the transition from black-and-white to color television. Novel capture and display hardware will soon enable consumers to enjoy the HDR experience in their own homes. The question remains, however, of what to do with existing images and movies, which are intrinsically low dynamic range (LDR). Can this enormous volume of legacy content also be displayed effectively on HDR displays? We have carried out a series of rigorous psychophysical investigations to determine how LDR images are best displayed on a state-of-the-art HDR monitor, and to identify which stages of the HDR imaging pipeline are perceptually most critical. Our main findings are: (1) As expected, HDR displays outperform LDR ones. (2) Surprisingly, HDR images that are tone-mapped for display on standard monitors are often no better than the best single LDR exposure from a bracketed sequence. (3) Most importantly of all, LDR data does not necessarily require sophisticated treatment to produce a compelling HDR experience. Simply boosting the range of an LDR image linearly to fit the HDR display can equal or even surpass the appearance of a true HDR image. Thus the potentially tricky process of inverse tone mapping can be largely circumvented.

High dynamic range images may be created by capturing multiple images of a scene with varying exposures. Images created in this manner are prone to ghosting artifacts, which appear if there is movement in the scene at the time of capture. This paper describes a novel approach to removing ghosting artifacts from high dynamic range images, without the need for explicit object detection and motion estimation. Weights are computed iteratively and then applied to pixels to determine their contribution to the final image. We use a non-parametric model for the static part of the scene, and a pixel's membership in this model determines its weight. In contrast to previous approaches, our technique does not rely on explicit object detection, tracking, or pixelwise motion estimates. Ghost-free images of different scenes demonstrate the effectiveness of our technique.