Variable-rate learning excels in standard dynamic range (SDR) image compression, but extending it to high dynamic range (HDR) images is challenging. We propose an end-to-end Variable-Rate Learned HDR (VRLHDR) compression framework.

Reproduction of occlusions and opaque surfaces are the major challenges of additive optical see-through (OST) displays. This is because the user of an OST display sees a linear mixture of display and environment light, which creates an impression of transparency unless the displayed color is sufficiently bright. The primary goal of this work is to determine how bright a displayed surface needs to be in relation to environment light to be perceived as opaque. We test multiple factors that could affect the perception of opacity: background luminance, contrast, spatial frequency, and accommodation depth in foveal vision. The subjective results, collected on a high-dynamic-range multi-focal stereo display, indicate that a virtual object needs to be, on average, 60 times brighter than the background environment light to be perceived as opaque. A higher contrast of the texture of the virtual object and a background that is out of focus can reduce the required luminance ratio. We demonstrate that a model of visual perception based on Weber’s law and accounting for contrast masking and defocus blur can predict the experimental data with an averaged prediction error of 8.29%. Existing perceptual image difference metrics (PSNR, FovVideoVDP and HDR-VDP-3) can also predict the effect of major factors, but with lower accuracy (e.g. prediction error of 34% for PSNR with PU21 encoding).

In this paper, we present a novel perception-driven hybrid rendering method leveraging the limitation of the human visual system (HVS). Features accounted in our model include: foveation from the visual acuity eccentricity (VAE), depth of field (DOF) from vergence & accommodation, and longitudinal chromatic aberration (LCA) from color vision. To allocate computational workload efficiently, first we apply a gaze-contingent geometry simplification. Then we convert the coordinates from screen space to polar space with a scaling strategy coherent with VAE. Upon that, we apply a stochastic sampling based on DOF. Finally, we post-process the Bokeh for DOF, which can at the same time achieve LCA and anti-aliasing. A virtual reality (VR) experiment on 6 Unity scenes with a head-mounted display (HMD) HTC VIVE Pro Eye yields frame rates range from 25.2 to 48.7 fps. Objective evaluation with FovVideoVDP - a perceptual based visible difference metric - suggests that the proposed method gives satisfactory just-objectionable-difference (JOD) scores across 6 scenes from 7.61 to 8.69 (in a 10 unit scheme). Our method achieves better performance compared with the existing methods while having the same or better level of quality scores.

This study presents a framework to predict, in a No Reference (NR) manner, Full Reference (FR) objective quality metrics. The methods are applied to infrared (IR) images acquired by Unmanned Aerial Vehicle (UAV) and compressed on-board and then streamed to a ground computer. The proposed method computes two kinds of features, namely Bitstream Based (BB) features which are estimated from the H.264 bitstream and Pixel Based (PB) features which are estimated from the decoded images. Two BB features are computed using the H.264 Quantization Parameter (QP) and estimated PSNR [1]. A total of 53 PB features are calculated based on spatial information and the rest of the features are based on NR quality assessment methods [1, 2, 3]. The most relevant ones are selected and nally mapped to predict FR objective scores using Support Vector Regression. For the performance evaluation, the proposed method is trained to predict scores of 6 FR image quality metrics (SSIM, NQM, MSSIM, FSIM, MAD and PSNR-HMA) using a set of 250 IR aerial images compressed at 4 levels with H.264/AVC as I-frames. For the SVR mapping, 80% of the contents are used for training (200 contents or 800 images) and the remaining 200 images (20%) for testing. We have evaluated our model for three cases; all features, only BB features and finally excluding BB features. The average SROCC values obtained are 0.970, 0.962 and 0.943, respectively. The BB only version achieves very close results to that of using all features. Thus the presented NR BB Image Quality Assessment (IQA) method for the considered IR image material is very ecient. We have compared our method with three NR methods [1, 2, 3]. The proposed method is competitive compared to the state-of-the-art NR algorithms.

In this paper viewpoint adaptive display of HDR images incorporating the effects of ambient light is presented and evaluated. LED backlight displays may render HDR images, but while at a global scale a high dynamic range may be achieved, locally the contrast is limited by the leakage of light through the LC elements of the display. To render high quality images, the display with backlight dimming can compute the values of the LED backlight and LC elements based on the input image, information about the viewpoint of the observer(s) and information of the ambient light. The goal is to achieve the best perceptual reproduction of the specified target image derived from the HDR input image in the specific viewing situation including multiple viewers, possibly having different preferences. An optimization based approach is presented. Some tests with reproduced images are also evaluated subjectively and by image quality metrics as HDR-VDP-2.

We propose a low-complexity High Dynamic Range (HDR) infrared image (IR) coding algorithm assuming the typical case of IR images with an active range of more than 8 bit depth, but less than 16 bit depth. First, we separate an input image into base and residual images with maximum 8 bit depth each. Then we compress each image by a JPEG baseline encoder and include the residual image bit stream into the application part of JPEG header of the base image. As a result, the base image can be reconstructed by JPEG baseline decoder. If the JPEG bit stream size of the residual image is higher than the raw data size, then we include the raw residual image instead. If the residual image contains only zero values or the quality factor for it is 0 then we do not include the residual image into the header. Experimental results show that compared with JPEG-XT Part 6 with `global Reinhard' tone-mapping, the proposed approach has lower complexity and similar rate-distortion performance on IR test images.

This paper investigates the impact of ambient light and peak white (maximum brightness of a display) on the perceived quality of videos displayed using local backlight dimming. Two subjective tests providing quality evaluations are presented and analyzed. The analyses of variance show significant interactions of the factors peak white and ambient light with the perceived quality. Therefore, we proceed to predict the subjective quality grades with objective measures. The rendering of the frames on liquid crystal displays with light emitting diodes backlight at various ambient light and peak white levels is computed using a model of the display. Widely used objective quality metrics are applied based on the rendering models of the videos to predict the subjective evaluations. As these predictions are not satisfying, three machine learning methods are applied: partial least square regression, elastic net, and support vector regression. The elastic net method obtains the best prediction accuracy with a spearman rank order correlation coefficient of 0.71, and two features are identified as having a major influence on the visual quality.

Local backlight dimming is a technology aiming at both saving energy and improving visual quality on television sets. As the rendition of the image is specified locally, the numerical signal corresponding to the displayed image needs to be computed through a model of the display. This simulated signal can then be used as input to objective quality metrics. The focus of this paper is on determining which characteristics of locally backlit displays influence quality assessment. A subjective experiment assessing the quality of highly contrasted videos displayed with various local backlight-dimming algorithms is set up. Subjective results are then compared with both objective measures and objective quality metrics using different display models. The first analysis indicates that the most significant objective features are temporal variations, power consumption (probably representing leakage), and a contrast measure. The second analysis shows that modeling of leakage is necessary for objective quality assessment of sequences displayed with local backlight dimming.

In this paper a subjective test in which participants evaluate the quality of JPEG-XT compressed HDR images is presented. Results show that for the selected test images and display, the subjective quality reached its saturation point starting around 3bpp. Objective evaluations are obtained by applying a model of the display and providing the modeled images to three objective metrics dedicated to HDR content. Objective grades are compared with subjective data both in physical domain and using a gamma correction to approximate perceptually uniform luminance coding. The MRSE metric obtains the best performance with the limit that it does not capture the quality saturation. The usage of the gamma correction prior to applying metrics depends on the characteristics of each objective metric.

Local backlight dimming of LCD with LED backlight can reduce power consumption and improve quality of displayed images and videos. However, important variations of LED over time produce a visually annoying artifact called flickering. In this work, we propose a new algorithm to reduce flickering while maintaining video quality. The proposed algorithm uses an adaptive second order Infinite Impulse Response (IIR) in which coefficients are calculated from the local image features. Experimental results show that the proposed method can reduce flickering while simultaneously keeping similar video quality in terms of PSNR and MSE.

Objective image and video quality metrics focus mostly on the digital representation of the signal. However, the display characteristics are also essential for the overall Quality of Experience (QoE). In this paper, we use a model of a backlight dimming system for Liquid Crystal Display (LCD) and show how the modeled image can be used as an input to quality assessment algorithms. For quality assessment, we propose an image quality metric, based on Peak Signal-to-Noise Ratio (PSNR) computation in the CIE L*a*b* color space. The metric takes luminance reduction, color distortion and loss of uniformity in the resulting image in consideration. Subjective evaluations of images generated using different backlight dimming algorithms and clipping strategies show that the proposed metric estimates the perceived image quality more accurately than conventional PSNR.