Abstract Image processing techniques can be used to modulate the pixel intensities of an image to reduce the power consumption of the display device. A simple example of this consists of uniformly dimming the entire image. Such algorithms should strive to minimize the impact on image quality while maximizing power savings. Techniques based on heuristics or human perception have been proposed, both for traditional flat panel displays and modern display modalities such as virtual and augmented reality (VR/AR). In this paper, we focus on developing and evaluating display power‐saving techniques that use machine learning (ML) in VR displays. We developed a U‐Net‐based technique paired with perceptual and power optimization loss functions that generates spatially varying dimming maps. These dimming maps are used to modulate input images, per‐pixel, to generate a power‐efficient image. Our pipeline was validated via quantitative analysis using image quality metrics and through a subjective study. Our subjective validation provides results scaled in perceptual just‐objectionable‐difference (JOD) units. This data, when rescaled, allows for comparisons of our technique with recent studies on VR display power optimization. Our results show that participants prefer our technique over a uniform dimming baseline for high target power saving conditions. This model and study serve as a template and baseline for future applications of deep learning to display power optimization. Model training code and data can be found at kenchen10.github.io/projects/mlpea/index.html .

Display power consumption is an emerging concern for untethered devices. This goes double for augmented and virtual extended reality (XR) displays, which target high refresh rates and high resolutions while conforming to an ergonomically light form factor. A number of image mapping techniques have been proposed to extend battery usage. However, there is currently no comprehensive quantitative understanding of how the power savings provided by these methods compare to their impact on visual quality. We set out to answer this question. To this end, we present a perceptual evaluation of algorithms (PEA) for power optimization in XR displays (PODs). Consolidating a portfolio of six power-saving display mapping approaches, we begin by performing a large-scale perceptual study to understand the impact of each method on perceived quality in the wild. This results in a unified quality score for each technique, scaled in just-objectionable-difference (JOD) units. In parallel, each technique is analyzed using hardware-accurate power models. The resulting JOD-to-Milliwatt transfer function provides a first-of-its-kind look into tradeoffs offered by display mapping techniques, and can be directly employed to make architectural decisions for power budgets on XR displays. Finally, we leverage our study data and power models to address important display power applications like the choice of display primary, power implications of eye tracking, and more 1 .

Tone mapping operators aim to remap content to a display’s dynamic range. Virtual reality is a popular new display modality that has significant differences from other media, making the use of traditional tone mapping techniques difficult. Moreover, real-time adaptive estimation of tone curves that faithfully maintain appearance remains a significant challenge. In this work, we propose a real-time perceptual contrast-matching framework, that allows us to optimally remap scenes for target displays. Our framework is optimized for efficiency and runs on a mobile Quest 2 headset in under 1ms per frame. A subjective study on an HDR-VR prototype demonstrates our method’s effectiveness across a wide range of display luminances, producing imagery that is preferred to alternatives tone mapped at peak luminances an order of magnitude higher. This result highlights the importance of good tone mapping for visual quality in VR.