Computer graphics (CG) rendering platforms produce imagery with ever-increasing photo realism. The narrowing domain gap between real and synthetic imagery makes it possible to use CG images as training data for deep learning models targeting high-level computer vision tasks, such as autonomous driving and semantic segmentation. CG images, however, are currently not suitable for low-level vision tasks targeting RAW sensor images. This is because RAW images are encoded in sensor-specific color spaces and incur pre-white-balance color casts caused by the sensor’s response to scene illumination. CG images are rendered directly to a device-independent perceptual color space without needing white balancing. As a result, it is necessary to apply a mapping procedure to close the domain gap between graphics and RAW images. To this end, we introduce a framework to process graphics images to mimic RAW sensor images accurately. Our approach allows a one-to-many mapping, where a single graphics image can be transformed to match multiple sensors and multiple scene illuminations. In addition, our approach requires only a handful of example RAW-DNG files from the target sensor as parameters for the mapping process. We compare our method to alternative strategies and show that our approach produces more realistic RAW images and provides better results on three low-level vision tasks: RAW denoising, illumination estimation, and neural rendering for night photography. Finally, as part of this work, we provide a dataset of 292 realistic CG images for training low-light imaging models.

Most smartphones support the use of real-time camera filters to impart visual effects to captured images. Currently, such filters come preinstalled on-device or need to be downloaded and installed before use (e.g., Instagram filters). Recent work [24] proposed a method to extract a camera filter directly from an example photo that has already had a filter applied. The work in [24] focused only on the color and tonal aspects of the underlying filter. In this paper, we introduce a method to extract two spatially varying effects commonly used by on-device camera filters—namely, image vignetting and image grain. Specifically, we show how to extract the parameters for vignetting and image grain present in an example image and replicate these effects as an on-device filter. We use lightweight CNNs to estimate the filter parameters and employ efficient techniques—isotropic Gaussian filters and simplex noise—for regenerating the filters. Our design achieves a reasonable trade-off between efficiency and realism. We show that our method can extract vignetting and image grain filters from stylized photos and replicate the filters on captured images more faithfully, as compared to color and style transfer methods. Our method is significantly efficient and has been already deployed to millions of flagship smartphones.

In this paper, we propose a new method for an objective measurement of video quality based on edge degradation. One of the most important requirements for an objective method for video quality measurement is that it should provide consistent performances over a wide range of video sequences that are not used in the designing stage. By analyzing subjective scores of various video sequences, we found that the human visual system is sensitive to degradation around edges. In other words, when edge areas of a video are blurred, evaluators tend to give low scores to the video even though the overall mean squared error is not so large. Based on this observation, we propose an objective video quality measurement method that measures degradation around edges. In the proposed method, we first apply an edge detection algorithm to videos and find edge areas. Then, we measure degradation of those edge areas by computing mean squared error. From this mean squared error, we compute the PSNR and use it as video quality metric. Experimental results show that the proposed method compares favorably with the current objective methods for video quality measurement. Furthermore, when the proposed method is applied to test video sequences that are not used in the designing stage, it still consistently provides satisfactory performances.

We propose a modified 3-D SPIHT algorithm with 3-D resizing, which is especially targeted for low bit rate compression. In the proposed algorithm, we first reduce 3-D images using a 3-D interpolation algorithm and encode the reduced 3-D images using a 3-D SPIHT algorithm. At the decoder, the encoded bit stream is first decoded and the reconstructed 3-D images are enlarged to the original image size. Experimental results show that the proposed algorithm outperforms the conventional 3-D SPIHT algorithm at low bit rates.

We propose a modified 3-D zerotree algorithm, which enables random access of compressed hyperspectral images by using an efficient memory structure and rearranging encoding order. This method reduces accessing time and requirements of network and system resources.

We apply a fast 3D-SPIHT to compress hyperspectral images. The method uses a tree information matrix to determine whether a tree is significant or insignificant during encoding process. Experiments with AVIRIS data show that it is possible to save about 70% of processing time.