Recent advances in cloud rendering have brought us a promising alternative for interactive photorealistic rendering on lightweight devices, which used to be only available on high-end platforms equipped with powerful graphic cards. This technique enables users to perform rendering-related creative tasks, such as 3D product visualization and lighting design, from the comfort of any location using handheld devices, rather than being confined to the front of a noisy heat-generating workstation. However, existing large-scale cloud rendering systems that stream path-traced frames from the server to the client present extremely high rendering costs and transmission bandwidth requirements, even with advanced path-tracing acceleration and video compression techniques. To alleviate these problems, we propose a novel streaming-aware rendering framework that is able to learn a joint optimal model integrating two path-tracing acceleration techniques (adaptive sampling and denoising) and video compression technique. Our joint model can fully exploit the inherent connections between these techniques and thus achieve substantially reduced rendering costs and enhanced compression quality. We also introduce the collaboration of client rendering ability to assist the frame decoding by rendering G-buffers as the shared side information. We demonstrate that appropriately incorporating the geometry and material priors from G-buffers into a neural compression pipeline can significantly reduce the streaming bandwidth in a cloud rendering system, and lighten the compression module design for computation efficiency. Our experiments show that our method delivers the best quality at various bitrates compared to existing Monte Carlo rendering streaming schemes, while remaining lightweight and efficient for cross-platform thin clients, including mobiles and tablets.

Conventionally, interior lighting design is technically complex yet challenging and requires professional knowledge and aesthetic disciplines of designers. This article presents a new digital lighting design framework for virtual interior scenes, which allows novice users to automatically obtain lighting layouts and interior rendering images with visually pleasing lighting effects. The proposed framework utilizes neural networks to retrieve and learn underlying design guidelines and the principles beneath the existing lighting designs, e.g., a newly constructed dataset of 6k 3D interior scenes from professional designers with dense annotations of lights. With a 3D furniture-populated indoor scene as the input, the framework takes two stages to perform lighting design: (1) lights are iteratively placed in the room; (2) the colors and intensities of the lights are optimized by an adversarial scheme, resulting in lighting designs with aesthetic lighting effects. Quantitative and qualitative experiments show that the proposed framework effectively learns the guidelines and principles and generates lighting designs that are preferred over the rule-based baseline and comparable to those of professional human designers.

Abstract Real‐time Monte Carlo denoising aims at removing severe noise under low samples per pixel (spp) in a strict time budget. Recently, kernel‐prediction methods use a neural network to predict each pixel's filtering kernel and have shown a great potential to remove Monte Carlo noise. However, the heavy computation overhead blocks these methods from real‐time applications. This paper expands the kernel‐prediction method and proposes a novel approach to denoise very low spp (e.g., 1‐spp) Monte Carlo path traced images at real‐time frame rates. Instead of using the neural network to directly predict the kernel map, i.e., the complete weights of each per‐pixel filtering kernel, we predict an encoding of the kernel map, followed by a high‐efficiency decoder with unfolding operations for a high‐quality reconstruction of the filtering kernels. The kernel map encoding yields a compact single‐channel representation of the kernel map, which can significantly reduce the kernel‐prediction network's throughput. In addition, we adopt a scalable kernel fusion module to improve denoising quality. The proposed approach preserves kernel prediction methods’ denoising quality while roughly halving its denoising time for 1‐spp noisy inputs. In addition, compared with the recent neural bilateral grid‐based real‐time denoiser, our approach benefits from the high parallelism of kernel‐based reconstruction and produces better denoising results at equal time.