Abstract Rendering shadows for semi‐transparent objects like smoke significantly enhances the realism of the final image. With advancements in ray tracing hardware, tracing visibility rays in real time has become possible. However, generating shadows for semi‐transparent objects requires evaluating multiple or all intersections along the ray, resulting in a deep shadow ray. Deep Shadow Maps (DSM) offer an alternative but are constrained by their fixed resolution. We introduce Importance Deep Shadow Maps (IDSM), a real‐time algorithm that adaptively distributes Deep Shadow samples based on importance captured from the current camera viewport. Additionally, we propose a novel DSM data structure built on the ray tracing acceleration structure, improving performance for scenarios requiring many samples per DSM texel. Our IDSM approach achieves speedups of up to ×6.89 compared to hardware ray tracing while maintaining a nearly indistinguishable quality level.

Abstract A main challenge for today's renderers is the ever‐growing size of 3D scenes, exceeding the capacity of typically available main memory. This especially holds true for graphics processing units (GPUs) which could otherwise be used to greatly reduce rendering time. A lot of the memory is spent on detailed geometry with mostly imperceptible influence on the final image, even in a global illumination context. Illumination‐driven mesh reduction, a Monte Carlo–based global illumination simulation, steers its mesh reduction towards areas with low visible contribution. While this works well for preserving high‐energy light paths such as caustics, it does have problems: First, objects casting shadows while not being visible themselves are not preserved, resulting in highly inaccurate shadows. Secondly, non‐transparent objects lack proper reduction guidance since there is no importance gradient on their backside, resulting in visible over‐simplification. We present a solution to these problems by extending illumination‐driven mesh reduction with occluder information, focusing on their silhouettes as well as combining it with commonly used error quadrics to preserve geometric features. Additionally, we demonstrate that the combined algorithm still supports iterative refinement of initially reduced geometry, resulting in an image visually similar to an unreduced rendering and enabling out‐of‐core operation.

Abstract Today, Monte Carlo light transport algorithms are used in many applications to render realistic images. Depending on the complexity of the used methods, several light effects can or cannot be found by the sampling process. Especially, specular and smooth glossy surfaces often lead to high noise and missing light effects. Path space regularization provides a solution, improving any sampling algorithm, by modifying the material evaluation code. Previously, Kaplanyan and Dachsbacher [KD13] introduced the concept for pure specular interactions. We extend this idea to the commonly used microfacet models by manipulating the roughness parameter prior to the evaluation. We also show that this kind of regularization requires a change in the MIS weight computation and provide the solution. Finally, we propose two heuristics to adaptively reduce the introduced bias. Using our method, many complex light effects are reproduced and the fidelity of smooth objects is increased. Additionally, if a path was sampleable before, the variance is partially reduced.

Augmented Reality offers many applications today, especially on mobile devices. Due to the lack of mobile hardware for illumination measurements, photorealistic rendering with consistent appearance of virtual objects is still an area of active research. In this paper, we present a full two-stage pipeline for environment acquisition and augmentation of live camera images using a mobile device with a depth sensor. We show how to directly work on a recorded 3D point cloud of the real environment containing high dynamic range color values. For unknown and automatically changing camera settings, a color compensation method is introduced. Based on this, we show photorealistic augmentations using variants of differential light simulation techniques. The presented methods are tailored for mobile devices and run at interactive frame rates. However, our methods are scalable to trade performance for quality and can produce quality renderings on desktop hardware.

In this paper we present a new real-time approach for indirect global illumination under dynamic lighting conditions. We use surfels to gather a sampling of the local illumination and propagate the light through the scene using a hierarchy and a set of precomputed light transport paths. The light is then aggregated into caches for lighting of static and dynamic geometry. By using a spherical harmonics representation, caches preserve incident light directions to allow both diffuse and slightly glossy BRDFs for indirect lighting.

Abstract In recent years, much work was devoted to the design of light editing methods such as relighting and light path editing. So far, little work addressed the target‐based manipulation and animation of caustics, for instance to a differently‐shaped caustic, text or an image. The aim of this work is the animation of caustics by blending towards a given target irradiance distribution. This enables an artist to coherently change appearance and style of caustics, e.g., for marketing applications and visual effects. Generating a smooth animation is nontrivial, as photon density and caustic structure may change significantly. Our method is based on the efficient solution of a discrete assignment problem that incorporates constraints appropriate to make intermediate blends plausibly resemble caustics. The algorithm generates temporally coherent results that are rendered with stochastic progressive photon mapping. We demonstrate our system in a number of scenes and show blends as well as a key frame animation.

Mobile devices are part of our everyday life and allow augmented reality (AR) with their integrated camera image. Recent research has shown that even photorealistic augmentations with consistent illumination are possible. A method, achieving this first, distributed lighting computations and the extraction of the important light sources. To reach real-time frame rates on a mobile device, the number of these extracted light sources must be low, limiting the scope of possible illumination scenarios and the quality of shadows. In this paper, we show how to reduce the computational cost per light using a combination of tile-based rendering and frustum culling techniques tailored for AR applications. Our approach runs entirely on the GPU and does not require any precomputation. Without reducing the displayed image quality, we achieve up to 2.2× speedup for typical AR scenarios.

Abstract Even though much research was dedicated to the acceleration of consistent, progressive light transport simulations, the computation of fully converged images is still very time‐consuming. This is problematic, as for the practical use in production pipelines, the rapid editing of lighting effects is important. While previous approaches restart the simulation with every scene manipulation, we make use of the coherence between frames before and after a modification in order to accelerate convergence of the context that remained similar. This is especially beneficial if a scene is edited that has already been converging for a long time, because much of the previous result can be reused, e.g., sharp caustics cast or received by the unedited scene parts. In its essence, our method performs the scene modification stochastically by predicting and accounting for the difference image. In addition, we employ two heuristics to handle cases in which stochastic removal is likely to lead to strong noise. Typical scene interactions can be broken down into object adding and removal, material substitution, camera movement and light editing, which we all examine in a number of test scenes both qualitatively and quantitatively. As we focus on caustics, we chose stochastic progressive photon mapping as the underlying light transport algorithm. Further, we show preliminary results of bidirectional path tracing and vertex connection and merging.

Abstract Progressive light transport simulations aspire a physically‐based, consistent rendering to obtain visually appealing illumination effects, depth and realism. Thereby, the handling of large scenes is a difficult problem, as in typical scene subdivision approaches the parallel processing requires frequent synchronization due to the bouncing of light throughout the scene. In practice, however, only few object parts noticeably contribute to the radiance observable in the image, whereas large areas play only a minor role. In fact, a mesh simplification of the latter can go unnoticed by the human eye. This particular importance to the visible radiance in the image calls for an output‐sensitive mesh reduction that allows to render originally out‐of‐core scenes on a single machine without swapping of memory. Thus, in this paper, we present a preprocessing step that reduces the scene size under the constraint of radiance preservation with focus on high‐frequency effects such as caustics. For this, we perform a small number of preliminary light transport simulation iterations. Thereby, we identify mesh parts that contribute significantly to the visible radiance in the scene, and which we thus preserve during mesh reduction.

At present, photorealistic augmentation is not yet possible since the computational power of mobile devices is insufficient. Even streaming solutions from stationary PCs cause a latency that affects user interactions considerably. Therefore, we introduce a differential rendering method that allows for a consistent illumination of the inserted virtual objects on mobile devices, avoiding delays. The computation effort is shared between a stationary PC and the mobile devices to make use of the capacities available on both sides. The method is designed such that only a minimum amount of data has to be transferred asynchronously between the participants. This allows for an interactive illumination of virtual objects with a consistent appearance under both temporally and spatially varying real illumination conditions. To describe the complex near-field illumination in an indoor scenario, HDR video cameras are used to capture the illumination from multiple directions. In this way, sources of illumination can be considered that are not directly visible to the mobile device because of occlusions and the limited field of view. While our method focuses on Lambertian materials, we also provide some initial approaches to approximate non-diffuse virtual objects and thereby allow for a wider field of application at nearly the same cost.

Mobile devices become more and more important today, especially for augmented reality (AR) applications in which the camera of the mobile device acts like a window into the mixed reality world. Up to now, no photorealistic augmentation is possible since the computational power of the mobile devices is still too weak. Even a streaming solution from a stationary PC would cause a latency that affects user interactions considerably. Therefore, we introduce a differential illumination method that allows for a consistent illumination of the inserted virtual objects on mobile devices, avoiding a delay. The necessary computation effort is shared between a stationary PC and the mobile devices to make use of the capacities available on both sides. The method is designed such that only a minimum amount of data has to be transferred asynchronously between the stationary PC and one or multiple mobile devices. This allows for an interactive illumination of virtual objects with a consistent appearance under both temporally and spatially varying real illumination conditions. To describe the complex near-field illumination in an indoor scenario, multiple HDR video cameras are used to capture the illumination from multiple directions. In this way, sources of illumination can be considered that are not directly visible to the mobile device because of occlusions and the limited field of view of built-in cameras.

Abstract At present, stochastic progressive photon mapping (SPPM) is one of the most comprehensive methods for a consistent global illumination computation. Even though the number of photons is unlimited due to their progressive nature, the scene size is still bound by the available main memory. In this paper, we present the first consistent out‐of‐core SPPM algorithm. In order to cope with large scenes, we automatically subdivide the geometry and parallelly trace photons and eye rays in a portal‐based system, distributed across multiple machines in a commodity cluster. Moreover, modifications of the original SPPM method are introduced that keep both the utilization of tracer machines high and the network traffic low. Therefore, compared to a portal‐based single machine setup, our distributed approach achieves a significant speedup. We compare a GPU‐based with a CPU‐based implementation and demonstrate our system in multiple large test scenes of up to 90 million triangles.

Abstract The interaction of light and matter in the world surrounding us is of striking complexity and beauty. Since the very beginning of computer graphics, adequate modelling of these processes and efficient computation is an intensively studied research topic and still not a solved problem. The inherent complexity stems from the underlying physical processes as well as the global nature of the interactions that let light travel within a scene. This paper reviews the state of the art in interactive global illumination (GI) computation, i.e., methods that generate an image of a virtual scene in less than 1 s with an as exact as possible, or plausible, solution to the light transport. Additionally, the theoretical background and attempts to classify the broad field of methods are described. The strengths and weaknesses of different approaches, when applied to the different visual phenomena, arising from light interaction are compared and discussed. Finally, the paper concludes by highlighting design patterns for interactive GI and a list of open problems.

Abstract Stochastic Progressive Photon Mapping (SPPM) is a method to simulate consistent global illumination. It is especially useful for complicated light paths like caustics seen through a glass surface. Up to now, SPPM can only be applied to a static scene and noise‐free images require hours to compute. Our approach is to extend this method to dynamic scenes (DSPPM) for an efficient simulation of animated objects and materials. We identify both hit point and photon information that can be re‐used for the pixel statistics of multiple frames. In comparison to an SPPM simulation performed for each frame, we achieve a 1.96 −9.53 speedup in our test scenes without changing correctness or simulation quality.

Computing a global illumination solution in real-time is still an open problem. We introduce Voxel-based Global Illumination (VGI), a scalable technique that ranges from real-time near-field illumination to interactive global illumination solutions. To obtain a voxelized scene representation, we introduce a new atlas-based boundary voxelization algorithm and an extension to a fast ray-voxel intersection test. Similar to screen-space illumination methods, VGI is independent of the scene complexity. Using voxels for indirect visibility enables real-time near-field illumination without the screen-space artifacts of alternative methods. Furthermore, VGI can be extended to interactive, multi-bounce global illumination solutions like path tracing and instant radiosity.

Volume caustics are intricate illumination patterns formed by light first interacting with a specular surface and subsequently being scattered inside a participating medium. Although this phenomenon can be simulated by existing techniques, image synthesis is usually non-trivial and time-consuming.

Physically plausible illumination at real-time framerates is often achieved using approximations. One popular example is ambient occlusion (AO), for which very simple and efficient implementations are used extensively in production. Recent methods approximate AO between nearby geometry in screen space (SSAO). The key observation described in this paper is, that screen-space occlusion methods can be used to compute many more types of effects than just occlusion, such as directional shadows and indirect color bleeding. The proposed generalization has only a small overhead compared to classic SSAO, approximates direct and one-bounce light transport in screen space, can be combined with other methods that simulate transport for macro structures and is visually equivalent to SSAO in the worst case without introducing new artifacts. Since our method works in screen space, it does not depend on the geometric complexity. Plausible directional occlusion and indirect lighting effects can be displayed for large and fully dynamic scenes at real-time frame rates.

In this article we evaluate the use of approximate visibility for efficient global illumination. Traditionally, accurate visibility is used in light transport. However, the indirect illumination we perceive on a daily basis is rarely of high-frequency nature, as the most significant aspect of light transport in real-world scenes is diffuse, and thus displays a smooth gradation. This raises the question of whether accurate visibility is perceptually necessary in this case. To answer this question, we conduct a psychophysical study on the perceptual influence of approximate visibility on indirect illumination. This study reveals that accurate visibility is not required and that certain approximations may be introduced.

Recent approaches to global illumination for dynamic scenes achieve interactive frame rates by using coarse approximations to geometry, lighting, or both, which limits scene complexity and rendering quality. High-quality global illumination renderings of complex scenes are still limited to methods based on ray tracing. While conceptually simple, these techniques are computationally expensive. We present an efficient and scalable method to compute global illumination solutions at interactive rates for complex and dynamic scenes. Our method is based on parallel final gathering running entirely on the GPU. At each final gathering location we perform micro-rendering: we traverse and rasterize a hierarchical point-based scene representation into an importance-warped micro-buffer , which allows for BRDF importance sampling. The final reflected radiance is computed at each gathering location using the micro-buffers and is then stored in image-space. We can trade quality for speed by reducing the sampling rate of the gathering locations in conjunction with bilateral upsampling. We demonstrate the applicability of our method to interactive global illumination, the simulation of multiple indirect bounces, and to final gathering from photon maps.

We present a method for interactive computation of indirect illumination in large and fully dynamic scenes based on approximate visibility queries. While the high-frequency nature of direct lighting requires accurate visibility, indirect illumination mostly consists of smooth gradations, which tend to mask errors due to incorrect visibility. We exploit this by approximating visibility for indirect illumination with imperfect shadow maps ---low-resolution shadow maps rendered from a crude point-based representation of the scene. These are used in conjunction with a global illumination algorithm based on virtual point lights enabling indirect illumination of dynamic scenes at real-time frame rates. We demonstrate that imperfect shadow maps are a valid approximation to visibility, which makes the simulation of global illumination an order of magnitude faster than using accurate visibility.

We present a new approach for enhancing local scene contrast by unsharp masking over arbitrary surfaces under any form of illumination. Our adaptation of a well-known 2D technique to 3D interactive scenarios is designed to aid viewers in tasks like understanding complex or detailed geometric models, medical visualization and navigation in virtual environments. Our holistic approach enhances the depiction of various visual cues, including gradients from surface shading, surface reflectance, shadows, and highlights, to ease estimation of viewpoint, lighting conditions, shapes of objects and their world-space organization. Motivated by recent perceptual findings on 3D aspects of the Cornsweet illusion, we create scene coherent enhancements by treating cues in terms of their 3D context; doing so has a stronger effect than approaches that operate in a 2D image context and also achieves temporal coherence. We validate our unsharp masking in 3D with psychophysical experiments showing that the enhanced images are perceived to have better contrast and are preferred over unenhanced originals. Our operator runs at real-time rates on a GPU and the effect is easily controlled interactively within the rendering pipeline.

Inserting virtual objects in real camera images with correct lighting is an active area of research. Current methods use a high dynamic range camera with a fish-eye lens to capture the incoming illumination. The main problem with this approach is the limitation to distant illumination. Therefore, the focus of our work is a real-time description of both near - and far-field illumination for interactive movement of virtual objects in the camera image of a real room. The daylight, which is coming in through the windows, produces a spatially varying distribution of indirect light in the room; therefore a near-field description of incoming light is necessary. Our approach is to measure the daylight from outside and to simulate the resulting indirect light in the room. To accomplish this, we develop a special dynamic form of the irradiance volume for real-time updates of indirect light in the room and combine this with importance sampling and shadow maps for light from outside. This separation allows object movements with interactive frame rates (10--17 fps). To verify the correctness of our approach, we compare images of synthetic objects with real objects.