Abstract Path tracing uses Monte Carlo integration to solve the rendering equation by evaluating the integrand at random sampling points. The convergence rate of the error can be significantly improved by using correlated instead of random sampling, especially on smooth integrands. However, on integrands with discontinuities due to, e.g., occlusion, the improvement is less pronounced. Prior work has shown that the variance of the estimator is equal to the product of the power spectrum of the integrand and the expected power spectrum of the sampling pattern. Discontinuous integrands have anisotropic power spectra that exhibit high energies along the directions of the discontinuities, which need to match the low‐energy directions of the sampling pattern to reduce variance. However, existing anisotropic sampling patterns have at most two low‐energy directions. Therefore, we propose an optimization‐based algorithm to synthesize two‐dimensional correlated sampling patterns with spectra that have more than two low‐energy directions, leading to improved convergence behavior. Further, we propose a practical and sample‐efficient algorithm that estimates the directions of discontinuities in the power spectra of two‐dimensional integrands. We show that our algorithm can reliably estimate these directions, allowing us to align the low‐energy directions of anisotropic correlated sampling patterns with the predicted directions. We demonstrate in an offline path tracer with light source sampling that our aligned sampling patterns improve the convergence rate on two‐dimensional integrands with multiple discontinuities compared to existing anisotropic sampling patterns and thus reduce the error more quickly.

Abstract We introduce an efficient Two‐Level Monte Carlo (subset of Multi‐Level Monte Carlo, MLMC) estimator for real‐time rendering of scenes with global illumination. Using MLMC we split the shading integral into two parts: the radiance cache integral and the residual error integral that compensates for the bias of the first one. For the first part, we developed the Neural Incident Radiance Cache (NIRC) leveraging the power of tiny neural networks [MRNK21] as a building block, which is trained on the fly. The cache is designed to provide a fast and reasonable approximation of the incident radiance: an evaluation takes 2–25 × less compute time than a path tracing sample. This enables us to estimate the radiance cache integral with a high number of samples and by this achieve faster convergence. For the residual error integral, we compute the difference between the NIRC predictions and the unbiased path tracing simulation. Our method makes no assumptions about the geometry, materials, or lighting of a scene and has only few intuitive hyper‐parameters. We provide a comprehensive comparative analysis in different experimental scenarios. Since the algorithm is trained in an on‐line fashion, it demonstrates significant noise level reduction even for dynamic scenes and can easily be combined with other noise reduction techniques.

Abstract Compared to classic ray marching‐based approaches, Monte Carlo ray tracing for volume visualization can provide faster frame times through progressive rendering, improved image quality, and allows for advanced illumination models more easily. Techniques such as the view‐dependent optimization of visibility and illumination of important regions, however, have been formulated for ray marching and rely on stepwise sampling along rays, and are thus incompatible with free‐flight distance sampling of state‐of‐the‐art Monte Carlo methods. In this paper we derive such a view‐dependent optimization for Monte Carlo ray tracing where the visibility to the camera, the illumination and opacity of important regions is optimized for both single and multiple scattering rendering. For this we define a post‐interpolative importance function, introduce an efficient data structure to sample, approximate and optimize the integrated extinction along rays, and devise an efficient Monte Carlo estimator for interactive visualization. Our method enables view‐dependent visibility optimization with moderate memory overhead and unbiased, progressive Monte Carlo volume visualization. We demonstrate our method for various volume data sets as well as for data‐dependent and spatially‐dependent importance functions.

Abstract Segmentation volumes are voxel data sets often used in machine learning, connectomics, and natural sciences. Their large sizes make compression indispensable for storage and processing, including GPU video memory constrained real‐time visualization. Fast Compressed Segmentation Volumes (CSGV) [PD24] provide strong brick‐wise compression and random access at the brick level. Voxels within a brick, however, have to be decoded serially and thus rendering requires caching of visible full bricks, consuming extra memory. Without caching, accessing voxels can have a worst‐case decoding overhead of up to a full brick (typically over 32.000 voxels). We present CSGV‐R which provide true multi‐resolution random access on a per‐voxel level. We leverage Huffman‐shaped Wavelet Trees for random accesses to variable bit‐length encoding and their rank operation to query label palette offsets in bricks. Our real‐time segmentation volume visualization removes decoding artifacts from CSGV and renders CSGV‐R volumes without caching bricks at faster render times. CSGV‐R has slightly lower compression rates than CSGV, but outperforms Neuroglancer, the state‐of‐the‐art compression technique with true random access, with 2× to 4× smaller data sets at rates between 0.648% and 4.411% of the original volume sizes.

Tu Anh Dinh, Carlos Mullov, Leonard Bärmann, Zhaolin Li, Danni Liu, Simon Reiß, Jueun Lee, Nathan Lerzer, Jianfeng Gao, Fabian Peller-Konrad, Tobias Röddiger, Alexander Waibel, Tamim Asfour, Michael Beigl, Rainer Stiefelhagen, Carsten Dachsbacher, Klemens Böhm, Jan Niehues. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 2024.

Abstract Explicit sampling of and connecting to light sources is often essential for reducing variance in Monte Carlo rendering. In dense, forward‐scattering participating media, its benefit declines, as significant transport happens over longer multiple‐scattering paths around the straight connection to the light. Sampling these paths is challenging, as their contribution is shaped by the product of reciprocal squared distance terms and the phase functions. Previous work demonstrates that sampling several of these terms jointly is crucial. However, these methods are tied to low‐order scattering or struggle with highly‐peaked phase functions. We present a method for sampling a bridge: a subpath of arbitrary vertex count connecting two vertices. Its probability density is proportional to all phase functions at inner vertices and reciprocal squared distance terms. To achieve this, we importance sample the phase functions first, and subsequently all distances at once. For the latter, we sample an independent, preliminary distance for each edge of the bridge, and afterwards scale the bridge such that it matches the connection distance. The scale factor can be marginalized out analytically to obtain the probability density of the bridge. This approach leads to a simple algorithm and can construct bridges of any vertex count. For the case of one or two inserted vertices, we also show an alternative without scaling or marginalization. For practical path sampling, we present a method to sample the number of bridge vertices whose distribution depends on the connection distance, the phase function, and the collision coefficient. While our importance sampling treats media as homogeneous we demonstrate its effectiveness on heterogeneous media.

Voxel-based segmentation volumes often store a large number of labels and voxels, and the resulting amount of data can make storage, transfer, and interactive visualization difficult. We present a lossless compression technique which addresses these challenges. It processes individual small bricks of a segmentation volume and compactly encodes the labelled regions and their boundaries by an iterative refinement scheme. The result for each brick is a list of labels, and a sequence of operations to reconstruct the brick which is further compressed using rANS-entropy coding. As the relative frequencies of operations are very similar across bricks, the entropy coding can use global frequency tables for an entire data set which enables efficient and effective parallel (de)compression. Our technique achieves high throughput (up to gigabytes per second both for compression and decompression) and strong compression ratios of about 1% to 3% of the original data set size while being applicable to GPU-based rendering. We evaluate our method for various data sets from different fields and demonstrate GPU-based volume visualization with on-the-fly decompression, level-of-detail rendering (with optional on-demand streaming of detail coefficients to the GPU), and a caching strategy for decompressed bricks for further performance improvement.

Abstract We present a novel technique to efficiently render complex direct illumination in real‐time. It is based on a spatio‐temporal randomized mixture model of von Mises‐Fisher (vMF) distributions in screen space. For every pixel we determine the vMF distribution to sample from using a Markov chain process which is targeted to capture important features of the integrand. By this we avoid the storage overhead of finite‐component deterministic mixture models, for which, in addition, determining the optimal component count is challenging. We use stochastic multiple importance sampling (SMIS) to be independent of the equilibrium distribution of our Markov chain process, since it cancels out in the estimator. Further, we use the same sample to advance the Markov chain and to construct the SMIS estimator and local Markov chain state permutations avoid the resulting bias due to dependent sampling. As a consequence we require one ray per sample and pixel only. We evaluate our technique using implementations in a research renderer as well as a classic game engine with highly dynamic content. Our results show that it is efficient and quickly readapts to dynamic conditions. We compare to spatio‐temporal resampling (ReSTIR), which can suffer from correlation artifacts due to its non‐adapting candidate distributions that can deviate strongly from the integrand. While we focus on direct illumination, our approach is more widely applicable and we exemplarily show the rendering of caustics.

Finding optimal parameters for acceleration structures for raytracing is key to improved performance. Previous research has shown that a speedup of over 10% of rendering time is possible. Some parameters are interdependent which complicates the process of finding an optimal configuration. It is hence interesting to find them efficiently. Autotuning is an automatic optimization scheme able to search for optimal configurations and has been applied successfully to kD-trees in the past, which we apply today on BVHs. The more parameters to optimize, the more difficult it is to find optimal solutions. In this article, we analyze in detail the behavior of the parameters and their impact on acceleration structure building and rendering time. We show the interdependence and context sensitivity (i.e., scene, viewpoint) of the parameters. Based on the use case, this allows to target only crucial parameters. Convergence speed towards an optimal configuration is essential. To find better parameters, the autotuner needs to build the acceleration structure over and over, changing parameters every time. We introduce a hybrid model-based prediction and online autotuning method to address this issue. The prediction model allows for both instantaneous near-optimal configurations when inputs are known or similar, and efficient search of the configuration space when inputs are completely new. Online autotuning outperforms configurations recommended in literature by up to 11% median. The prediction model achieves 95% of the maximum speedup of the autotuner while reducing 90% of its overhead. Thus, hybrid online autonuning enables always-on tuning in ray tracing.

Abstract Good importance sampling strategies are decisive for the quality and robustness of photorealistic image synthesis with Monte Carlo integration. Path guiding approaches use transport paths sampled by an existing base sampler to build and refine a guiding distribution. This distribution then guides subsequent paths in regions that are otherwise hard to sample. We observe that all terms in the measurement contribution function sampled during path construction depend on at most three consecutive path vertices. We thus propose to build a 9D guiding distribution over vertex triplets that adapts to the full measurement contribution with a 9D Gaussian mixture model (GMM). For incremental path sampling, we query the model for the last two vertices of a path prefix, resulting in a 3D conditional distribution with which we sample the next vertex along the path. To make this approach scalable, we partition the scene with an octree and learn a local GMM for each leaf separately. In a learning phase, we sample paths using the current guiding distribution and collect triplets of path vertices. We resample these triplets online and keep only a fixed‐size subset in reservoirs. After each progression, we obtain new GMMs from triplet samples by an initial hard clustering followed by expectation maximization. Since we model 3D vertex positions, our guiding distribution naturally extends to participating media. In addition, the symmetry in the GMM allows us to query it for paths constructed by a light tracer. Therefore our method can guide both a path tracer and light tracer from a jointly learned guiding distribution.

We present a novel image-based representation to interactively visualize large and arbitrarily structured volumetric data. This image-based representation is created from a fixed view and models the scalar densities along each viewing ray. Then, any transfer function can be applied and changed interactively to visualize the data. In detail, we transform the density in each pixel to the Fourier basis and store Fourier coefficients of a bounded signal, i.e. bounded trigonometric moments. To keep this image-based representation compact, we adaptively determine the number of moments in each pixel and present a novel coding and quantization strategy. Additionally, we perform spatial and temporal interpolation of our image representation and discuss the visualization of introduced uncertainties. Moreover, we use our representation to add single scattering illumination. Lastly, we achieve accurate results even with changes in the view configuration. We evaluate our approach on two large volume datasets and a time-dependent SPH dataset.

Abstract In this paper, we present a novel method for the direct volume rendering of large smoothed‐particle hydrodynamics (SPH) simulation data without transforming the unstructured data to an intermediate representation. By directly visualizing the unstructured particle data, we avoid long preprocessing times and large storage requirements. This enables the visualization of large, time‐dependent, and multivariate data both as a post‐process and in situ. To address the computational complexity, we introduce stochastic volume rendering that considers only a subset of particles at each step during ray marching. The sample probabilities for selecting this subset at each step are thereby determined both in a view‐dependent manner and based on the spatial complexity of the data. Our stochastic volume rendering enables us to scale continuously from a fast, interactive preview to a more accurate volume rendering at higher cost. Lastly, we discuss the visualization of free‐surface and multi‐phase flows by including a multi‐material model with volumetric and surface shading into the stochastic volume rendering.

Rapidly growing data sizes of scientific simulations pose significant challenges for interactive visualization and analysis techniques. In this work, we propose a compact probabilistic representation to interactively visualize large scattered datasets. In contrast to previous approaches that represent blocks of volumetric data using probability distributions, we model clusters of arbitrarily structured multivariate data. In detail, we discuss how to efficiently represent and store a high-dimensional distribution for each cluster. We observe that it suffices to consider low-dimensional marginal distributions for two or three data dimensions at a time to employ common visual analysis techniques. Based on this observation, we represent high-dimensional distributions by combinations of low-dimensional Gaussian mixture models. We discuss the application of common interactive visual analysis techniques to this representation. In particular, we investigate several frequency-based views, such as density plots in 1D and 2D, density-based parallel coordinates, and a time histogram. We visualize the uncertainty introduced by the representation, discuss a level-of-detail mechanism, and explicitly visualize outliers. Furthermore, we propose a spatial visualization by splatting anisotropic 3D Gaussians for which we derive a closed-form solution. Lastly, we describe the application of brushing and linking to this clustered representation. Our evaluation on several large, real-world datasets demonstrates the scaling of our approach.

Abstract Physically based rendering of glittering surfaces is a challenging problem in computer graphics. Several methods have proposed off‐line solutions, but none is dedicated to high‐performance graphics. In this work, we propose a novel physically based BRDF for real‐time rendering of glints. Our model can reproduce the appearance of sparkling materials (rocks, rough plastics, glitter fabrics, etc.). Compared to the previous real‐time method [ZK16], which is not physically based, our BRDF uses normalized NDFs and converges to the standard microfacet BRDF [CT82] for a large number of microfacets. Our method procedurally computes NDFs with hundreds of sharp lobes. It relies on a dictionary of 1D marginal distributions: at each location two of them are randomly picked and multiplied (to obtain a NDF), rotated (to increase the variety), and scaled (to control standard deviation/roughness). The dictionary is multiscale, does not depend on roughness, and has a low memory footprint (less than 1 MiB).

Abstract Fluorescent materials can shift energy between wavelengths, thereby creating bright and saturated colors both in natural and artificial materials. However, rendering fluorescence for continuous wavelengths or combined with wavelength dependent path configurations so far has only been feasible using spectral unidirectional methods. We present a regularization‐based approach for supporting fluorescence in a spectral bidirectional path tracer. Our algorithm samples camera and light sub‐paths with independent wavelengths, and when connecting them mollifies the BSDF at one of the connecting vertices such that it reradiates light across multiple wavelengths. We discuss arising issues such as color bias in early iterations, consistency of the method and MIS weights in the presence of spectral mollification. We demonstrate our method in scenes combining fluorescence and transport phenomena that are difficult to render with unidirectional or spectrally discrete methods.

Markov Chain Monte Carlo (MCMC) rendering is extensively studied, yet it remains largely unused in practice. We propose solutions to several practicability issues, opening up path space MCMC to become an adaptive sampling framework around established Monte Carlo (MC) techniques. We address non-uniform image quality by deriving an analytic target function for imagespace sample stratification. The function is based on a novel connection between variance and path differentials, allowing analytic variance estimates for MC samples, with potential uses in other adaptive algorithms outside MCMC. We simplify these estimates down to simple expressions using only quantities known in any MC renderer. We also address the issue that most existing MCMC renderers rely on bi-directional path tracing and reciprocal transport, which can be too costly and/or too complex in practice. Instead, we apply our theoretical framework to optimize an adaptive MCMC algorithm that only uses forward path construction. Notably, we construct our algorithm by adapting (with minimal changes) a full-featured path tracer into a single-path state space Markov Chain, bridging another gap between MCMC and existing MC techniques.

We propose a data reduction technique for scattered data based on statistical sampling. Our void-and-cluster sampling technique finds a representative subset that is optimally distributed in the spatial domain with respect to the blue noise property. In addition, it can adapt to a given density function, which we use to sample regions of high complexity in the multivariate value domain more densely. Moreover, our sampling technique implicitly defines an ordering on the samples that enables progressive data loading and a continuous level-of-detail representation. We extend our technique to sample time-dependent trajectories, for example pathlines in a time interval, using an efficient and iterative approach. Furthermore, we introduce a local and continuous error measure to quantify how well a set of samples represents the original dataset. We apply this error measure during sampling to guide the number of samples that are taken. Finally, we use this error measure and other quantities to evaluate the quality, performance, and scalability of our algorithm.

Abstract Physically based spectral rendering has become increasingly important in recent years. However, asset textures in such systems are usually still drawn or acquired as RGB tristimulus values. While a number of RGB to spectrum upsampling techniques are available, none of them support upsampling of all colours in the full spectral locus, as it is intrinsically bigger than the gamut of physically valid reflectance spectra. But with display technology moving to increasingly wider gamuts, the ability to achieve highly saturated colours becomes an increasingly important feature. Real materials usually exhibit smooth reflectance spectra, while computationally generated spectra become more blocky as they represent increasingly bright and saturated colours. In print media, plastic or textile design, fluorescent dyes are added to extend the boundaries of the gamut of reflectance spectra. We follow the same approach for rendering: we provide a method which, given an input RGB tristimulus value, automatically provides a mixture of a regular, smooth reflectance spectrum plus a fluorescent part. For highly saturated input colours, the combination yields an improved reconstruction compared to what would be possible relying on a reflectance spectrum alone. At the core of our technique is a simple parametric spectral model for reflectance, excitation, and emission that allows for compact storage and is compatible with texture mapping. The model can then be used as a fluorescent diffuse component in an existing more complex BRDF model. We also provide importance sampling routines for practical application in a path tracer.

Abstract Physically based rendering is a well‐understood technique to produce realistic‐looking images. However, different algorithms exist for efficiency reasons, which work well in certain cases but fail or produce rendering artefacts in others. Few tools allow a user to gain insight into the algorithmic processes. In this work, we present such a tool, which combines techniques from information visualization and visual analytics with physically based rendering. It consists of an interactive parallel coordinates plot, with a built‐in sampling‐based data reduction technique to visualize the attributes associated with each light sample. Two‐dimensional (2D) and three‐dimensional (3D) heat maps depict any desired property of the rendering process. An interactively rendered 3D view of the scene displays animated light paths based on the user's selection to gain further insight into the rendering process. The provided interactivity enables the user to guide the rendering process for more efficiency. To show its usefulness, we present several applications based on our tool. This includes differential light transport visualization to optimize light setup in a scene, finding the causes of and resolving rendering artefacts, such as fireflies, as well as a path length contribution histogram to evaluate the efficiency of different Monte Carlo estimators.

We present a compact and efficient representation of spectra for accurate rendering using more than three dimensions. While tristimulus color spaces are sufficient for color display, a spectral renderer has to simulate light transport per wavelength. Consequently, emission spectra and surface albedos need to be known at each wavelength. It is practical to store dense samples for emission spectra but for albedo textures, the memory requirements of this approach are unreasonable. Prior works that approximate dense spectra from tristimulus data introduce strong errors under illuminants with sharp peaks and in indirect illumination. We represent spectra by an arbitrary number of Fourier coefficients. However, we do not use a common truncated Fourier series because its ringing could lead to albedos below zero or above one. Instead, we present a novel approach for reconstruction of bounded densities based on the theory of moments. The core of our technique is our bounded maximum entropy spectral estimate. It uses an efficient closed form to compute a smooth signal between zero and one that matches the given Fourier coefficients exactly. Still, a ground truth that localizes all of its mass around a few wavelengths can be reconstructed adequately. Therefore, our representation covers the full gamut of valid reflectances. The resulting textures are compact because each coefficient can be stored in 10 bits. For compatibility with existing tristimulus assets, we implement a mapping from tristimulus color spaces to three Fourier coefficients. Using three coefficients, our technique gives state of the art results without some of the drawbacks of related work. With four to eight coefficients, our representation is superior to all existing representations. Our focus is on offline rendering but we also demonstrate that the technique is fast enough for real-time rendering.

Believable immersive virtual environments require accurate sound in addition to plausible graphics to be convincing to the user. We present a practical and fast method to compute sound reflections for moving sources and listeners in virtual scenes in realtime, even on low end hardware. In a preprocessing step we compute transport paths starting from possible source positions and store virtual diffuse sources along the path. At runtime we connect these subpaths to the actual source position and the virtual diffuse sources to the microphone to compute the impulse response. Our method can be implemented as a single pass GPU rendering process and requires only a few milliseconds for each update.

Abstract Samples with high contribution but low probability density, often called fireflies, occur in all practical Monte Carlo estimators and are part of computing unbiased estimates. For finite‐sample estimates, however, they can lead to excessive variance. Rejecting all samples classified as outliers, as suggested in previous work, leads to estimates that are too low and can cause undesirable artefacts. In this paper, we show how samples can be re‐weighted depending on their contribution and sampling frequency such that the finite‐sample estimate gets closer to the correct expected value and the variance can be controlled. For this, we first derive a theory for how samples should ideally be re‐weighted and that this would require the probability density function of the optimal sampling strategy. As this probability density function is generally unknown, we show how the discrepancy between the optimal and the actual sampling strategy can be estimated and used for re‐weighting in practice. We describe an efficient algorithm that allows for the necessary analysis of per‐pixel sample distributions in the context of Monte Carlo rendering without storing any individual samples, with only minimal changes to the rendering algorithm. It causes negligible runtime overhead, works in constant memory and is well suited for parallel and progressive rendering. The re‐weighting runs as a fast post‐process, can be controlled interactively and our approach is non‐destructive in that the unbiased result can be reconstructed at any time.

Markov chain Monte Carlo (MCMC) rendering utilizes a sequence of correlated path samples which is obtained by iteratively mutating the current state to the next. The efficiency of MCMC rendering depends on how well the mutation strategy is designed to adapt to the local structure of the state space. We present a novel MCMC rendering method that automatically adapts the step sizes of the mutations to the geometry of the rendered scene. Our geometry-aware path space perturbation largely avoids tentative samples with zero contribution due to occlusion. Our method limits the mutation step size by estimating the maximum opening angle of a cone, centered around a segment of a light transport path, where no geometry obstructs visibility. This geometry-aware mutation increases the acceptance rates, while not degrading the sampling quality. As this cone estimation introduces a considerable overhead if done naively, to make our approach efficient, we discuss and analyze fast approximate methods for cone angle estimation which utilize the acceleration structure already present for the ray-geometry intersection. Our new approach, integrated into the framework of Metropolis light transport, can achieve results with lower error and less artifact in equal time compared to current path space mutation techniques.

Finding good global importance sampling strategies for Monte Carlo light transport is challenging. While estimators using local methods (such as BSDF sampling or next event estimation) often work well in the majority of a scene, small regions in path space can be sampled insufficiently (e.g. a reflected caustic). We propose a novel data-driven guided sampling method which selectively adapts to such problematic regions and complements the unguided estimator. It is based on complete transport paths, i.e. is able to resolve the correlation due to BSDFs and free flight distances in participating media. It is conceptually simple and places anisotropic truncated Gaussian distributions around guide paths to reconstruct a continuous probability density function (guided PDF). Guide paths are iteratively sampled from the guided as well as the unguided PDF and only recorded if they cause high variance in the current estimator. While plain Monte Carlo samples paths independently and Markov chain-based methods perturb a single current sample, we determine the reconstruction kernels by a set of neighbouring paths. This enables local exploration of the integrand without detailed balance constraints or the need for analytic derivatives. We show that our method can decompose the path space into a region that is well sampled by the unguided estimator and one that is handled by the new guided sampler. In realistic scenarios, we show 4× speedups over the unguided sampler.

We consider the problem of sampling n numbers from the range { 1,… , N } without replacement on modern architectures. The main result is a simple divide-and-conquer scheme that makes sequential algorithms more cache efficient and leads to a parallel algorithm running in expected time O ( n / p +log p ) on p processors, i.e., scales to massively parallel machines even for moderate values of n . The amount of communication between the processors is very small (at most O (log p )) and independent of the sample size. We also discuss modifications needed for load balancing, online sampling, sampling with replacement, Bernoulli sampling, and vectorization on SIMD units or GPUs.

In this paper we present a novel GPU-friendly real-time voxelization technique for rendering homogeneous media that is defined by particles, e.g., fluids obtained from particle-based simulations such as Smoothed Particle Hydrodynamics (SPH). Our method computes view-adaptive binary voxelizations with on-the-fly compression of a tiled perspective voxel grid, achieving higher resolutions than previous approaches. It allows for interactive generation of realistic images, enabling advanced rendering techniques such as ray casting-based refraction and reflection, light scattering and absorption, and ambient occlusion. In contrast to previous methods, it does not rely on preprocessing such as expensive, and often coarse, scalar field conversion or mesh generation steps. Our method directly takes unsorted particle data as input. It can be further accelerated by identifying fully populated simulation cells during simulation. The extracted surface can be filtered to achieve smooth surface appearance. Finally, we provide a new scheme for accelerated ray casting inside the voxelization.

Optimized machine vision setups are a requirement for precise and efficient product quality assurance. As the design space is high-dimensional, a manual design requires a lot of engineering experience and experimental work, associated with high costs and often non-optimal results. For automatic evaluation, there are evaluation metrics such as measurement uncertainty and scan resolution to evaluate the quality of an inspection. However, it is not trivial how to combine different criteria to optimize the setup based on the inspection requirements. We propose to fuse the metrics through a probabilistic surface inference to quantify the amount of information gained by a specific setup configuration. To this end, the product surface is modeled by a random process and the problem is adapted to a Gaussian Process (GP) inference. We introduce a local inference based on the local surface orientation and propose a novel spectrum-based approach to determine the GP parameters based on the required inspection resolution. The inference results, based on simulations, are demonstrated for the inspection of a cylinder head.

Abstract Emissive media are often challenging to render: in thin regions where only few scattering events occur the emission is poorly sampled, while sampling events for emission can be disadvantageous due to absorption in dense regions. We extend the standard path space measurement contribution to also collect emission along path segments, not only at vertices. We apply this extension to two estimators: extending paths via scattering and distance sampling, and next event estimation. In order to do so, we unify the two approaches and derive the corresponding Monte Carlo estimators to interpret next event estimation as a solid angle sampling technique. We avoid connecting paths to vertices hidden behind dense absorbing layers of smoke by also including transmittance sampling into next event estimation. We demonstrate the advantages of ourline integrationapproach which generates estimators with lower variance since entire segments are accounted for. Also, our novelforward next event estimationtechnique yields faster run times compared to previous next event estimation as it penetrates less deeply into dense volumes.

We present a new technique called Multiple Vertex Next Event Estimation, which outperforms current direct lighting techniques in forward scattering, optically dense media with the Henyey-Greenstein phase function. Instead of a one-segment connection from a vertex within the medium to the light source, an entire sub path of arbitrary length can be created and we show experimentally that 4–10 segments work best in practice. This is done by perturbing a seed path within the Monte Carlo context. Our technique was integrated in a Monte Carlo renderer, combining random walk path tracing with multiple vertex next event estimation via multiple importance sampling for an unbiased result. We evaluate this new technique against standard next event estimation and show that it significantly reduces noise and increases performance of multiple scattering renderings in highly anisotropic, optically dense media. Additionally, we discuss multiple light sources and performance implications of memory-heavy heterogeneous media.

Rendering algorithms using Markov chain Monte Carlo (MCMC) currently build upon two different state spaces. One of them is the path space, where the algorithms operate on the vertices of actual transport paths. The other state space is the primary sample space, where the algorithms operate on sequences of numbers used for generating transport paths. While the two state spaces are related by the sampling procedure of transport paths, all existing MCMC rendering algorithms are designed to work within only one of the state spaces. We propose a first framework which provides a comprehensive connection between the path space and the primary sample space. Using this framework, we can use mutation strategies designed for one space with mutation strategies in the respective other space. As a practical example, we take a combination of manifold exploration and multiplexed Metropolis light transport using our framework. Our results show that the simultaneous use of the two state spaces improves the robustness of MCMC rendering. By combining efficient local exploration in the path space with global jumps in primary sample space, our method achieves more uniform convergence as compared to using only one space.

Normal mapping enhances the amount of visual detail of surfaces by using shading normals that deviate from the geometric normal. However, the resulting surface model is geometrically impossible and normal mapping is thus often considered a fundamentally flawed approach with unavoidable problems for Monte Carlo path tracing, such as asymmetry, back-facing normals, and energy loss arising from this incoherence. These problems are usually sidestepped in real-time renderers, but they cannot be fixed robustly in a path tracer: normal mapping breaks either the appearance (black fringes, energy loss) or the integrator (different forward and backward light transport); in practice, workarounds and tweaked normal maps are often required to hide artifacts. We present microfacet-based normal mapping, an alternative way of faking geometric details without corrupting the robustness of Monte Carlo path tracing. It takes the same input data as classic normal mapping and works with any input BRDF. Our idea is to construct a geometrically valid microfacet surface made of two facets per shading point: the one given by the normal map at the shading point and an additional facet that compensates for it such that the average normal of the microsurface equals the geometric normal. We derive the resulting microfacet BRDF and show that it mimics geometric detail in a plausible way, although it does not replicate the appearance of classic normal mapping. However, our microfacet-based normal mapping model is well-defined, symmetric, and energy conserving, and thus yields identical results with any path tracing algorithm (forward, backward, or bidirectional).

We present a novel method to optimize the attenuation of light for the single scattering model in direct volume rendering. A common problem of single scattering is the high dynamic range between lit and shadowed regions due to the exponential attenuation of light along a ray. Moreover, light is often attenuated too strong between a sample point and the camera, hampering the visibility of important features. Our algorithm employs an importance function to selectively illuminate important structures and make them visible from the camera. With the importance function, more light can be transmitted to the features of interest, while contextual structures cast shadows which provide visual cues for perception of depth. At the same time, more scattered light is transmitted from the sample point to the camera to improve the primary visibility of important features. We formulate a minimization problem that automatically determines the extinction along a view or shadow ray to obtain a good balance between sufficient transmittance and attenuation. In contrast to previous approaches, we do not require a computationally expensive solution of a global optimization, but instead provide a closed-form solution for each sampled extinction value along a view or shadow ray and thus achieve interactive performance.

Despite recent advances in Monte Carlo rendering techniques, dense, high-albedo participating media such as wax or skin still remain a difficult problem. In such media, random walks tend to become very long, but may still lead to a large contribution to the image. The Dwivedi sampling scheme, which is based on zero variance random walks, biases the sampling probability distributions to exit the medium as quickly as possible. This can reduce variance considerably under the assumption of a locally homogeneous medium with constant phase function. Prior work uses the normal at the Point of Entry as the bias direction. We demonstrate that this technique can fail in common scenarios such as thin geometry with a strong backlight. We propose two new biasing strategies, Closest Point and Incident Illumination biasing, and show that these techniques can speed up convergence by up to an order of magnitude. Additionally, we propose a heuristic approach for combining biased and classical sampling techniques using Multiple Importance Sampling.

Abstract Mimicking the appearance of the real world is a longstanding goal of computer graphics, with several important applications in the feature film, architecture and medical industries. Images with well‐designed shading are an important tool for conveying information about the world, be it the shape and function of a computer‐aided design (CAD) model, or the mood of a movie sequence. However, authoring this content is often a tedious task, even if undertaken by groups of highly trained and experienced artists. Unsurprisingly, numerous methods to facilitate and accelerate this appearance editing task have been proposed, enabling the editing of scene objects' appearances, lighting and materials, as well as entailing the introduction of new interaction paradigms and specialized preview rendering techniques. In this review, we provide a comprehensive survey of artistic appearance, lighting and material editing approaches. We organize this complex and active research area in a structure tailored to academic researchers, graduate students and industry professionals alike. In addition to editing approaches, we discuss how user interaction paradigms and rendering back ends combine to form usable systems for appearance editing. We conclude with a discussion of open problems and challenges to motivate and guide future research.

Rendering with accurate camera models greatly increases realism and improves the match of synthetic imagery to real-life footage. Photographic lenses can be simulated by ray tracing, but the performance depends on the complexity of the lens system, and some operations required for modern algorithms, such as deterministic connections, can be difficult to achieve. We generalise the approach of polynomial optics, i.e. expressing the light field transformation from the sensor to the outer pupil using a polynomial, to work with extreme wide angle (fisheye) lenses and aspherical elements. We also show how sparse polynomials can be constructed from the large space of high-degree terms (we tested up to degree 15). We achieve this using a variant of orthogonal matching pursuit instead of a Taylor series when computing the polynomials. We show two applications: photorealistic rendering using Monte Carlo methods, where we introduce a new aperture sampling technique that is suitable for light tracing, and an interactive preview method suitable for rendering with deep images.

Modeling multiple scattering in microfacet theory is considered an important open problem because a non-negligible portion of the energy leaving rough surfaces is due to paths that bounce multiple times. In this paper we derive the missing multiple-scattering components of the popular family of BSDFs based on the Smith microsurface model. Our derivations are based solely on the original assumptions of the Smith model. We validate our BSDFs using raytracing simulations of explicit random Beckmann surfaces. Our main insight is that the microfacet theory for surfaces with the Smith model can be derived as a special case of the microflake theory for volumes, with additional constraints to enforce the presence of a sharp interface, i.e. to transform the volume into a surface. We derive new free-path distributions and phase functions such that plane-parallel scattering from a microvolume with these distributions exactly produces the BSDF based on the Smith microsurface model, but with the addition of higher-order scattering. With this new formulation, we derive multiple-scattering micro-facet BSDFs made of either diffuse, conductive, or dielectric material. Our resulting BSDFs are reciprocal, energy conserving, and support popular anisotropic parametric normal distribution functions such as Beckmann and GGX. While we do not provide closed-form expressions for the BSDFs, they are mathematically well-defined and can be evaluated at arbitrary precision. We show how to practically use them with Monte Carlo physically based rendering algorithms by providing analytic importance sampling and unbiased stochastic evaluation. Our implementation is analytic and does not use per-BSDF precomputed data, which makes our BSDFs usable with textured albedos, roughness, and anisotropy.

We present a novel method to compute anisotropic shading for direct volume rendering to improve the perception of the orientation and shape of surface-like structures. We determine the scale-aware anisotropy of a shading point by analyzing its ambient region. We sample adjacent points with similar scalar values to perform a principal component analysis by computing the eigenvectors and eigenvalues of the covariance matrix. In particular, we estimate the tangent directions, which serve as the tangent frame for anisotropic bidirectional reflectance distribution functions. Moreover, we exploit the ratio of the eigenvalues to measure the magnitude of the anisotropy at each shading point. Altogether, this allows us to model a data-driven, smooth transition from isotropic to strongly anisotropic volume shading. In this way, the shape of volumetric features can be enhanced significantly by aligning specular highlights along the principal direction of anisotropy. Our algorithm is independent of the transfer function, which allows us to compute all shading parameters once and store them with the data set. We integrated our method in a GPU-based volume renderer, which offers interactive control of the transfer function, light source positions, and viewpoint. Our results demonstrate the benefit of anisotropic shading for visualization to achieve data-driven local illumination for improved perception compared to isotropic shading.

Abstract In this paper, we present improvements to half vector space light transport (HSLT) [ KHD14 ], which make this approach more practical, robust for difficult input geometry, and faster. Our first contribution is the computation of half vector space ray differentials in a different domain than the original work. This enables a more uniform stratification over the image plane during Markov chain exploration. Furthermore, we introduce a new multi chain perturbation in half vector space, which, if combined appropriately with half vector perturbation, makes the mutation strategy both more robust to geometric configurations with fine displacements and faster due to reduced number of ray casts. We provide and analyze the results of improved HSLT and discuss possible applications of our new half vector ray differentials.

Abstract In this paper, we extend the concept of pre‐filtered shadow mapping to stochastic rasterization, enabling real‐time rendering of soft shadows from planar area lights. Most existing soft shadow mapping methods lose important visibility information by relying on pinhole renderings from an area light source, providing plausible results only for small light sources. Since we sample the entire 4D shadow light field stochastically, we are able to closely approximate shadows of large area lights as well. In order to efficiently reconstruct smooth shadows from this sparse data, we exploit the analogy of soft shadow computation to rendering defocus blur, and introduce a multiplane pre‐filtering algorithm. We demonstrate how existing pre‐filterable approximations of the visibility function, such as variance shadow mapping, can be extended to four dimensions within our framework.

Abstract In photorealistic image synthesis the radiative transfer equation is often not solved by simulating every wavelength of light, but instead by computing tristimulus transport, for instance using sRGB primaries as a basis. This choice is convenient, because input texture data is usually stored in RGB colour spaces. However, there are problems with this approach which are often overlooked or ignored. By comparing to spectral reference renderings, we show how rendering in tristimulus colour spaces introduces colour shifts in indirect light, violation of energy conservation, and unexpected behaviour in participating media. Furthermore, we introduce a fast method to compute spectra from almost any given XYZ input colour. It creates spectra that match the input colour precisely. Additionally, like in natural reflectance spectra, their energy is smoothly distributed over wide wavelength bands. This method is both useful to upsample RGB input data when spectral transport is used and as an intermediate step for corrected tristimulus‐based transport. Finally, we show how energy conservation can be enforced in RGB by mapping colours to valid reflectances.

Abstract Volumetric scalar data sets are common in many scientific, engineering and medical applications where they originate from measurements or simulations. Furthermore, they can represent geometric scene content, e.g. as distance or density fields. Often isosurfaces are extracted, either for indirect volume visualization in the former category, or to simply obtain a polygonal representation in case of the latter. However, even moderately sized volume data sets can result in complex isosurfaces which are challenging to recompute in real time, e.g. when the user modifies the isovalue or when the data itself are dynamic. In this paper, we present a GPU‐friendly algorithm for the extraction of isosurfaces, which provides adaptive level of detail rendering with view‐dependent tessellation. It is based on a longest edge bisection scheme where the resulting tetrahedral cells are subdivided into four hexahedra, which then form the domain for the subsequent isosurface extraction step. Our algorithm generates meshes with good triangle quality even for highly non‐linear scalar data. In contrast to previous methods, it does not require any stitching between regions of different levels of detail. As all computation is performed at run time and no pre‐processing is required, the algorithm naturally supports dynamic data and allows us to change isovalues at any time.

Abstract In this paper, we introduce a novel technique for pre‐filtering multi‐layer shadow maps. The occluders in the scene are stored as variable‐length lists of fragments for each texel. We show how this representation can be filtered by progressively merging these lists. In contrast to previous pre‐filtering techniques, our method better captures the distribution of depth values, resulting in a much higher shadow quality for overlapping occluders and occluders with different depths. The pre‐filtered maps are generated and evaluated directly on the GPU, and provide efficient queries for shadow tests with arbitrary filter sizes. Accurate soft shadows are rendered in real‐time even for complex scenes and difficult setups. Our results demonstrate that our pre‐filtered maps are general and particularly scalable.

Abstract Many‐light methods approximate the light transport in a scene by computing the direct illumination from many virtual point light sources (VPLs), and render low‐noise images covering a wide range of performance and quality goals. However, they are very inefficient at representing glossy light transport. This is because a VPL on a glossy surface illuminates a small fraction of the scene only, and a tremendous number of VPLs might be necessary to render acceptable images. In this paper, we introduceRich‐VPLswhich, in contrast to standard VPLs, represent a multitude of light paths and thus have a more widespread emission profile on glossy surfaces and in scenes with multiple primary light sources. By this, a single Rich‐VPL contributes to larger portions of a scene with negligible additional shading cost. Our second contribution is a placement strategy for (Rich‐)VPLs proportional to sensor importance times radiance. Although both Rich‐VPLs and improved placement can be used individually, they complement each other ideally and share interim computation. Furthermore, both complement existing many‐light methods, e.g. Lightcuts or the Virtual Spherical Lights method, and can improve their efficiency as well as their application for scenes with glossy materials and many primary light sources.

Abstract Inspired by vector field topology, an established tool for the extraction and identification of important features of flows and vector fields, we develop means for the analysis of the structure of light transport. For that, we derive an analogy to vector field topology that defines coherent structures in light transport. We also introduceFinite‐Time Path Deflection(FTPD), a scalar quantity that represents the deflection characteristic of all light transport paths passing through a given point in space. For virtual scenes, the FTPD can be computed directly using path‐space Monte Carlo integration. We visualize the FTPD field for several example scenes and discuss the revealed structures. Lastly, we show that the coherent regions visualized by the FTPD are closely related to the coherent regions in our new topologically‐motivated analysis of light transport. FTPD visualizations are thus also visualizations of the structure of light transport.

We introduce the Symmetric GGX (SGGX) distribution to represent spatially-varying properties of anisotropic microflake participating media. Our key theoretical insight is to represent a microflake distribution by the projected area of the microflakes. We use the projected area to parameterize the shape of an ellipsoid, from which we recover a distribution of normals. The representation based on the projected area allows for robust linear interpolation and prefiltering, and thanks to its geometric interpretation, we derive closed form expressions for all operations used in the microflake framework. We also incorporate microflakes with diffuse reflectance in our theoretical framework. This allows us to model the appearance of rough diffuse materials in addition to rough specular materials. Finally, we use the idea of sampling the distribution of visible normals to design a perfect importance sampling technique for our SGGX microflake phase functions. It is analytic, deterministic, simple to implement, and one order of magnitude faster than previous work.

We address the problem of modeling and rendering granular materials---such as large structures made of sand, snow, or sugar---where an aggregate object is composed of many randomly oriented, but discernible grains. These materials pose a particular challenge as the complex scattering properties of individual grains, and their packing arrangement, can have a dramatic effect on the large-scale appearance of the aggregate object. We propose a multi-scale modeling and rendering framework that adapts to the structure of scattered light at different scales. We rely on path tracing the individual grains only at the finest scale, and---by decoupling individual grains from their arrangement---we develop a modular approach for simulating longer-scale light transport. We model light interactions within and across grains as separate processes and leverage this decomposition to derive parameters for classical radiative transport, including standard volumetric path tracing and a diffusion method that can quickly summarize the large scale transport due to many grain interactions. We require only a one-time precomputation per exemplar grain, which we can then reuse for arbitrary aggregate shapes and a continuum of different packing rates and scales of grains. We demonstrate our method on scenes containing mixtures of tens of millions of individual, complex, specular grains that would be otherwise infeasible to render with standard techniques.

Abstract Recent years have seen increasing attention and significant progress in many‐light rendering, a class of methods for efficient computation of global illumination. The many‐light formulation offers a unified mathematical framework for the problem reducing the full lighting transport simulation to the calculation of the direct illumination from many virtual light sources. These methods are unrivaled in their scalability: they are able to produce plausible images in a fraction of a second but also converge to the full solution over time. In this state‐of‐the‐art report, we give an easy‐to‐follow, introductory tutorial of the many‐light theory; provide a comprehensive, unified survey of the topic with a comparison of the main algorithms; discuss limitations regarding materials and light transport phenomena and present a vision to motivate and guide future research. We will cover both the fundamental concepts as well as improvements, extensions and applications of many‐light rendering.

Abstract Subpixel rendering increases the apparent display resolution by taking into account the subpixel structure of a given display. In essence, each subpixel is addressed individually, allowing the underlying signal to be sampled more densely. Unfortunately, naïve subpixel sampling introduces colour aliasing, as each subpixel only displays a specific colour (usually R, G and B subpixels are used). As previous work has shown, chromatic aliasing can be reduced significantly by taking the sensitivity of the human visual system into account. In this work, we find optimal filters for subpixel rendering for a diverse set of 1D and 2D subpixel layout patterns. We demonstrate that these optimal filters can be approximated well with analytical functions. We incorporate our filters into GPU‐based multi‐sample anti‐aliasing to yield subpixel rendering at a very low cost (1–2 ms filtering time at HD resolution). We also show that texture filtering can be adapted to perform efficient subpixel rendering. Finally, we analyse the findings of a user study we performed, which underpins the increased visual fidelity that can be achieved for diverse display layouts, by using our optimal filters.

Abstract In this paper we present a novel approach to simulate image formation for a wide range of real world lenses in the Monte Carlo ray tracing framework. Our approach sidesteps the overhead of tracing rays through a system of lenses and requires no tabulation. To this end we first improve the precision of polynomial optics to closely match ground‐truth ray tracing. Second, we show how the Jacobian of the optical system enables efficient importance sampling, which is crucial for difficult paths such as sampling the aperture which is hidden behind lenses on both sides. Our results show that this yields converged images significantly faster than previous methods and accurately renders complex lens systems with negligible overhead compared to simple models, e.g. the thin lens model. We demonstrate the practicality of our method by incorporating it into a bidirectional path tracing framework and show how it can provide information needed for sophisticated light transport algorithms.

Abstract In this paper we present a fractional parametric splitting scheme for Reyes‐style adaptive tessellation. Our parallel algorithm generates crack‐free tessellation from a parametric surface, which is also free of sudden temporal changes under animation. Continuous level of detail is not addressed by existing Reyes‐style methods, since these aim to produce subpixel‐sized micropolygons, where topology changes are no longer noticeable. Using our method, rendering pipelines that use larger triangles, thus sensitive to geometric popping, may also benefit from the quality of the split‐dice tessellation stages of Reyes. We demonstrate results on a real‐time GPU implementation, going beyond the limited quality and resolution of the hardware tessellation unit. In contrast to previous split‐dice methods, our split stage is compatible with the fractional hardware tessellation scheme that has been designed for continuous level of detail.

Abstract In this work we detail a method that leverages the two color heads of recent low‐end fused deposition modeling (FDM) 3D printers to produce continuous tone imagery. The challenge behind producing such two‐tone imagery is how to finely interleave the two colors while minimizing the switching between print heads, making each color printed span as long and continuous as possible to avoid artifacts associated with printing short segments. The key insight behind our work is that by applying small geometric offsets, tone can be varied without the need to switch color print heads within a single layer. We can now effectively print (two‐tone) texture mapped models capturing both geometric and color information in our output 3D prints.

Global illumination algorithms using Markov chain Monte Carlo (MCMC) sampling are well-known for their efficiency in scenes with complex light transport. Samples in such algorithms are generated as a history of Markov chain states so that they are distributed according to the contributions to the image. The whole process is done based only on the information of the path contributions and user-defined transition probabilities from one state to the others. In light transport simulation, however, there is more information that can be used to improve the efficiency of path sampling. A notable example is multiple importance sampling (MIS) in bidirectional path tracing, which utilizes the probability densities of constructing a given path with different estimators. While MIS is a powerful ordinary Monte Carlo method, how to incorporate such additional information into MCMC sampling has been an open problem. We introduce a novel MCMC sampling framework, primary space serial tempering , which fuses the ideas of MCMC sampling and MIS for the first time. The key idea is to explore not only the sample space using a Markov chain, but also different estimators to generate samples by utilizing the information already available for MIS. Based on this framework, we also develop a novel rendering algorithm, multiplexed Metropolis light transport , which automatically and adaptively constructs paths with appropriate techniques as predicted by MIS. The final algorithm is very easy to implement, yet in many cases shows comparable (or even better) performance than significantly more complex MCMC rendering algorithms.

The path integral formulation of light transport is the basis for (Markov chain) Monte Carlo global illumination methods. In this paper we present half vector space light transport (HSLT) , a novel approach to sampling and integrating light transport paths on surfaces. The key is a partitioning of the path space into subspaces in which a path is represented by its start and end point constraints and a sequence of generalized half vectors. We show that this representation has several benefits. It enables importance sampling of all interactions along paths in between two endpoints. Based on this, we propose a new mutation strategy, to be used with Markov chain Monte Carlo methods such as Metropolis light transport (MLT), which is well-suited for all types of surface transport paths (diffuse/glossy/specular interaction). One important characteristic of our approach is that the Fourier-domain properties of the path integral can be easily estimated. These can be used to achieve optimal correlation of the samples due to well-chosen mutation step sizes, leading to more efficient exploration of light transport features. We also propose a novel approach to control stratification in MLT with our mutation strategy.

We present a novel and efficient method to compute volumetric soft shadows for interactive direct volume visualization to improve the perception of spatial depth. By direct control of the softness of volumetric shadows, disturbing visual patterns due to hard shadows can be avoided and users can adapt the illumination to their personal and application-specific requirements. We compute the shadowing of a point in the data set by employing spatial filtering of the optical depth over a finite area patch pointing toward each light source. Conceptually, the area patch spans a volumetric region that is sampled with shadow rays; afterward, the resulting optical depth values are convolved with a low-pass filter on the patch. In the numerical computation, however, to avoid expensive shadow ray marching, we show how to align and set up summed area tables for both directional and point light sources. Once computed, the summed area tables enable efficient evaluation of soft shadows for each point in constant time without shadow ray marching and the softness of the shadows can be controlled interactively. We integrated our method in a GPU-based volume renderer with ray casting from the camera, which offers interactive control of the transfer function, light source positions, and viewpoint, for both static and time-dependent data sets. Our results demonstrate the benefit of soft shadows for visualization to achieve user-controlled illumination with many-point lighting setups for improved perception combined with high rendering speed.

Abstract We propose a simple yet powerful regularization framework for robust light transport simulation. It builds on top of existing unbiased methods and resorts to a consistent estimation using regularization only for paths which cannot be sampled in an unbiased way. To introduce as little bias as possible, we selectively regularize individual interactions along paths, and also derive the regularization consistency conditions. Our approach is compatible with the majority of unbiased methods, e.g. (bidirectional) path tracing and Metropolis light transport (MLT), and only a simple modification is required to adapt existing renderers. We compare to recent unbiased and consistent methods and show examples of scenes with difficult light paths, where regularization is required to account for all illumination features. When coupled with MLT we are able to sample all phenomena, like recent consistent methods, while achieving superior convergence.

Abstract Ray tracing has become a viable alternative to rasterization for interactive applications and also forms the basis of most global illumination methods. However, even today's fastest ray‐tracers offer only a tight budget of rays per pixel per frame. Rendering performance can be improved by increasing this budget, or by developing methods that use it more efficiently. In this paper we propose a global visibility caching algorithm that reduces the number of shadow rays required for shading to a fraction of less than 2% in some cases. We quantize the visibility function's domain while ensuring a minimal degradation of the final image quality. To control the introduced error, we adapt the quantization locally, accounting for variations in geometry, sampling densities on both endpoints of the visibility queries, and the light signal itself. Compared to previous approaches for approximating visibility, e.g. shadow mapping, our method has several advantages: (1) it allows caching of arbitrary visibility queries between surface points and is thus applicable to all ray tracing based methods; (2) the approximation error is uniform over the entire image and can be bounded by a user‐specified parameter; (3) the cache is created on‐the‐fly and does not waste any resources on queries that will never be used. We demonstrate the benefits of our method on Whitted‐style ray tracing combined with instant radiosity, as well as an integration with bidirectional path tracing.

This article introduces a novellocally adaptiveprogressive photon mapping technique which optimally balances noise and bias in rendered images to minimize the overall error. It is the result of an analysis of the radiance estimation in progressive photon mapping. As a first step, we establish a connection to the field of recursive estimation and regression in statistics and derive the optimal estimation parameters for the asymptotic convergence of existing approaches. Next, we show how to reformulate photon mapping as a spatial regression in the measurement equation of light transport. This reformulation allows us to derive a novel data-driven bandwidth selection technique for estimating a pixel's measurement. The proposed technique possesses attractive convergence properties with finite numbers of samples, which is important for progressive rendering, and it also provides better results for quasi-converged images. Our results show the practical benefits of using our adaptive method.

Industry-quality content creation relies on tools for lighting artists to quickly prototype, iterate, and refine final renders. As industry-leading studios quickly adopt physically-based rendering (PBR) across their art generation pipelines, many existing tools have become unsuitable as they address only simple effects without considering underlying PBR concepts and constraints. We present a novel light transport manipulation technique that operates directly on path-space solutions of the rendering equation. We expose intuitive direct and indirect manipulation approaches to edit complex effects such as (multi-refracted) caustics, diffuse and glossy indirect bounces, and direct/indirect shadows. With our sketch- and object-space selection, all built atop a parameterized regular expression engine, artists can search and isolate shading effects to inspect and edit. We classify and filter paths on the fly and visualize the selected transport phenomena. We survey artists who used our tool to manipulate complex phenomena on both static and animated scenes.

In this paper we present decoupled deferred shading: a rendering technique based on a new data structure called compact geometry buffer, which stores shading samples independently from the visibility. This enables caching and efficient reuse of shading computation, e.g. for stochastic rasterization techniques. In contrast to previous methods, our decoupled shading can be efficiently implemented on current graphics hardware. We describe two variants which differ in the way the shading samples are cached: the first maintains a single cache for the entire image in global memory, while the second pursues a tile-based approach leveraging local memory of the GPU's multiprocessors. We demonstrate the application of decoupled deferred shading to speed up the rendering in applications with stochastic supersampling, depth of field, and motion blur.

Abstract In this paper we present a novel method for high‐quality rendering of scenes with participating media. Our technique is based on instant radiosity, which is used to approximate indirect illumination between surfaces by gathering light from a set of virtual point lights (VPLs). It has been shown that this principle can be applied to participating media as well, so that the combined single scattering contribution of VPLs within the medium yields full multiple scattering. As in the surface case, VPL methods for participating media are prone to singularities, which appear as bright “splotches” in the image. These artifacts are usually countered by clamping the VPLs' contribution, but this leads to energy loss within the short‐distance light transport. Bias compensation recovers the missing energy, but previous approaches are prohibitively costly. We investigate VPL‐based methods for rendering scenes with participating media, and propose a novel and efficient approximate bias compensation technique. We evaluate our technique using various test scenes, showing it to be visually indistinguishable from ground truth.

Abstract We present the rasterized bounding volume hierarchy (RBVH), a compact data structure that accelerates approximate ray casting of complex meshes and provides adjustable level of detail. During construction, we identify subtrees of BVHs containing surfaces that can be represented by height fields. For these subtrees the conventional ray‐surface intersection, which possibly involves a large number of triangles, is replaced by a simple ray marching procedure to find the intersection with the surface. We describe GPU algorithms for construction, ray casting, and data querying of the RBVH that achieve comparable or higher performance than state of the art acceleration structures for triangle meshes. Moreover, RBVHs provide an inherent surface parameterization for storing data on the surfaces and natively handle triangle and point‐based surface representations. We also show that RBVHs support adaptive level‐of‐detail and can be combined with traditional BVHs to handle complex scenes.

Abstract A recent technique that formsvirtual ray lights(VRLs) from path segments in media, reduces the artifacts common to VPL approaches in participating media, however, distracting singularities still remain. We present Virtual Beam Lights (VBLs), a progressive many‐lights algorithm for rendering complex indirect transport paths in, from, and to media. VBLs are efficient and can handle heterogeneous media, anisotropic scattering, and moderately glossy surfaces, while provably converging to ground truth. We inflate ray lights intobeam lightswith finite thicknesses to eliminate the remaining singularities. Furthermore, we devise several practical schemes for importance sampling the various transport contributions between camera rays, light rays, and surface points. VBLs produce artifact‐free images faster than VRLs, especially when glossy surfaces and/or anisotropic phase functions are present. Lastly, we employ a progressive thickness reduction scheme for VBLs in order to render results that converge to ground truth.

Abstract This paper presents novel interactive visualization techniques for inspecting the global light transport in virtual scenes. First, we propose a simple extension to photon mapping to gather required lighting information. We then introduce a set of five light inspection tools which process this data to provide further insights. Corresponding visualizations help the user to comprehend how light travels within a scene, how the lighting affects the appearance of a surface, and how objects cause lighting effects such as caustics. We implemented all tools for direct usage in real production environments. Rendering is based on progressive photon mapping, providing interactivity and immediate visual feedback. We conducted a user study to evaluate all techniques in various application scenarios and hence discuss their individual strengths and weaknesses. Moreover, we present feedback from domain experts.

Abstract The interaction of light and matter in the world surrounding us is of striking complexity and beauty. Since the very beginning of computer graphics, adequate modelling of these processes and efficient computation is an intensively studied research topic and still not a solved problem. The inherent complexity stems from the underlying physical processes as well as the global nature of the interactions that let light travel within a scene. This paper reviews the state of the art in interactive global illumination (GI) computation, i.e., methods that generate an image of a virtual scene in less than 1 s with an as exact as possible, or plausible, solution to the light transport. Additionally, the theoretical background and attempts to classify the broad field of methods are described. The strengths and weaknesses of different approaches, when applied to the different visual phenomena, arising from light interaction are compared and discussed. Finally, the paper concludes by highlighting design patterns for interactive GI and a list of open problems.

We present an efficient many-light algorithm for simulating indirect illumination in, and from, participating media. Instead of creating discrete virtual point lights (VPLs) at vertices of random-walk paths, we present a continuous generalization that places virtual ray lights (VRLs) along each path segment in the medium. Furthermore, instead of evaluating the lighting independently at discrete points in the medium, we calculate the contribution of each VRL to entire camera rays through the medium using an efficient Monte Carlo product sampling technique. We prove that by spreading the energy of virtual lights along both light and camera rays, the singularities that typically plague VPL methods are significantly diminished. This greatly reduces the need to clamp energy contributions in the medium, leading to robust and unbiased volumetric lighting not possible with current many-light techniques. Furthermore, by acting as a form of final gather, we obtain higher-quality multiple-scattering than existing density estimation techniques like progressive photon beams.

Molecular dynamics is a widely used simulation technique to investigate material properties and structural changes under external forces. The availability of more powerful clusters and algorithms continues to increase the spatial and temporal extents of the simulation domain. This poses a particular challenge for the visualization of the underlying processes which might consist of millions of particles and thousands of time steps. Some application domains have developed special visual metaphors to only represent the relevant information of such data sets but these approaches typically require detailed domain knowledge that might not always be available or applicable. We propose a general technique that replaces the huge amount of simulated particles by a smaller set of representatives that are used for the visualization instead. The representatives capture the characteristics of the underlying particle density and exhibit coherency over time. We introduce loose capacity-constrained Voronoi diagrams for the generation of these representatives by means of a GPU-friendly, parallel algorithm. This way we achieve visualizations that reflect the particle distribution and geometric structure of the original data very faithfully. We evaluate our approach using real-world data sets from the application domains of material science, thermodynamics and dynamical systems theory.

Caustics are detailed patterns of light reflected or refracted on specular surfaces into participating media or onto surfaces. In this paper we present a novel adaptive and scalable algorithm for rendering surface and volume caustics in single-scattering participating media at real-time frame rates. Motivated by both caustic mapping and triangle-based volumetric methods, our technique captures the specular surfaces in light-space, but traces beams of light instead of single photons. The beams are adaptively generated from a grid projected from the light source onto the scene's surfaces, which is iteratively refined according to discontinuities in the geometry and photon distribution. This allows us to reconstruct sharp volume caustic patterns while reducing sampling resolution and fill-rate at defocused regions. We demonstrate our technique combined with approximate ray tracing techniques to render surfaces with two-sided refractions as well as multiple caustic light bounces.

In this paper we present a method that targets high-quality global illumination at interactive frame rates. As many techniques in this context, our method is based on instant radiosity, which represents the indirect illumination in a scene with a set of virtual point lights, and therefore enables efficient GPU rendering. Instant radiosity captures light transport over larger distances well, but it requires clamping of the point lights' contribution to avoid bright splotches on nearby surfaces. By bounding the short distance light transport, the algorithm removes some energy and thus introduces bias, that is visible as incorrect darkening near edges and corners. Our method improves the quality and correctness of the rendered images by removing bias using a hierarchical screen space approach. We show that the bias compensation can be formulated as a post-processing step and demonstrate renderings comparable to results from offline algorithms at interactive speed.

Abstract We present an optimized pruning algorithm that allows for considerable geometry reduction in large botanical scenes while maintaining high and coherent rendering quality. We improve upon previous techniques by applying model‐specific geometry reduction functions and optimized scaling functions. For this we introduce the use of Precision and Recall (PR) as a measure of quality to rendering and show how PR‐scores can be used to predict better scaling values. We conducted a user‐study letting subjects adjust the scaling value, which shows that the predicted scaling matches the preferred ones. Finally, we extend the originally purely stochastic geometry prioritization for pruning to account for view‐optimized geometry selection, which allows to take global scene information, such as occlusion, into consideration. We demonstrate our method for the rendering of scenes with thousands of complex tree models in real‐time.

Many algorithms, such as level of detail rendering and occlusion culling methods, make decisions based on the degree of visibility of an object, but do not analyze the distribution, or structure, of the visible and occluded regions across surfaces. We present an efficient method to classify different visibility configurations and show how this can be used on top of existing methods based on visibility determination. We adapt co-occurrence matrices for visibility analysis and generalize them to operate on clusters of triangular surfaces instead of pixels. We employ machine learning techniques to reliably classify the thus extracted feature vectors. Our method allows perceptually motivated level of detail methods for real-time rendering applications by detecting configurations with expected visual masking. We exemplify the versatility of our method with an analysis of area light visibility configurations in ray tracing and an area-to-area visibility analysis suitable for hierarchical radiosity refinement. Initial results demonstrate the robustness, simplicity, and performance of our method in synthetic scenes, as well as real applications.

Scattering in participating media, such as fog or haze, generates volumetric lighting effects known as crepuscular or god rays. Rendering such effects greatly enhances the realism in virtual scenes, but is inherently costly as scattering events occur at every point in space and thus it requires costly integration of the light scattered towards the observer. This is typically done using ray marching which is too expensive for every pixel on the screen for interactive applications. We propose a rendering technique for textured light sources in single-scattering media, that draws from the concept of epipolar geometry to place samples in image space: the inscattered light varies orthogonally to crepuscular rays, but mostly smoothly along these rays. These are epipolar lines of a plane of light rays that projects onto one line on the image plane. Our method samples sparsely along epipolar lines and interpolates between samples where adequate, but preserves high frequency details that are caused by shadowing of light rays. We show that our method is very simple to implement on the GPU, yields high quality images, and achieves high frame rates.

This paper introduces a new scalable technique for approximating indirect illumination in fully dynamic scenes for real-time applications, such as video games. We use lattices and spherical harmonics to represent the spatial and angular distribution of light in the scene. Our technique does not require any precomputation and handles large scenes with nested lattices. It is primarily targeted at rendering single-bounce indirect illumination with occlusion, but can be extended to handle multiple bounces and participating media. We demonstrate that our method produces plausible results even when running on current game console hardware with a budget of only a few milliseconds for performing all computation steps for indirect lighting. We evaluate our technique and show it in combination with a variety of popular real-time rendering techniques.

Abstract Molecular dynamics simulations are a principal tool for studying molecular systems. Such simulations are used to investigate molecular structure, dynamics, and thermodynamical properties, as well as a replacement for, or complement to, costly and dangerous experiments. With the increasing availability of computational power the resulting data sets are becoming increasingly larger, and benchmarks indicate that the interactive visualization on desktop computers poses a challenge when rendering substantially more than millions of glyphs. Trading visual quality for rendering performance is a common approach when interactivity has to be guaranteed. In this paper we address both problems and present a method for high‐quality visualization of massive molecular dynamics data sets. We employ several optimization strategies on different levels of granularity, such as data quantization, data caching in video memory, and a two‐level occlusion culling strategy: coarse culling via hardware occlusion queries and a vertex‐level culling using maximum depth mipmaps. To ensure optimal image quality we employ GPU raycasting and deferred shading with smooth normal vector generation. We demonstrate that our method allows us to interactively render data sets containing tens of millions of high‐quality glyphs.

We present an interactive system for the artistic control of visual phenomena visible on surfaces. Our method allows the user to intuitively reposition shadows, caustics, and indirect illumination using a simple click-and-drag user interface working directly on surfaces. In contrast to previous approaches, the positions of the lights or objects in the scene remain unchanged, enabling localized edits of individual shading components. Our method facilitates the editing by computing a mapping from one surface location to another. Based on this mapping, we can not only edit shadows, caustics, and indirect illumination but also other surface properties, such as color or texture, in a unified way. This is achieved using an intuitive user-interface that allows the user to specify position constraints with drag-and-drop or sketching operations directly on the surface. Our approach requires no explicit surface parametrization and handles scenes with arbitrary topology. We demonstrate the applicability of the approach to interactive editing of shadows, reflections, refractions, textures, caustics, and diffuse indirect light. The effectiveness of the system to achieve an artistic goal is evaluated by a user study.

Efficient visibility queries are key in many interactive rendering techniques, such as occlusion culling, level of detail determination, and perceptual rendering. The occlusion query mechanism natively supported by GPUs is carried out for batches of rendered geometry. In this paper, we present two novel ways of determining visibility by intelligently querying summed area tables and computing a variant of item buffers. This enables visibility queries of finer granularity, e.g., for sub-regions of objects and for instances created within a single draw call. Our method determines the visibility of a large number of objects simultaneously which can be used in geometry shaders to cull triangles, or to control the level of detail in geometry and pixel shaders under certain rendering scenarios. We demonstrate the benefits of our method with two different real-time rendering techniques.

Abstract Modeling large architectural environments is a difficult task due to the intricate nature of these models and the complex dependencies between the structures represented. Moreover, textures are an essential part of architectural models. While the number of geometric primitives is usually relatively low (i.e., many walls are at surfaces), textures actually contain many detailed architectural elements. We present an approach for modeling architectural scenes by reshaping and combining existing textured models, where the manipulation of the geometry and texture are tightly coupled. For geometry, preserving angles such as oor orientation or vertical walls is of key importance. We thus allow the user to interactively modify lengths of edges, while constraining angles. Our texture reshaping solution introduces a measure of directional autosimilarity to focus stretching in areas of stochastic content and to preserve details in such areas. We show results on several challenging models, and show two applications: Building complex road structures from simple initial pieces and creating complex game‐levels from an existing game based on pre‐existing model pieces.

Abstract We present a methodical new approach to visualize the aspects of general relativity from a self‐centered perspective. We focus on the visualization of the Gödel universe, which is an exact solution to Einstein's field equations of general relativity. This model provides astounding features such as the existence of an optical horizon and the possibility of time travel. Although we know that our universe is not of Gödel type, we can – using this solution to Einstein's equations – visualize and understand the effects resulting from the theory of relativity, which itself has been verified on the large scale in numerous experiments over the last century. We derive the analytical solution to the geodesic equations of Gödel's universe for special initial conditions. Along with programmable graphics hardware we achieve a tremendous speedup for the visualization of general relativity. This enables us to interactively explore the physical aspects and optical effects of Gödel's universe. We also demonstrate how the analytical solution enables dynamic lighting with local illumination models. Our implementation is tailored for Gödel's universe and five orders of magnitude faster than previous approaches. It can be adapted to manifolds for which an analytical expression of the propagation of light is available.

In this article we evaluate the use of approximate visibility for efficient global illumination. Traditionally, accurate visibility is used in light transport. However, the indirect illumination we perceive on a daily basis is rarely of high-frequency nature, as the most significant aspect of light transport in real-world scenes is diffuse, and thus displays a smooth gradation. This raises the question of whether accurate visibility is perceptually necessary in this case. To answer this question, we conduct a psychophysical study on the perceptual influence of approximate visibility on indirect illumination. This study reveals that accurate visibility is not required and that certain approximations may be introduced.

Recent approaches to global illumination for dynamic scenes achieve interactive frame rates by using coarse approximations to geometry, lighting, or both, which limits scene complexity and rendering quality. High-quality global illumination renderings of complex scenes are still limited to methods based on ray tracing. While conceptually simple, these techniques are computationally expensive. We present an efficient and scalable method to compute global illumination solutions at interactive rates for complex and dynamic scenes. Our method is based on parallel final gathering running entirely on the GPU. At each final gathering location we perform micro-rendering: we traverse and rasterize a hierarchical point-based scene representation into an importance-warped micro-buffer , which allows for BRDF importance sampling. The final reflected radiance is computed at each gathering location using the micro-buffers and is then stored in image-space. We can trade quality for speed by reducing the sampling rate of the gathering locations in conjunction with bilateral upsampling. We demonstrate the applicability of our method to interactive global illumination, the simulation of multiple indirect bounces, and to final gathering from photon maps.

We present a development environment for distributed GPU computing targeted for multi-GPU systems, as well as graphics clusters. Our system is based on CUDA and logically extends its parallel programming model for graphics processors to higher levels of parallelism, namely, the PCI bus and network interconnects. While the extended API mimics the full function set of current graphics hardware-including the concept of global memory-on all distribution layers, the underlying communication mechanisms are handled transparently for the application developer. To allow for high scalability, in particular for network-interconnected environments, we introduce an automatic GPU-accelerated scheduling mechanism that is aware of data locality. This way, the overall amount of transmitted data can be heavily reduced, which leads to better GPU utilization and faster execution. We evaluate the performance and scalability of our system for bus and especially network-level parallelism on typical multi-GPU systems and graphics clusters.

We present a method for interactive computation of indirect illumination in large and fully dynamic scenes based on approximate visibility queries. While the high-frequency nature of direct lighting requires accurate visibility, indirect illumination mostly consists of smooth gradations, which tend to mask errors due to incorrect visibility. We exploit this by approximating visibility for indirect illumination with imperfect shadow maps ---low-resolution shadow maps rendered from a crude point-based representation of the scene. These are used in conjunction with a global illumination algorithm based on virtual point lights enabling indirect illumination of dynamic scenes at real-time frame rates. We demonstrate that imperfect shadow maps are a valid approximation to visibility, which makes the simulation of global illumination an order of magnitude faster than using accurate visibility.

Texture mapping with atlases suffer from several drawbacks: Wasted memory, seams, uniform resolution and no support of implicit surfaces. Texture mapping in a volume solves most of these issues, but unfortunately it induces an important space and time overhead.

We reformulate the rendering equation to alleviate the need for explicit visibility computation, thus enabling interactive global illumination on graphics hardware. This is achieved by treating visibility implicitly and propagating an additional quantity, called antiradiance , to compensate for light transmitted extraneously. Our new algorithm shifts visibility computation to simple local iterations by maintaining additional directional antiradiance information with samples in the scene. It is easy to parallelize on a GPU. By correctly treating discretization and filtering, we can compute indirect illumination in scenes with dynamic objects much faster than traditional methods. Our results show interactive update of indirect illumination with moving characters and lights.

In this paper we present a novel method for plausible real-time rendering of indirect illumination effects for diffuse and non-diffuse surfaces. The scene geometry causing indirect illumination is captured by an extended shadow map, as proposed in previous work, and secondary light sources are distributed on directly lit surfaces. One novelty is the rendering of these secondary lights' contribution by splatting in a deferred shading process, which decouples rendering time from scene complexity. An importance sampling strategy, implemented entirely on the GPU, allows efficient selection of secondary light sources. Adapting the light's splat shape to surface glossiness also allows efficient rendering of caustics. Unlike previous approaches the approximated indirect lighting does barely exhibit coarse artifacts - even under unfavorable viewing and lighting conditions. We describe an implementation on contemporary graphics hardware, show a comparison to previous approaches, and present adaptation to and results in game-typical applications.

In this paper we present "reflective shadow maps", an algorithm for interactive rendering of plausible indirect illumination. A reflective shadow map is an extension to a standard shadow map, where every pixel is considered as an indirect light source. The illumination due to these indirect lights is evaluated on-the-fly using adaptive sampling in a fragment shader. By using screen-space interpolation of the indirect lighting, we achieve interactive rates, even for complex scenes. Since we mainly work in screen space, the additional effort is largely independent of scene complexity. The resulting indirect light is approximate, but leads to plausible results and is suited for dynamic scenes. We describe an implementation on current graphics hardware and show results achieved with our approach.

In this paper we present sequential point trees, a data structure that allows adaptive rendering of point clouds completely on the graphics processor. Sequential point trees are based on a hierarchical point representation, but the hierarchical rendering traversal is replaced by sequential processing on the graphics processor, while the CPU is available for other tasks. Smooth transition to triangle rendering for optimized performance is integrated. We describe optimizations for backface culling and texture adaptive point selection. Finally, we discuss implementation issues and show results.