Abstract Rendering tree‐based analytical Signed Distance Fields (SDFs) through sphere tracing often requires to evaluate many primitives per tracing step, for many steps per pixel of the end image. This cost quickly becomes prohibitive as the number of primitives that constitute the SDF grows. In this paper, we alleviate this cost by computing local pruned trees that are equivalent to the full tree within their region of space while being much faster to evaluate. We introduce an efficient hierarchical tree pruning method based on the Lipschitz property of SDFs, which is compatible with hard and smooth CSG operators. We propose a GPU implementation that enables real‐time sphere tracing of complex SDFs composed of thousands of primitives with dynamic animation. Our pruning technique provides significant speedups for SDF evaluation in general, which we demonstrate on sphere tracing tasks but could also lead to significant improvement for SDF discretization or polygonization.

Abstract Non‐uniform random point sampling on 3D surfaces offers a powerful framework to capture complex distributions such as fur seeds, scattered geometric instances or light emitter flux. As most other on‐surface signals, practitioners design such distribution by the means of 2D maps, parameterized over the surface, and indicating locally the desired point density that should be synthesized, along with any primitive‐specific attribute such as fiber thickness or instance size. Numerous application scenarios imply large such point sets which would ideally be generated in real‐time to be consumed immediately by downstream applications. We propose a method to distribute such white noise point sets under non‐uniform densities, designed to cope with parallel GPU execution and able to produce, in real time, hundreds of millions of density‐constrained samples over arbitrary 3D triangle meshes. At the core of our method, we introduce a stratified rejection sampling scheme where triangles act as strata, greatly improving the locality of the sampling process, and significantly diminishing the probability of rejecting a sample. Our method relies on a series of simple GPU kernels, introducing a new fast and exact texel‐triangle overlap computation method as well as the notion of unordered cumulative sum. As a result, our approach provides real‐time systems with the ability to tailor, on‐the‐fly, highly dynamic density distributions in the form of procedural or raster maps. We illustrate its application with interactive fur design, geometry instancing, and Monte Carlo light sampling.

Geometric features between the micro and macro scales produce an expressive family of visual effects grouped under the term "glints". Efficiently rendering these effects amounts to finding the highlights caused by the geometry under each pixel. To allow for fast rendering, we represent our faceted geometry as a 4D point process on an implicit multiscale grid, designed to efficiently find the facets most likely to cause a highlight. The facets' normals are generated to match a given micro-facet normal distribution such as Trowbridge-Reitz (GGX) or Beckmann, to which our model converges under increasing surface area. Our method is simple to implement, memory-and-precomputation-free, allows for importance sampling and covers a wide range of different appearances such as anisotropic as well as individually colored particles. We provide a base implementation as a standalone fragment shader.

We introduce Sphere Carving , a novel method for automatically computing bounding volumes that closely bound a procedurally defined implicit surface. Starting from an initial bounding volume located far from the object, we iteratively approach the surface by leveraging the signed distance function information. Field function queries define a set of empty spheres, from which we extract intersection points that are used to compute a bounding volume. Our method is agnostic of the function representation and only requires a conservative signed distance field as input. This encompasses a large set of procedurally defined implicit surface models such as exact or Lipschitz functions, BlobTrees, or even neural representations. Sphere Carving is conceptually simple, independent of the function representation, requires a small number of function queries to create bounding volumes, and accelerates queries in Sphere Tracing and polygonization.

High-performance ray tracing of triangle meshes equipped with displacement maps is a challenging task. Existing methods either rely on pre-tessellation, taking full advantage of the hardware but with a poor memory/quality tradeoff, or use custom displacement-centric acceleration structures, preserving all the geometric details but being orders of magnitude slower. We introduce a method that efficiently probes the displacement-map space to find ray-surface intersections without relying on pre-tessellation. Our method combines inverse displacement mapping and on-the-fly surface-bound computation. It employs a novel data structure that provides tight displacement bounds over rectangular regions in the displacement-map space. We demonstrate the effectiveness of our approach in a production GPU path tracer. It can achieve over an order of magnitude speed-up in render time compared to state of the art in the most challenging real-time path-tracing scenarios, while maintaining a low memory footprint.

We present MIPNet, a novel approach for SVBRDF mipmapping which preserves material appearance under varying view distances and lighting conditions. As in classical mipmapping, our method explicitly encodes the multiscale appearance of materials in a SVBRDF mipmap pyramid. To do so, we use a tensor-based representation, coping with gradient-based optimization, for encoding anisotropy which is compatible with existing real-time rendering engines. Instead of relying on a simple texture patch average for each channel independently, we propose a cascaded architecture of multilayer perceptrons to approximate the material appearance using only the fixed material channels. Our neural model learns simple mipmapping filters using a differentiable rendering pipeline based on a rendering loss and is able to transfer signal from normal to anisotropic roughness. As a result, we obtain a drop-in replacement for standard material mipmapping, offering a significant improvement in appearance preservation while still boiling down to a single per-pixel mipmap texture fetch. We report extensive experiments on two distinct BRDF models.

Abstract Image‐Based Rendering allows users to easily capture a scene using a single camera and then navigate freely with realistic results. However, the resulting renderings are completely static, and dynamic effects – such as fire, waterfalls or small waves – cannot be reproduced. We tackle the challenging problem of enabling free‐viewpoint navigation including such stationary dynamic effects, but still maintaining the simplicity of casual capture. Using a single camera – instead of previous complex synchronized multi‐camera setups – means that we have unsynchronized videos of the dynamic effect from multiple views, making it hard to blend them when synthesizing novel views. We present a solution that allows smooth free‐viewpoint video‐based rendering (VBR) of such scenes using temporal Laplacian pyramid decomposition video, enabling spatio‐temporal blending. For effects such as fire and waterfalls, that are semi‐transparent and occupy 3D space, we first estimate their spatial volume. This allows us to create per‐video geometries and alpha‐matte videos that we can blend using our frequency‐dependent method. We also extend Laplacian blending to the temporal dimension to remove additional temporal seams. We show results on scenes containing fire, waterfalls or rippling waves at the seaside, bringing these scenes to life.

Displacement mapping is a powerful mechanism for adding fine to medium geometric details over a 3D surface using a 2D map encoding them. While GPU rasterization supports it through the hardware tessellation unit, ray tracing surface meshes textured with high quality displacement requires a significant amount of memory. More precisely, the input surface needs to be pre-tessellated at the displacement map resolution before being enriched with its mandatory acceleration data structure. Consequently, designing displacement maps interactively while enjoying a full physically-based rendering is often impossible, as simply tiling multiple times the map quickly saturates the graphics memory. In this work we introduce a new tessellation-free displacement mapping approach for ray tracing. Our key insight is to decouple the displacement from its base domain by mapping a displacement-specific acceleration structures directly on the mesh. As a result, our method shows low memory footprint and fast high resolution displacement rendering, making interactive displacement editing possible.

Abstract We propose a novel method to handle thin structures in Image‐Based Rendering (IBR), and specifically structures supported by simple geometric shapes such as planes, cylinders, etc. These structures, e.g. railings, fences, oven grills etc, are present in many man‐made environments and are extremely challenging for multi‐view 3D reconstruction, representing a major limitation of existing IBR methods. Our key insight is to exploit multi‐view information. After a handful of user clicks to specify the supporting geometry, we compute multi‐view and multi‐layer alpha mattes to extract the thin structures. We use two multi‐view terms in a graph‐cut segmentation, the first based on multi‐view foreground color prediction and the second ensuring multiview consistency of labels. Occlusion of the background can challenge reprojection error calculation and we use multiview median images and variance, with multiple layers of thin structures. Our end‐to‐end solution uses the multi‐layer segmentation to create per‐view mattes and the median colors and variance to create a clean background. We introduce a new multi‐pass IBR algorithm based on depth‐peeling to allow free‐viewpoint navigation of multi‐layer semi‐transparent thin structures. Our results show significant improvement in rendering quality for thin structures compared to previous image‐based rendering solutions.

We propose a method to remove objects such as people and cars from multi-view urban image datasets, enabling free-viewpoint IBR in the edited scenes. Our method combines information from multi-view 3D reconstruction with image inpainting techniques, by formulating the problem as an optimization of a global patch-based objective function. We use Image-Based Rendering (IBR) techniques to reproject information from neighboring views, and 3D multi-view stereo reconstruction to perform multiview coherent initialization for inpainting of pixels not filled by reprojection. Our algorithm performs multi-view consistent inpainting for color and 3D by blending reprojections with patch-based image inpainting. We run our algorithm on casually captured datasets, and Google StreetViewdata, removing objects cars, people and pillars, showing that our approach produces results of sufficient quality for free-viewpoint IBR on "cleaned up" scenes, as well as IBR scene editing, such as limited motion of real objects.