Abstract Path tracing remains the gold standard for high‐fidelity subsurface scattering despite requiring numerous paths for noise‐free estimates. We introduce a novel variance‐reduction method based on two complementary zero‐variance‐theory‐based approaches. The first one, analytical Dwivedi sampling, is lightweight but struggles with complex lighting. The second one, surface path guiding, learns incident illumination at boundaries to guide sampled paths, but it does not reduce variance from subsurface scattering. In our novel method, we enhance Dwivedi sampling by incorporating the radiance field learned only at the volume boundary. We use the average normal of points on an illuminated boundary region or directions sampled from distributions of incident light at the boundary as our analytical Dwivedi slab normals. Unlike previous methods based on Dwivedi sampling, our method is efficient even in scenes with complex light rigs typical for movie production and under indirect illumination. We achieve comparable noise reduction and even slightly improved estimates in some scenes compared to volume path guiding, and our method can be easily added on top of any existing surface path guiding system. Our method is particularly effective for homogeneous, isotropic media, bypassing the extensive training and caching inside the 3D volume that volume path guiding requires.

Abstract We present a Monte Carlo path tracing technique to sample extended next event estimation contributions in participating media: we consider one additional scattering vertex on the way to the next event, accounting for focused blur, resulting in visually interesting image features. Our technique is tailored to thin homogeneous media with strongly forward scattering phase functions, such as water or atmospheric haze. Previous methods put emphasis on sampling transmittances or geometric factors, and are either limited to isotropic scattering, or used tabulation or polynomial approximation to account for some specific phase functions. We will show how to jointly importance sample the product of an arbitrary phase function with analytic sampling in the solid angle domain and the two reciprocal squared distance terms of the adjacent edges of the transport path. The technique is fast and simple to implement in an existing rendering system. Our estimator is designed specifically for forward scattering, so the new technique has to be combined with other estimators to cover the backward scattering contributions.

Abstract We present Optimised Path Space Regularisation (OPSR), a novel regularisation technique for forward path tracing algorithms. Our regularisation controls the amount of roughness added to materials depending on the type of sampled paths and trades a small error in the estimator for a drastic reduction of variance in difficult paths, including indirectly visible caustics. We formulate the problem as a joint bias‐variance minimisation problem and use differentiable rendering to optimise our model. The learnt parameters generalise to a large variety of scenes irrespective of their geometric complexity. The regularisation added to the underlying light transport algorithm naturally allows us to handle the problem of near‐specular and glossy path chains robustly. Our method consistently improves the convergence of path tracing estimators, including state‐of‐the‐art path guiding techniques where it enables finding otherwise hard‐to‐sample paths and thus, in turn, can significantly speed up the learning of guiding distributions.

The Manuka rendering architecture has been designed in the spirit of the classic reyes rendering architecture: to enable the creation of visually rich computer generated imagery for visual effects in movie production. Following in the footsteps of reyes over the past 30 years, this means supporting extremely complex geometry, texturing, and shading. In the current generation of renderers, it is essential to support very accurate global illumination as a means to naturally tie together different assets in a picture. This is commonly achieved with Monte Carlo path tracing, using a paradigm often called shade on hit , in which the renderer alternates tracing rays with running shaders on the various ray hits. The shaders take the role of generating the inputs of the local material structure, which is then used by path-sampling logic to evaluate contributions and to inform what further rays to cast through the scene. We propose a shade before hit paradigm instead and minimise I/O strain on the system, leveraging locality of reference by running pattern generation shaders before we execute light transport simulation by path sampling. We describe a full architecture built around this approach, featuring spectral light transport and a flexible implementation of multiple importance sampling ( mis ), resulting in a system able to support a comparable amount of extensibility to what made the reyes rendering architecture successful over many decades.

Abstract We present manifold next event estimation (MNEE), a specialised technique for Monte Carlo light transport simulation to render refractive caustics by connecting surfaces to light sources (next event estimation) across transmissive interfaces. We employ correlated sampling by means of a perturbation strategy to explore all half vectors in the case of rough transmission while remaining outside of the context of Markov chain Monte Carlo, improving temporal stability. MNEE builds on differential geometry and manifold walks. It is very lightweight in its memory requirements, as it does not use light caching methods such as photon maps or importance sampling records. The method integrates seamlessly with existing Monte Carlo estimators via multiple importance sampling.

Abstract We present a spectral rendering technique that offers a compelling set of advantages over existing approaches. The key idea is to propagate energy along paths for a small, constant number of changing wavelengths. The first of these, the hero wavelength, is randomly sampled for each path, and all directional sampling is solely based on it. The additional wavelengths are placed at equal distances from the hero wavelength, so that all path wavelengths together always evenly cover the visible range. A related technique, spectral multiple importance sampling, was already introduced a few years ago. We propose a simplified and optimised version of this approach which is easier to implement, has good performance characteristics, and is actually more powerful than the original method. Our proposed method is also superior to techniques which use a static spectral representation, as it does not suffer from any inherent representation bias. We demonstrate the performance of our method in several application areas that are of critical importance for production work, such as fidelity of colour reproduction, sub‐surface scattering, dispersion and volumetric effects. We also discuss how to couple our proposed approach with several technologies that are important in current production systems, such as photon maps, bidirectional path tracing, environment maps, and participating media.

We introduce two fourth-order regularization methods that remove geometric noise without destroying significant geometric features. These methods leverage ideas from image denoising and simplification of high contrast images in which piecewise affine functions are preserved up to infinitesimally small transition zones. We combine the regularization techniques with active contour models and apply them to segmentation of polygonal objects in aerial images. To avoid loss of features during the computation of the external driving forces we use total variation–based inverse scale-space techniques on the input data. Furthermore, we use the models for feature-preserving removal of geometric texture on surfaces.

Multimodal image registration significantly benefits from previous denoising and structure segmentation and vice versa. In particular combined information of different image modalities makes segmentation significantly more robust. Indeed, fundamental tasks in image processing are highly interdependent. A variational approach is presented, which combines the detection of corresponding edges, an edge preserving denoising and the morphological registration via a non-rigid deformation for a pair of images with structural correspondence. The morphology of an image function is split into a singular part consisting of the edge set and a regular part represented by the field of normals on the ensemble of level sets. A Mumford-Shah type free discontinuity problem is applied to treat the singular morphology and the matching of corresponding edges under the deformation. The matching of the regular morphology is quantified by a second contribution which compares deformed normals and normals at deformed positions. Finally, a nonlinear elastic energy controls the deformation itself and ensures smoothness and injectivity. A multi scale approach that is based on a phase field approximation leads to an effective and efficient algorithm. Numerical experiments underline the robustness of the presented approach and show applications on medical images.

This paper presents a new algorithm based on the Mumford-Shah model for simultaneously detecting the edge features of two images and jointly estimating a consistent set of transformations to match them. Compared to the current asymmetric methods in the literature, this fully symmetric method allows one to determine one-to-one correspondences between the edge features of two images. The entire variational model is realized in a multiscale framework of the finite element approximation. The optimization process is guided by an estimation minimization-type algorithm and an adaptive generalized gradient flow to guarantee a fast and smooth relaxation. The algorithm is tested on T1 and T2 magnetic resonance image data to study the parameter setting. We also present promising results of four applications of the proposed algorithm: interobject monomodal registration, retinal image registration, matching digital photographs of neurosurgery with its volume data, and motion estimation for frame interpolation.

We present a novel variational method to nonrigid registration of multimodal data. A suitable deformation will be determined via the minimization of a morphological, i.e., contrast invariant, matching functional along with an appropriate regularization energy. Here we want to give special focus on the practical issues involving scale-space methods, regularization of the corresponding gradient flow and hyperelastic regularization.

A two-handed 3D styling system for free-form surfaces in a table-like Virtual Environment, the Responsive Workbench (RWB)TM, is described. Intuitive curve and surface deformation tools based on variational modeling and interaction techniques adapted to 3D VR modeling applications are proposed. The user draws curves (cubic B-splines) directly in the Virtual Environment using a stylus as an input device. The curves are connected automatically, such that a curve network develops. A combination of automatic and user-controlled topology extraction modules creates the connectivity information. The underlying surface model is based on B-spline surfaces, or, alternatively, uses multisided patches [20] bounded by closed loops of curve pieces.