Abstract Implicit surface representations have regained popularity because of their use in machine learning. A common component in optimization is regularization, penalizing the deviation of the surface from its original shape. The popular as‐rigid‐as‐possible (A rap ) energy strikes a good compromise between realistic deformation behavior and efficient computation, at least for piecewise linear meshes. We develop an approach for computing the A rap energy of a deformation function based on point sampling of the surface. The implicit representation is exploited to provide differentials in each sample. The evaluation is efficient and exact in each sample (up to numerical precision). We demonstrate the general applicability of the method to neural shape processing in several applications and contrast its properties with alternatives from the literature.

We investigate the construction of visually smooth spline surfaces that interpolate the vertices of triangulations by blending local patches. Each triangle star carries a locally interpolating surface patch. The patches are only required to interpolate the vertex, whereas in previous methods the patches are often defined per edge, imposing multiple constraints on local approximations. We adopt simple rational blend functions for the triangular domains, that are constructed so that they retain the interpolation and tangent behavior on the patch boundaries. Decoupling local approximation from blending facilitates the exploration of visually pleasing constructions, while controlling the complexity.

Differentiable rendering has become a key ingredient in solving challenging inverse problems in computer graphics and vision. Existing systems can simulate and differentiate the spatial propagation of light. We exploit the duality of light transport simulations and geometric acoustics to apply differential rendering techniques to established acoustic simulation methods. The resulting system is capable of simulating sound according to the geometrical acoustics model and computing derivatives of the output energy spectrograms with respect to arbitrary parameters of the scene, including materials, emitters, microphones, and scene geometry. Contrary to current differentiable transient rendering, we can handle arbitrary simulation depths and achieve constant memory and linear execution times by presenting a temporal extension of Path Replay Backpropagation [Vicini et al. 2021]. We verify our model against established simulation software, and demonstrate the capabilities of optimization with gradients at examples of inverse acoustics and optimizing room parameters. This opens up a new field of research for acoustic optimization that could be as impactful for the acoustic community as differentiable rendering was for the graphics community.

All rendering methods aim at striking a balance between realism and efficiency. This is particularly relevant for differentiable rendering, where the additional aspect of differentiablity w.r.t. scene parameters causes increased computational complexity while, on the other hand, in the common application of inverse rendering, the diverse effects of real image formation must be faithfully reproduced. An important effect in rendering is the attenuation of light as it travels through different media (visibility, shadows, transmittance, transparency). This can be modeled as an integral over non-negative functions and has been successfully approximated in forward rendering by so-called moments. We show that moment-based approximations are differentiable in the parameters defining the moments, and that this leads to efficient and practical methods for inverse rendering. In particular, we demonstrate the method at the examples of shadow mapping and visibility in volume rendering, leading to approximations that are similar in efficiency to existing ad-hoc techniques while being significantly more accurate.

We show how shadows can be efficiently generated in differentiable rendering of triangle meshes. Our central observation is that pre-filtered shadow mapping, a technique for approximating shadows based on rendering from the perspective of a light, can be combined with existing differentiable rasterizers to yield differentiable visibility information. We demonstrate at several inverse graphics problems that differentiable shadow maps are orders of magnitude faster than differentiable light transport simulation with similar accuracy - while differentiable rasterization without shadows often fails to converge.

We propose an efficient method for differentiable rendering of parametric surfaces and curves, which enables their use in inverse graphics problems. Our central observation is that a representative triangle mesh can be extracted from a continuous parametric object in a differentiable and efficient way. We derive differentiable meshing operators for surfaces and curves that provide varying levels of approximation granularity. With triangle mesh approximations, we can readily leverage existing machinery for differentiable mesh rendering to handle parametric geometry. Naively combining differentiable tessellation with inverse graphics settings lacks robustness and is prone to reaching undesirable local minima. To this end, we draw a connection between our setting and the optimization of triangle meshes in inverse graphics and present a set of optimization techniques, including regularizations and coarse-to-fine schemes. We show the viability and efficiency of our method in a set of image-based computer-aided design applications.

We propose an analysis-by-synthesis method for fast multi-view 3D reconstruction of opaque objects with arbitrary materials and illumination. State-of-the-art methods use both neural surface representations and neural rendering. While flexible, neural surface representations are a significant bottleneck in optimization runtime. Instead, we represent surfaces as triangle meshes and build a differentiable rendering pipeline around triangle rasterization and neural shading. The renderer is used in a gradient descent optimization where both a triangle mesh and a neural shader are jointly optimized to reproduce the multi-view images. We evaluate our method on a public 3D reconstruction dataset and show that it can match the reconstruction accuracy of traditional baselines and neural approaches while surpassing them in optimization runtime. Additionally, we investigate the shader and find that it learns an interpretable representation of appearance, enabling applications such as 3D material editing.

Volumetric video is regarded worldwide as the next important development step in the field of media production. Especially in the context of the extremely rapid development of the Virtual Reality (VR) and Augmented Reality (AR) markets, volumetric video is becoming a key technology. In this paper, a new capture and processing system for volumetric video is presented, called 3D Human Body Reconstruction (3DHBR). The system is based on 16 stereo pairs of high-resolution cameras capturing a moving person in 360 degree. A novel stereo approach provides depth information from all perspectives, which is then fused to a single consistent 3D point cloud. A meshing and mesh reduction algorithm finally produces a sequence of meshes that can be integrated into common render engines. Given that, an integration of realistic dynamic 3D reconstructions of moving persons in VR and AR applications is possible.