Generative AI’s function as a frictionless "answer engine" creates a paradox in educational HCI: the very tools that can enhance intellect may also weaken it by allowing users to circumvent crucial cognitive processes. This risks creating a "hollowed mind"—knowledge that is broad but superficial, and a user experience that diminishes learner agency. The convenience of cognitive offloading introduces a motivational challenge that traditional cognitive scaffolding cannot address. We argue that designing genuine human-AI partnerships in learning requires moving beyond cognitive support to motivation-aware scaffolding. This paper provides a toolkit for building motivation-aware AI systems. At its core is the Dual Zone of Proximal Development (DZPD), a conceptual framework building on foundational work in educational psychology. We introduce an overarching design principle, concrete design principles, illustrative archetypes, and examples of measurable indicators. These conceptual tools offer essential guidance for the next wave of empirical HCI research in education.

We present a numerically robust algorithm for augmenting inconsistent meshes and polygon soups with curves defined on a mesh proxy. Unlike prior approaches, which transfer only vertex positions to the proxy, our method represents input edges on the proxy by geodesic paths and merges them with proxy-defined edge paths such as UV seams. This enables reliable detection of intersections between geodesic paths and proxy seams and avoids explicit edge-edge intersection tests, which makes the construction stable even on poorly conditioned meshes. We demonstrate the method in texture transfer, where projected input faces are cut along UV seams of the proxy so that the reconstructed result mesh reproduces the proxy seam layout explicitly and avoids distortions that previous approaches could only mitigate heuristically. To construct this result mesh, we first build an overlay curve network consisting of geodesic segments and proxy seam edges, and then reconstruct a manifold mesh aligned with the input mesh. Our experiments show that this reconstruction removes seam artifacts and reduces the associated high-distortion outliers of prior heuristic methods. Beyond UV seams, the construction can be used more generally to transfer proxy-defined curves onto disjoint meshes, such as collections of NURBS patches.

Simulating inviscid, incompressible fluids on non-simply-connected curved surfaces requires careful treatment of the flow's local and global behavior. While recent theoretical advancements have established the critical dynamics of the harmonic component in such flows, practical applications remain computationally restricted by a lack of spatial and temporal adaptivity. Furthermore, simulations on poor-quality meshes often lead to numerical instability and a failure to preserve the flow's underlying harmonic component when using naive interpolation methods. In this paper, we introduce Adaptive Fluid Cohomology, a framework that integrates dynamic spatial and temporal refinement into the simulation of the Euler equations. We leverage a posteriori error estimation to adjust spatial resolution on the fly, alongside a standard Dormand-Prince 5(4) time-stepping scheme for temporal accuracy. To ensure stability during mesh mutations, we develop a novel method that robustly transfers the harmonic basis during remeshing. While our experimental evaluation focuses on 2D surface flows, the underlying theoretical formulation is presented to capture the 3D setting as well. Our evaluation demonstrates that this adaptive approach accurately recreates the dynamics of high-resolution simulations while reducing the memory footprint by up to 86% and maintaining numerical stability even on poor-quality triangulations where static methods fail.

The reconstruction of three-dimensional dynamic scenes is a well-established yet challenging task within the domain of computer vision. In this paper, we propose a novel approach that combines the domains of 3D geometry reconstruction and appearance estimation for physically based rendering and present a system that is able to perform both tasks for fabrics, utilizing only a single monocular RGB video sequence as input. In order to obtain realistic and high-quality deformations and renderings, a physical simulation of the cloth geometry and differentiable rendering are employed. In this paper, we introduce two novel regularization terms for the 3D reconstruction task that improve the plausibility of the reconstruction by addressing the depth ambiguity problem in monocular video. In comparison with the most recent methods in the field, we have reduced the error in the 3D reconstruction by a factor of 2.64 while requiring a medium runtime of 30 min per scene. Furthermore, the optimized motion achieves sufficient quality to perform an appearance estimation of the deforming object, recovering sharp details from this single monocular RGB video.

Efficient physics simulations are essential for numerous applications, ranging from realistic cloth animations in video games, to analyzing pollutant dispersion in environmental sciences, to calculating vehicle drag coefficients in engineering applications. Unfortunately, analytical solutions to the underlying physical equations are rarely available, and numerical solutions are computationally demanding. Latest developments in the field of physics-based Deep Learning have led to promising efficiency gains but still suffer from limited generalization capabilities across multiple different PDEs. Thus, in this work, we introduce **Metamizer**, a novel neural optimizer that iteratively solves a wide range of physical systems without retraining by minimizing a physics-based loss function. To this end, our approach leverages a scale-invariant architecture that enhances gradient descent updates to accelerate convergence. Since the neural network itself acts as an optimizer, training this neural optimizer falls into the category of meta-optimization approaches. We demonstrate that Metamizer achieves high accuracy across multiple PDEs after training on the Laplace, advection-diffusion and incompressible Navier-Stokes equation as well as on cloth simulations. Remarkably, the model also generalizes to PDEs that were not covered during training such as the Poisson, wave and Burgers equation.

While neural radiance fields (NeRF) led to a break-through in photorealistic novel view synthesis, handling mirroring surfaces still denotes a particular challenge as they introduce severe inconsistencies in the scene representation. Previous attempts either focus on reconstructing single reflective objects or rely on strong supervision guidance in terms of additional user-provided annotations of visible image regions of the mirrors, thereby limiting the practical usability. In contrast, in this paper, we present NeRF-MD, a method which shows that NeRFs can be considered as mir-ror detectors and which is capable of reconstructing neu-ral radiance fields of scenes containing mirroring surfaces without the need for prior annotations. To this end, we first compute an initial estimate of the scene geometry by training a standard NeRF using a depth reprojection loss. Our key insight lies in the fact that parts of the scene corresponding to a mirroring surface will still exhibit a significant pho-tometric inconsistency, whereas the remaining parts are al-ready reconstructed in a plausible manner. This allows us to detect mirror surfaces by fitting geometric primitives to such inconsistent regions in this initial stage of the training. Using this information, we then jointly optimize the radiance field and mirror geometry in a second training stage to refine their quality. We demonstrate the capability of our method to allow the faithful detection of mirrors in the scene as well as the reconstruction of a single consistent scene representation, and demonstrate its potential in comparison to baseline and mirror-aware approaches.

Reconstructing an object's shape and appearance in terms of a mesh textured by a spatially-varying bidirectional reflectance distribution function (SVBRDF) from a limited set of images captured under collocated light is an ill-posed problem. Previous state-of-the-art approaches either aim to reconstruct the appearance directly on the geometry or additionally use texture normals as part of the appearance features. However, this requires detailed but inefficiently large meshes, that would have to be simplified in a post-processing step, or suffers from well-known limitations of normal maps such as missing shadows or incorrect silhouettes. Another limiting factor is the fixed and typically low resolution of the texture estimation resulting in loss of important surface details. To overcome these problems, we present ROSA, an inverse rendering method that directly optimizes mesh geometry with spatially adaptive mesh resolution solely based on the image data. In particular, we refine the mesh and locally condition the surface smoothness based on the estimated normal texture and mesh curvature. In addition, we enable the reconstruction of fine appearance details in high-resolution textures through a pioneering tile-based method that operates on a single pre-trained decoder network but is not limited by the network output resolution.

Abstract Dynamic surface reconstruction of objects from point cloud sequences is a challenging field in computer graphics. Existing approaches either require multiple regularization terms or extensive training data which, however, lead to compromises in reconstruction accuracy as well as over‐smoothing or poor generalization to unseen objects and motions. To address these limitations, we introduce Preconditioned Deformation Grids , a novel technique for estimating coherent deformation fields directly from unstructured point cloud sequences without requiring or forming explicit correspondences. Key to our approach is the use of multi‐resolution voxel grids that capture the overall motion at varying spatial scales, enabling a more flexible deformation representation. In conjunction with incorporating grid‐based Sobolev preconditioning into gradient‐based optimization, we show that applying a Chamfer loss between the input point clouds as well as to an evolving template mesh is sufficient to obtain accurate deformations. To ensure temporal consistency along the object surface, we include a weak isometry loss on mesh edges which complements the main objective without constraining deformation fidelity. Extensive evaluations demonstrate that our method achieves superior results, particularly for long sequences, compared to state‐of‐the‐art techniques.

Abstract Image‐based lighting is a widely used technique to reproduce shading under real‐world lighting conditions, especially in real‐time rendering applications. A particularly challenging scenario involves materials exhibiting a sparkling or glittering appearance, caused by discrete microfacets scattered across their surface. In this paper, we propose an efficient approximation for image‐based lighting of glints, enabling fully dynamic material properties and environment maps. Our novel approach is grounded in real‐time glint rendering under area light illumination and employs standard environment map filtering techniques. Crucially, our environment map filtering process is sufficiently fast to be executed on a per‐frame basis. Our method assumes that the environment map is partitioned into few homogeneous regions of constant radiance. By filtering the corresponding indicator functions with the normal distribution function, we obtain the probabilities for individual microfacets to reflect light from each region. During shading, these probabilities are utilized to hierarchically sample a multinomial distribution, facilitated by our novel dual‐gated Gaussian approximation of binomial distributions. We validate that our real‐time approximation is close to ground‐truth renderings for a range of material properties and lighting conditions, and demonstrate robust and stable performance, with little overhead over rendering glints from a single directional light. Compared to rendering smooth materials without glints, our approach requires twice as much memory to store the prefiltered environment map.

3D reconstruction of dynamic scenes is a long-standing problem in computer graphics and increasingly difficult the less information is available. Shape-from-Template (SfT) methods aim to reconstruct a template-based geometry from RGB images or video sequences, often leveraging just a single monocular camera without depth information, such as regular smartphone recordings. Unfortunately, existing reconstruction methods are either unphysical and noisy or slow in optimization. To solve this problem, we propose a novel SfT reconstruction algorithm for cloth using a pre-trained neural surrogate model that is fast to evaluate, stable, and produces smooth reconstructions due to a regularizing physics simulation. Differentiable rendering of the simulated mesh enables pixel-wise comparisons between the reconstruction and a target video sequence that can be used for a gradient-based optimization procedure to extract not only shape information but also physical parameters such as stretching, shearing, or bending stiffness of the cloth. This allows to retain a precise, stable, and smooth reconstructed geometry while reducing the runtime by a factor of 400–500 compared to ϕ-SfT, a state-of-the-art physics-based SfT approach.

In 1997, the very first tour guide robot RHINO was deployed in a museum in Germany. With the ability to navigate autonomously through the environment, the robot gave tours to over 2,000 visitors. Today, RHINO itself has become an exhibit and is no longer operational In this paper, we present RHINO-VR, an interactive museum exhibit using virtual reality (VR) that allows museum visitors to experience the historical robot RHINO in operation in a virtual museum. RHINO-VR, unlike static exhibits, enables users to familiarize themselves with basic mobile robotics concepts without the fear of damaging the exhibit. In the virtual environment, the user is able to interact with RHINO in VR by pointing to a location to which the robot should navigate and observing the corresponding actions of the robot. To include other visitors who cannot use the VR, we provide an external observation view to make RHINO visible to them. We evaluated our system by measuring the frame rate of the VR simulation, comparing the generated virtual 3D models with the originals, and conducting a user study. The user study showed that RHINO-VR improved the visitors’ understanding of the robot’s functionality and that they would recommend experiencing the VR exhibit to others.

We investigate the capabilities of neural inverse procedural modeling to infer high-quality procedural yarn models with fiber-level details from single images of depicted yarn samples. While directly inferring all parameters of the underlying yarn model based on a single neural network may seem an intuitive choice, we show that the complexity of yarn structures in terms of twisting and migration characteristics of the involved fibers can be better encountered in terms of ensembles of networks that focus on individual characteristics. We analyze the effect of different loss functions including a parameter loss to penalize the deviation of inferred parameters to ground truth annotations, a reconstruction loss to enforce similar statistics of the image generated for the estimated parameters in comparison to training images as well as an additional regularization term to explicitly penalize deviations between latent codes of synthetic images and the average latent code of real images in the encoder’s latent space. We demonstrate that the combination of a carefully designed parametric, procedural yarn model with respective network ensembles as well as loss functions even allows robust parameter inference when solely trained on synthetic data. Since our approach relies on the availability of a yarn database with parameter annotations and we are not aware of such a respectively available dataset, we additionally provide, to the best of our knowledge, the first dataset of yarn images with annotations regarding the respective yarn parameters. For this purpose, we use a novel yarn generator that improves the realism of the produced results over previous approaches.

Many materials, such as sand, rice, wheat, or other kinds of seeds, consist of numerous individual grains that determine the visual appearance of these materials. When generating images of these mixtures, the primary challenge is to simulate the interaction of light with each individual grain. While subsurface scattering effects are crucial for producing realistic images, the computation of light transport using standard path tracing methods for each grain can be prohibitively expensive. Although there have been several methods developed to address this issue, they all assume that bounding spheres of individual grains do not intersect. This restriction limits the application of these methods to almost spherical grains. Nonetheless, various grains, such as seeds and rice, are non-spherical, making this assumption lead to impractical stackings in situations involving coarse-grained materials. We address this issue by presenting a subsurface scattering model that utilizes a neural network and is trained using an optimal transport framework. Our model surpasses path tracing approaches conclusively, allowing for efficient rendering of granular mixtures that were previously unfeasible. Additionally, this method can be utilized in large-scale procedural generated scenes based on sphere packings and obtains similar results as previous methods in these cases.

Abstract Implicit representations like neural radiance fields (NeRF) showed impressive results for photorealistic rendering of complex scenes with fine details. However, ideal or near‐perfectly specular reflecting objects such as mirrors, which are often encountered in various indoor scenes, impose ambiguities and inconsistencies in the representation of the re‐constructed scene leading to severe artifacts in the synthesized renderings. In this paper, we present a novel reflection tracing method tailored for the involved volume rendering within NeRF that takes these mirror‐like objects into account while avoiding the cost of straightforward but expensive extensions through standard path tracing. By explicitly modelling the reflection behaviour using physically plausible materials and estimating the reflected radiance with Monte‐Carlo methods within the volume rendering formulation, we derive efficient strategies for importance sampling and the transmittance computation along rays from only few samples. We show that our novel method enables the training of consistent representations of such challenging scenes and achieves superior results in comparison to previous state‐of‐the‐art approaches.

We present incomplete gamma kernels, a generalization of Locally Optimal Projection (LOP) operators. In particular, we reveal the relation of the classical localized$ L_{1}$estimator, used in the LOP operator for point cloud denoising, to the common Mean Shift framework via a novel kernel. Furthermore, we generalize this result to a whole family of kernels that are built upon the incomplete gamma function and each represents a localized$ L_{p}$estimator. By deriving various properties of the kernel family concerning distributional, Mean Shift induced, and other aspects such as strict positive definiteness, we obtain a deeper understanding of the operator's projection behavior. From these theoretical insights, we illustrate several applications ranging from an improved Weighted LOP (WLOP) density weighting scheme and a more accurate Continuous LOP (CLOP) kernel approximation to the definition of a novel set of robust loss functions. These incomplete gamma losses include the Gaussian and LOP loss as special cases and can be applied to various tasks including normal filtering. Furthermore, we show that the novel kernels can be included as priors into neural networks. We demonstrate the effects of each application in a range of quantitative and qualitative experiments that highlight the benefits induced by our modifications.

Abstract Fourier PlenOctrees have shown to be an efficient representation for real-time rendering of dynamic neural radiance fields (NeRF). Despite its many advantages, this method suffers from artifacts introduced by the involved compression when combining it with recent state-of-the-art techniques for training the static per-frame NeRF models. In this paper, we perform an in-depth analysis of these artifacts and leverage the resulting insights to propose an improved representation. In particular, we present a novel density encoding that adapts the Fourier-based compression to the characteristics of the transfer function used by the underlying volume rendering procedure and leads to a substantial reduction of artifacts in the dynamic model. We demonstrate the effectiveness of our enhanced Fourier PlenOctrees in the scope of quantitative and qualitative evaluations on synthetic and real-world scenes.

In this paper, we present an inverse rendering method for the simple reconstruction of shape and appearance of real-world objects from only roughly calibrated RGB images captured under collocated point light illumination. To this end, we gradually reconstruct the lower-frequency geometry information using automatically generated occupancy mask images based on a visual hull initialization of the mesh, to infer the object topology, and a smoothness-preconditioned optimization. By combining this geometry estimation with learning-based SVBRDF parameter inference as well as intrinsic and extrinsic camera parameter refinement in a joint and unified formulation, our novel method is able to reconstruct shape and an isotropic SVBRDF from fewer input images than previous methods. Unlike in other works, we also estimate normal maps as part of the SVBRDF to capture and represent higher-frequency geometric details in a compact way. Furthermore, by regularizing the appearance estimation with a GAN-based SVBRDF generator, we are able to meaningfully limit the solution space. In summary, this leads to a robust automatic reconstruction algorithm for shape and appearance. We evaluated our algorithm on synthetic as well as on real-world data and demonstrate that our method is able to reconstruct complex objects with high-fidelity reflection properties in a robust way, also in the presence of imperfect camera parameter data.

Partial Differential Equations (PDEs) are notoriously difficult to solve. In general, closed form solutions are not available and numerical approximation schemes are computationally expensive. In this paper, we propose to approach the solution of PDEs based on a novel technique that combines the advantages of two recently emerging machine learning based approaches. First, physics-informed neural networks (PINNs) learn continuous solutions of PDEs and can be trained with little to no ground truth data. However, PINNs do not generalize well to unseen domains. Second, convolutional neural networks provide fast inference and generalize but either require large amounts of training data or a physics-constrained loss based on finite differences that can lead to inaccuracies and discretization artifacts. We leverage the advantages of both of these approaches by using Hermite spline kernels in order to continuously interpolate a grid-based state representation that can be handled by a CNN. This allows for training without any precomputed training data using a physics-informed loss function only and provides fast, continuous solutions that generalize to unseen domains. We demonstrate the potential of our method at the examples of the incompressible Navier-Stokes equation and the damped wave equation. Our models are able to learn several intriguing phenomena such as Karman vortex streets, the Magnus effect, Doppler effect, interference patterns and wave reflections. Our quantitative assessment and an interactive real-time demo show that we are narrowing the gap in accuracy of unsupervised ML based methods to industrial solvers for computational fluid dynamics (CFD) while being orders of magnitude faster.

We introduce canonical weight normalization for convolutional neural networks. Inspired by the canonical tensor decomposition, we express the weight tensors in so-called canonical networks as scaled sums of outer vector products. In particular, we train network weights in the decomposed form, where scale weights are optimized separately for each mode. Additionally, similarly to weight normalization, we include a global scaling parameter. We study the initialization of the canonical form by running the power method and by drawing randomly from Gaussian or uniform distributions. Our results indicate that we can replace the power method with cheaper initializations drawn from standard distributions. The canonical re-parametrization leads to competitive normalization performance on the MNIST, CIFAR10, and SVHN data sets. Moreover, the formulation simplifies network compression. Once training has converged, the canonical form allows convenient model-compression by truncating the parameter sums.

Image-based appearance measurements are fundamentally limited in spatial resolution by the acquisition hardware. Due to the ever-increasing resolution of displaying hardware, high-resolution representations of digital material appearance are desireable for authentic renderings. In the present paper, we demonstrate that high-resolution bidirectional texture functions (BTFs) for materials can be obtained from low-resolution measurements using single-image convolutional neural network (CNN) architectures for image super-resolution. In particular, we show that this approach works for high-dynamic-range data and produces consistent BTFs, even though it operates on an image-by-image basis. Moreover, the CNN can be trained on down-sampled measured data, therefore no high-resolution ground-truth data, which would be difficult to obtain, is necessary. We train and test our method's performance on a large-scale BTF database and evaluate against the current state-of-the-art in BTF super-resolution, finding superior performance.

As service robots are entering more and more homes it gets evermore important to find behavior strategies that ensure a harmonic coexistence between those systems and their users. In this paper, we present a novel approach to enable a mobile robot to provide timely assistance to a user moving in its environment, while simultaneously avoiding unnecessary movements as well as interferences with the user. We developed a framework that uses information about the last object interaction to predict possible future movement destinations of the user and infer where they might need assistance based on prior knowledge. Given this prediction, the robot chooses the best position for itself that minimizes the time until assistance can be provided as well as avoids interferences with other activities of the user. We evaluated our approach in comparison to state-of-the-art methods in simulated environments and performed a user study in a virtual reality environment. Our evaluation demonstrates that our approach is able to decrease both the time until assistance is provided and the travel distance of the robot as well as increases the average distance between the user and the robot in comparison to state-of-the-art systems. Additionally, the robot behavior generated by our method is rated as more pleasant by our study participants than comparable literature approaches.

Abstract Plant growth visualization from a series of 3D scanner measurements is a challenging task. Time intervals between successive measurements are typically too large to allow a smooth animation of the growth process. Therefore, obtaining a smooth animation of the plant growth process requires a temporal upsampling of the point cloud sequence in order to obtain approximations of the intermediate states between successive measurements. Additionally, there are suddenly arising structural changes due to the occurrence of new plant parts such as new branches or leaves. We present a novel method that addresses these challenges via semantic segmentation and the generation of a segment hierarchy per scan, the matching of the hierarchical representations of successive scans and the segment‐wise computation of optimal transport. The transport problems' solutions yield the information required for a realistic temporal upsampling, which is generated in real time. Thereby, our method does not require shape templates, good correspondences or huge databases of examples. Newly grown and decayed parts of the plant are detected as unmatched segments and are handled by identifying corresponding bifurcation points and introducing virtual segments in the previous, respectively successive time step. Our method allows the generation of realistic upsampled growth animations with moderate computational effort.

Real-time reflectance reconstruction under uncontrolled illumination conditions is well-known to be a challenging task due to the complex interplay of scene geometry, surface reflectance and illumination. Nonetheless, recent works succeed in recovering both unknown reflectance and illumination in an uncontrolled setting. However, they are either limited regarding the scene complexity (single objects / homogeneous materials) or are not suitable for real-time applications. Our proposed method enables the recovery of heterogeneous surface reflectance (multiple objects and spatially varying materials) in complex scenes at real-time frame rates. We achieve this goal in the following way: First, we perform a 3D scene reconstruction from an input RGB-D stream in real-time. We then use a deep learning based method to estimate Ward BRDF parameters from observations gathered from individual segmented scene objects. Subsequently we refine these reflectance parameters to allow for spatial variations across the object surfaces. We evaluate our method on synthetic scenes and successfully apply it to real-world data.

The analysis and modeling of cloth has received a lot of attention in recent years. While recent approaches are focused on woven cloth, we present a novel practical approach for the inference of more complex knitwear structures as well as the respective knitting instructions from only a single image without attached annotations. Knitwear is produced by repeating instances of the same pattern, consisting of grid-like arrangements of a small set of basic stitch types. Our framework addresses the identification and localization of the occurring stitch types, which is challenging due to huge appearance variations. The resulting coarsely localized stitch types are used to infer the underlying grid structure as well as for the extraction of the knitting instruction of pattern repeats, taking into account principles of Gestalt theory. Finally, the derived instructions allow the reproduction of the knitting structures, either as renderings or by actual knitting, as demonstrated in several examples.

Applications like disaster management and industrial inspection often require experts to enter contaminated places. To circumvent the need for physical presence, it is desirable to generate a fully immersive individual live teleoperation experience. However, standard video-based approaches suffer from a limited degree of immersion and situation awareness due to the restriction to the camera view, which impacts the navigation. In this paper, we present a novel VR-based practical system for immersive robot teleoperation and scene exploration. While being operated through the scene, a robot captures RGB-D data that is streamed to a SLAM-based live multiclient telepresence system. Here, a global 3D model of the already captured scene parts is reconstructed and streamed to the individual remote user clients where the rendering for e.g. head-mounted display devices (HMDs) is performed. We introduce a novel lightweight robot client component which transmits robot-specific data and enables a quick integration into existing robotic systems. This way, in contrast to first- person exploration systems, the operators can explore and navigate in the remote site completely independent of the current position and view of the capturing robot, complementing traditional input devices for teleoperation. We provide a proof-of-concept implementation and demonstrate the capabilities as well as the performance of our system regarding interactive object measurements and bandwidth-efficient data streaming and visualization. Furthermore, we show its benefits over purely video-based teleoperation in a user study revealing a higher degree of situation awareness and a more precise navigation in challenging environments.

Sharing live telepresence experiences for teleconferencing or remote collaboration receives increasing interest with the recent progress in capturing and AR/VR technology. Whereas impressive telepresence systems have been proposed on top of on-the-fly scene capture, data transmission and visualization, these systems are restricted to the immersion of single or up to a low number of users into the respective scenarios. In this paper, we direct our attention on immersing significantly larger groups of people into live-captured scenes as required in education, entertainment or collaboration scenarios. For this purpose, rather than abandoning previous approaches, we present a range of optimizations of the involved reconstruction and streaming components that allow the immersion of a group of more than 24 users within the same scene - which is about a factor of 6 higher than in previous work - without introducing further latency or changing the involved consumer hardware setup. We demonstrate that our optimized system is capable of generating high-quality scene reconstructions as well as providing an immersive viewing experience to a large group of people within these live-captured scenes.

Abstract Iridescence is a natural phenomenon that is perceived as gradual color changes, depending on the view and illumination direction. Prominent examples are the colors seen in oil films and soap bubbles. Unfortunately, iridescent effects are particularly difficult to recreate in real‐time computer graphics. We present a high‐quality real‐time method for rendering iridescent effects under image‐based lighting. Previous methods model dielectric thin‐films of varying thickness on top of an arbitrary micro‐facet model with a conducting or dielectric base material, and evaluate the resulting reflectance term, responsible for the iridescent effects, only for a single direction when using real‐time image‐based lighting. This leads to bright halos at grazing angles and over‐saturated colors on rough surfaces, which causes an unnatural appearance that is not observed in ground truth data. We address this problem by taking the distribution of light directions, given by the environment map and surface roughness, into account when evaluating the reflectance term. In particular, our approach prefilters the first and second moments of the light direction, which are used to evaluate a filtered version of the reflectance term. We show that the visual quality of our approach is superior to the ones previously achieved, while having only a small negative impact on performance.

Abstract The use of spatially varying reflectance models (SVBRDF) is the state of the art in physically based rendering and the ultimate goal is to acquire them from real world samples. Recently several promising deep learning approaches have emerged that create such models from a few uncalibrated photos, after being trained on synthetic SVBRDF datasets. While the achieved results are already very impressive, the reconstruction accuracy that is achieved by these approaches is still far from that of specialized devices. On the other hand, fitting SVBRDF parameter maps to the gibabytes of calibrated HDR images per material acquired by state of the art high quality material scanners takes on the order of several hours for realistic spatial resolutions. In this paper, we present a first deep learning approach that is capable of producing SVBRDF parameter maps more than two orders of magnitude faster than state of the art approaches, while still providing results of equal quality and generalizing to new materials unseen during the training. This is made possible by training our network on a large‐scale database of material scans that we have gathered with a commercially available SVBRDF scanner. In particular, we train a convolutional neural network to map calibrated input images to the 13 parameter maps of an anisotropic Ward BRDF, modified to account for Fresnel reflections, and evaluate the results by comparing the measured images against re‐renderings from our SVBRDF predictions. The novel approach is extensively validated on real world data taken from our material database, which we make publicly available under https://cg.cs.uni‐bonn.de/svbrdfs/ .

This paper presents a novel mixed reality based navigation system for accurate respiratory liver tumor punctures in radiofrequency ablation (RFA). Our system contains an optical see-through head-mounted display device (OST-HMD), Microsoft HoloLens for perfectly overlaying the virtual information on the patient, and a optical tracking system NDI Polaris for calibrating the surgical utilities in the surgical scene. Compared with traditional navigation method with CT, our system aligns the virtual guidance information and real patient and real-timely updates the view of virtual guidance via a position tracking system. In addition, to alleviate the difficulty during needle placement induced by respiratory motion, we reconstruct the patient-specific respiratory liver motion through statistical motion model to assist doctors precisely puncture liver tumors. The proposed system has been experimentally validated on vivo pigs with an accurate real-time registration approximately 5-mm mean FRE and TRE, which has the potential to be applied in clinical RFA guidance.

Real-time 3D scene reconstruction from RGB-D sensor data, as well as the exploration of such data in VR/AR settings, has seen tremendous progress in recent years. The combination of both these components into telepresence systems, however, comes with significant technical challenges. All approaches proposed so far are extremely demanding on input and output devices, compute resources and transmission bandwidth, and they do not reach the level of immediacy required for applications such as remote collaboration. Here, we introduce what we believe is the first practical client-server system for real-time capture and many-user exploration of static 3D scenes. Our system is based on the observation that interactive frame rates are sufficient for capturing and reconstruction, and real-time performance is only required on the client site to achieve lag-free view updates when rendering the 3D model. Starting from this insight, we extend previous voxel block hashing frameworks by introducing a novel thread-safe GPU hash map data structure that is robust under massively concurrent retrieval, insertion and removal of entries on a thread level. We further propose a novel transmission scheme for volume data that is specifically targeted to Marching Cubes geometry reconstruction and enables a 90% reduction in bandwidth between server and exploration clients. The resulting system poses very moderate requirements on network bandwidth, latency and client-side computation, which enables it to rely entirely on consumer-grade hardware, including mobile devices. We demonstrate that our technique achieves state-of-the-art representation accuracy while providing, for any number of clients, an immersive and fluid lag-free viewing experience even during network outages.

Abstract The advent of affordable consumer grade RGB‐D cameras has brought about a profound advancement of visual scene reconstruction methods. Both computer graphics and computer vision researchers spend significant effort to develop entirely new algorithms to capture comprehensive shape models of static and dynamic scenes with RGB‐D cameras. This led to significant advances of the state of the art along several dimensions. Some methods achieve very high reconstruction detail, despite limited sensor resolution. Others even achieve real‐time performance, yet possibly at lower quality. New concepts were developed to capture scenes at larger spatial and temporal extent. Other recent algorithms flank shape reconstruction with concurrent material and lighting estimation, even in general scenes and unconstrained conditions. In this state‐of‐the‐art report, we analyze these recent developments in RGB‐D scene reconstruction in detail and review essential related work. We explain, compare, and critically analyze the common underlying algorithmic concepts that enabled these recent advancements. Furthermore, we show how algorithms are designed to best exploit the benefits of RGB‐D data while suppressing their often non‐trivial data distortions. In addition, this report identifies and discusses important open research questions and suggests relevant directions for future work.

Estimating depth from a single RGB image is an ill-posed and inherently ambiguous problem. State-of-the-art deep learning methods can now estimate accurate 2D depth maps, but when the maps are projected into 3D, they lack local detail and are often highly distorted. We propose a fast-to-train two-streamed CNN that predicts depth and depth gradients, which are then fused together into an accurate and detailed depth map. We also define a novel set loss over multiple images; by regularizing the estimation between a common set of images, the network is less prone to overfitting and achieves better accuracy than competing methods. Experiments on the NYU Depth v2 dataset shows that our depth predictions are competitive with state-of-the-art and lead to faithful 3D projections.

We introduce a method for automated temporal segmentation of human motion data into distinct actions and compositing motion primitives based on self-similar structures in the motion sequence. We use neighborhood graphs for the partitioning and the similarity information in the graph is further exploited to cluster the motion primitives into larger entities of semantic significance. The method requires no assumptions about the motion sequences at hand and no user interaction is required for the segmentation or clustering. In addition, we introduce a feature bundling preprocessing technique to make the segmentation more robust to noise, as well as a notion of motion symmetry for more refined primitive detection. We test our method on several sensor modalities, including markered and markerless motion capture as well as on electromyograph and accelerometer recordings. The results highlight our system's capabilities for both segmentation and for analysis of the finer structures of motion data, all in a completely unsupervised manner.

Building upon previous works, we transfer the recently proposed moment shadow mapping to three new applications. Like variance shadow maps and convolution shadow maps, moment shadow maps can be filtered directly. Classically, this is used to filter hard shadows but previous works explore other applications. Prefiltered single scattering uses convolution shadow maps to render single scattering in homogenous participating media, variance soft shadow mapping uses variance shadow maps for approximate soft shadows and Fourier opacity mapping uses convolution shadow maps for translucent occluders. We combine these three techniques with moment shadow mapping to arrive at better heuristics with less computational overhead.

We present an automatic approach for the reconstruction of parametric 3D building models from indoor point clouds. While recently developed methods in this domain focus on mere local surface reconstructions which enable e.g. efficient visualization, our approach aims for a volumetric, parametric building model that additionally incorporates contextual information such as global wall connectivity. In contrast to pure surface reconstructions, our representation thereby allows more comprehensive use: first, it enables efficient high-level editing operations in terms of e.g. wall removal or room reshaping which always result in a topologically consistent representation. Second, it enables easy taking of measurements like e.g. determining wall thickness or room areas. These properties render our reconstruction method especially beneficial to architects or engineers for planning renovation or retrofitting. Following the idea of previous approaches, the reconstruction task is cast as a labeling problem which is solved by an energy minimization . This global optimization approach allows for the reconstruction of wall elements shared between rooms while simultaneously maintaining plausible connectivity between all wall elements. An automatic prior segmentation of the point clouds into rooms and outside area filters large-scale outliers and yields priors for the definition of labeling costs for the energy minimization. The reconstructed model is further enriched by detected doors and windows. We demonstrate the applicability and reconstruction power of our new approach on a variety of complex real-world datasets requiring little or no parameter adjustment.

Large image deformations pose a challenging problem for the visualization and statistical analysis of 3D image ensembles which have a multitude of applications in biology and medicine. Simple linear interpolation in the tangent space of the ensemble introduces artifactual anatomical structures that hamper the application of targeted visual shape analysis techniques. In this work we make use of the theory of stationary velocity fields to facilitate interactive non-linear image interpolation and plausible extrapolation for high quality rendering of large deformations and devise an efficient image warping method on the GPU. This does not only improve quality of existing visualization techniques, but opens up a field of novel interactive methods for shape ensemble analysis. Taking advantage of the efficient non-linear 3D image warping, we showcase four visualizations: 1) browsing on-the-fly computed group mean shapes to learn about shape differences between specific classes, 2) interactive reformation to investigate complex morphologies in a single view, 3) likelihood volumes to gain a concise overview of variability and 4) streamline visualization to show variation in detail, specifically uncovering its component tangential to a reference surface. Evaluation on a real world dataset shows that the presented method outperforms the state-of-the-art in terms of visual quality while retaining interactive frame rates. A case study with a domain expert was performed in which the novel analysis and visualization methods are applied on standard model structures, namely skull and mandible of different rodents, to investigate and compare influence of phylogeny, diet and geography on shape. The visualizations enable for instance to distinguish (population-)normal and pathological morphology, assist in uncovering correlation to extrinsic factors and potentially support assessment of model quality.

With today's advanced 3D scanner technology, huge amounts of point cloud data can be generated in short amounts of time. Data compression is thus necessary for storage and especially for transmission, e.g., via wireless networks. While previous approaches delivered good compression ratios and interesting theoretical insights, they are either computationally expensive or do not support incrementally acquired data and locally decompressing the data, two requirements we found necessary in many applications. We present a compression approach that is efficient in storage requirements as well as in computational cost, as it can compress and decompress point cloud data in real-time. Furthermore, it is capable of compressing incrementally acquired data, local decompression and of decompressing a subsampled representation of the original data. Our method is based on local 2D parameterizations of surface point cloud data, for which we describe an efficient approach. We suggest the usage of standard image compression techniques for the compression of local details. While exhibiting state-of-the-art compression ratios, our approach remains easy to implement. In our evaluation, we compare our approach to previous ones and discuss the choice of parameters. Due to our algorithm's efficiency, we consider it as a reference concerning speed and compression rates.

We present moment shadow mapping, a novel technique for fast, filtered hard shadows. Like variance shadow mapping it allows for the application of all kinds of efficient texture filtering and antialiasing to its moment shadow map. However it is designed to provide a substantially higher quality. Moment shadow maps store four moments of the depth within the filter kernel. Using this information, our efficient algorithm computes the sharpest possible lower bound as approximation to the shadow intensity. The choice to compute such a bound using four moments is based upon an automated evaluation of thousands of alternatives and thus known to be optimal. To reduce memory and bandwidth requirements we present an optimized quantization scheme to allow 16-bit quantization of moment shadow maps. Our evaluation demonstrates that moment shadow mapping produces high quality results with a single shadow map sample per fragment using 64 bits per shadow map texel.

Transient images help to analyze light transport in scenes. Besides two spatial dimensions, they are resolved in time of flight. Cost-efficient approaches for their capture use amplitude modulated continuous wave lidar systems but typically take more than a minute of capture time. We propose new techniques for measurement and reconstruction of transient images, which drastically reduce this capture time. To this end, we pose the problem of reconstruction as a trigonometric moment problem. A vast body of mathematical literature provides powerful solutions to such problems. In particular, the maximum entropy spectral estimate and the Pisarenko estimate provide two closed-form solutions for reconstruction using continuous densities or sparse distributions, respectively. Both methods can separate m distinct returns using measurements at m modulation frequencies. For m = 3 our experiments with measured data confirm this. Our GPU-accelerated implementation can reconstruct more than 100000 frames of a transient image per second. Additionally, we propose modifications of the capture routine to achieve the required sinusoidal modulation without increasing the capture time. This allows us to capture up to 18.6 transient images per second, leading to transient video. An important byproduct is a method for removal of multipath interference in range imaging.

Realistic visualization of cloth has many applications in computer graphics. An ongoing research problem is how to best represent and capture cloth models, specifically when considering computer aided design of cloth. Previous methods produce highly realistic images, however, they are either difficult to edit or require the measurement of large databases to capture all variations of a cloth sample. We propose a pipeline to reverse engineer cloth and estimate a parametrized cloth model from a single image. We introduce a geometric yarn model, integrating state-of-the-art textile research. We present an automatic analysis approach to estimate yarn paths, yarn widths, their variation and a weave pattern. Several examples demonstrate that we are able to model the appearance of the original cloth sample. Properties derived from the input image give a physically plausible basis that is fully editable using a few intuitive parameters.

Gaining insight into anatomic co variation helps the understanding of organismic shape variability in general and is of particular interest for delimiting morphological modules. Generation of hypotheses on structural co variation is undoubtedly a highly creative process, and as such, requires an exploratory approach. In this work we propose a new local anatomic covariance tensor which enables interactive visualizations to explore co variation at different levels of detail, stimulating rapid formation and (qualitative) evaluation of hypotheses. The effectiveness of the presented approach is demonstrated on a μCT dataset of mouse mandibles for which results from the literature are successfully reproduced, while providing a more detailed representation of co variation compared to state-of-the-art methods.

Ground truth generation for optical flow is hard and costly. Real images can be annotated with ground truth optical flow, but two problems persist: first, no flow measurement techniques for dynamic, large-scale data exist. Second, real-world parameters such as weather or surface wetness cannot be systematically varied. On the other hand, rendered images can have perfect ground truth while light, geometry and materials can be systematically varied. But do they resemble reality well enough to serve as a basis for performance evaluation? In this paper, we take computer graphic realism to an extreme: we compare three optical flow results on reflectance field renderings with supposedly identical real images. A systematic variation of reflectance realism reveals that computer graphics are a valid way to create complex synthetic sequences for optical flow evaluation — but only if modeled carefully.

In this paper, we present a novel, robust multi-view normal field integration technique for reconstructing the full 3D shape of mirroring objects. We employ a turntable-based setup with several cameras and displays. These are used to display illumination patterns which are reflected by the object surface. The pattern information observed in the cameras enables the calculation of individual volumetric normal fields for each combination of camera, display and turntable angle. As the pattern information might be blurred depending on the surface curvature or due to non-perfect mirroring surface characteristics, we locally adapt the decoding to the finest still resolvable pattern resolution. In complex real-world scenarios, the normal fields contain regions without observations due to occlusions and outliers due to interreflections and noise. Therefore, a robust reconstruction using only normal information is challenging. Via a non-parametric clustering of normal hypotheses derived for each point in the scene, we obtain both the most likely local surface normal and a local surface consistency estimate. This information is utilized in an iterative min-cut based variational approach to reconstruct the surface geometry.

Man-made objects such as mechanical construction parts can typically be described as a composition of shape primitives like cylinders, planes, cones and spheres. We propose a robust method for the detection and pose estimation of such objects in 3D point clouds. Our main contribution is to enhance a probabilistic graph-matching approach that detects objects using 3D shape primitives with distinct 2D primitives such as circular contours. With this extension, our method copes with difficult occlusion situations and can be applied for object manipulation in complex scenarios such as grasping from a pile or bin-picking. We demonstrate the performance of our approach in a comparison with a state-of-the-art feature-based method for objects of generic shape and a primitive-based approach using only 3D shapes and no contours.

Grasping individual objects from an unordered pile in a box has been investigated in static scenarios so far. In this paper, we demonstrate bin picking with an anthropomorphic mobile robot. To this end, we extend global navigation techniques by precise local alignment with a transport box. Objects are detected in range images using a shape primitive-based approach. Our approach learns object models from single scans and employs active perception to cope with severe occlusions. Grasps and arm motions are planned in an efficient local multiresolution height map. All components are integrated and evaluated in a bin picking and part delivery task.

Abstract Bidirectional Texture Functions (BTF) have proven to be a well‐suited representation for the reproduction of measured real‐world surface appearance and provide a high degree of realism. We present an approach for designing novel materials by interpolating between several measured BTFs. For this purpose, we transfer concepts from existing texture interpolation methods to the much more complex case of material interpolation. We employ a separation of the BTF into a heightmap and a parallax compensated BTF to cope with problems induced by parallax, masking and shadowing within the material. By working only on the factorized representation of the parallax compensated BTF and the heightmap, it is possible to efficiently perform the material interpolation. By this novel method to mix existing BTFs, we are able to design plausible and realistic intermediate materials for a large range of different opaque material classes. Furthermore, it allows for the synthesis of tileable and seamless BTFs and finally even the generation of gradually changing materials following user specified material distribution maps.

Abstract Visual prototyping of materials is relevant for many computer graphics applications. A large amount of modelling flexibility can be obtained by directly rendering micro‐geometry. While this is possible in principle, it is usually computationally expensive. Recently, bidirectional texture functions (BTFs) have become popular for efficient photorealistic rendering of surfaces. We propose an efficient system for the computation of synthetic BTFs using Monte Carlo path tracing of micro‐geometry. We observe that BTFs usually consist of many similar apparent bidirectional reflectance distribution functions. By exploiting structural similarity we can reduce rendering times by one order of magnitude. This is done in a process we call non‐local image reconstruction, which has been inspired by non‐local means filtering. Our results indicate that synthesizing BTFs is highly practical and may currently only take a few minutes for BTFs with 70 × 70 viewing and lighting directions and 128 × 128 pixels.

Abstract Bidirectional Texture Functions (BTFs) are among the highest quality material representations available today and thus well suited whenever an exact reproduction of the appearance of a material or complete object is required. In recent years, BTFs have started to find application in various industrial settings and there is also a growing interest in the cultural heritage domain. BTFs are usually measured from real‐world samples and easily consist of tens or hundreds of gigabytes. By using data‐driven compression schemes, such as matrix or tensor factorization, a more compact but still faithful representation can be derived. This way, BTFs can be employed for real‐time rendering of photo‐realistic materials on the GPU. However, scenes containing multiple BTFs or even single objects with high‐resolution BTFs easily exceed available GPU memory on today's consumer graphics cards unless quality is drastically reduced by the compression. In this paper, we propose the Bidirectional Sparse Virtual Texture Function, a hierarchical level‐of‐detail approach for the real‐time rendering of large BTFs that requires only a small amount of GPU memory. More importantly, for larger numbers or higher resolutions, the GPU and CPU memory demand grows only marginally and the GPU workload remains constant. For this, we extend the concept of sparse virtual textures by choosing an appropriate prioritization, finding a trade off between factorization components and spatial resolution. Besides GPU memory, the high demand on bandwidth poses a serious limitation for the deployment of conventional BTFs. We show that our proposed representation can be combined with an additional transmission compression and then be employed for streaming the BTF data to the GPU from from local storage media or over the Internet. In combination with the introduced prioritization this allows for the fast visualization of relevant content in the users field of view and a consecutive progressive refinement.

In this paper, we propose a 3D reconstruction approach which combines a structured light based consistency measure with dense normal information obtained by exploiting the Helmholtz reciprocity principle. This combination compensates for the individual limitations of techniques providing normal information, which are mainly affected by low-frequency drift, and those providing positional information, which are often not well-suited to recover fine details. To obtain Helmholtz reciprocal samples, we employ a turntable-based setup. Due to the reciprocity, the structured light directly provides the occlusion information needed during the normal estimation for both the cameras and light sources. We perform the reconstruction by solving one global variational problem which integrates all available measurements simultaneously, over all cameras, light source positions and turntable rotations. For this, we employ an octree-based continuous min-cut framework in order to alleviate metrification errors while maintaining memory efficiency. We evaluate the performance of our algorithm both on synthetic and real-world data.

Abstract In recent years, measuring surface reflectance has become an established method for high quality renderings. In this context, especially non‐parametric representations got a lot of attention as they allow for a very accurate representation of complex reflectance behavior. However, the acquisition of this data is a challenging task especially if complex object geometry is involved. Capturing images of the object under varying illumination and view conditions results in irregular angular samplings of the reflectance function with a limited angular resolution. Classical data‐driven techniques, like tensor factorization, are not well suited for such data sets as they require a resampling of the high dimensional measurement data to a regular grid. This grid has to be on a much higher angular resolution to avoid resampling artifacts which in turn would lead to data sets of enormous size. To overcome these problems we introduce a novel, compact data‐driven representation of reflectance functions based on a sum of separable functions which are fitted directly to the irregular set of data without any further resampling. The representation allows for efficient rendering and is also well suited for GPU applications. By exploiting spatial coherence of the reflectance function over the object a very precise reconstruction even of specular materials becomes possible already with a sparse input sampling. This would be impossible using standard data interpolation techniques. Since our algorithm exclusively operates on the compressed representation, it is both efficient in terms of memory use and computational complexity, depending only sub‐linearly on the size of the fully tabulated data. The quality of the reflectance function is evaluated on synthetic data sets as ground truth as well as on real world measurements.

We present a novel method for retrieval and classification of 3D building models that is tailored to the specific requirements of architects. In contrast to common approaches our algorithm relies on the interior spatial arrangement of rooms instead of exterior geometric shape. We first represent the internal topological building structure by a Room Connectivity Graph (RCG). To enable fast and efficient retrieval and classification with RCGs, we transform the structured graph representation into a vector-based one by introducing a new concept of subgraph embeddings. We provide comprehensive experiments showing that the introduced subgraph embeddings yield superior performance compared to state-of-the-art graph retrieval approaches.

Virtual reproduction of optical material properties is an important task for many computer graphics applications. Recently, bidirectional texture functions (BTFs) [Dana et al. 1999] have become quite popular for photo-realistic rendering of surfaces. BTFs can represent a huge variety of different materials. They can be easily integrated into any modern rendering system which offers programmable surface shaders. BTFs of real materials can be optically measured, they are known to be efficiently compressible and can be used for real-time graphics rendering applications even on the web using WebGL.

Photo-realistic 3D content is crucial for creating convincing digital images. For certain classes of everyday objects, such as food, the human perception is sensitive to even small inconsistencies, making the creation of such content difficult and time-consuming, even for experts. In this sketch, we will explore the use of an automated pipeline for capturing 3D shape and Bidirectional Texture Function of food. The acquired data is used to render photo-realistic images of purely virtual objects under arbitrary lighting and with full global illumination.

Abstract Realistic terrain models are required in many applications, especially in computer games. Commonly, procedural models are applied to generate the corresponding models and let users experience a wide variety of new environments. Existing algorithms generate landscapes immediately with view‐dependent resolution and without preprocessing. Unfortunately, landscapes generated by such algorithms lack river networks and therefore appear unnatural. Algorithms that integrate realistic river networks are computationally expensive and cannot be used to generate a locally adaptive high resolution landscape during a fly‐through. In this paper, we propose a novel algorithm to generate realistic river networks. Our procedural algorithm creates complete planets and landscapes with realistic river networks within seconds. It starts with a coarse base geometry of a planet without further preprocessing and user intervention. By exploiting current graphics hardware, the proposed algorithm is able to generate adaptively refined landscape geometry during fly‐throughs.

Abstract Efficient physically accurate modeling and rendering of woven cloth at a yarn level is an inherently complicated task due to the underlying geometrical and optical complexity. In this paper, a novel and general approach to physically accurate cloth rendering is presented. By using a statistical volumetric model approximating the distribution of yarn fibers, a prohibitively costly explicit geometrical representation is avoided. As a result, accurate rendering of even large pieces of fabrics containing orders of magnitudes more fibers becomes practical without sacrifying much generality compared to fiber‐based techniques. By employing the concept of local visibility and introducing the effective fiber density, limitations of existing volumetric approaches regarding self‐shadowing and fiber density estimation are greatly reduced.

Simple and effective geometric and radiometric calibration of camera devices has enabled the use of consumer digital cameras for HDR photography, for image based measurement and similar applications requiring a deeper understanding about the camera characteristics. However, to date no such practical methods for estimating the spectral response of cameras are available. Existing approaches require costly hardware and controlled acquisition conditions limiting their applicability. Consequently, even though being highly desirable for color correction and color processing purposes as well as for designing image-based measurement or photographic setups, the spectral response of a camera is rarely considered. Our objective is to close this gap. In this work a practical approach for multi-spectral characterization of trichromatic cameras is presented. Taking photographs of a color chart and measuring the average lighting using a spectrophotometer the effective spectral response of a camera can be estimated for a wide range of out-of-lab environments. By comprehensive cross validation experiments we prove that the new method performs well compared to costly reference measurements. Moreover, we show that our technique can also be used to generate ICC profiles with higher accuracy and less constrained capturing conditions compared to state-of-the-art ICC profilers.

Abstract In this paper, we present a novel exemplar‐based technique for the interpolation between two textures that combines patch‐based and statistical approaches. Motivated by the notion of texture as a largely local phenomenon, we warp and blend small image neighborhoods prior to patch‐based texture synthesis. In addition, interpolating and enforcing characteristic image statistics faithfully handles high frequency detail. We are able to create both intermediate textures as well as continuous transitions. In contrast to previous techniques computing a global morphing transformation on the entire input exemplar images, our localized and patch‐based approach allows us to successfully interpolate between textures with considerable differences in feature topology for which no smooth global warping field exists.

Abstract Traditional RGB reflectance and light data suffers from the problem of metamerism and is not suitable for rendering purposes where exact color reproduction under many different lighting conditions is needed. Nowadays many setups for cheap and fast acquisition of RGB or similar trichromatic datasets are available. In contrast to this, multi‐ or even hyper‐spectral measurements require costly hardware and have severe limitations in many cases. In this paper, we present an approach to combine efficiently captured RGB data with spectral data that can be captured with small additional effort for example by scanning a single line of an image using a spectral line‐scanner. Our algorithm can infer spectral reflectances and illumination from such sparse spectral and dense RGB data. Unlike other approaches, our method reaches acceptable perceptual errors with only three channels for the dense data and thus enables further use of highly efficient RGB capture systems. This way, we are able to provide an easier and cheaper way to capture spectral textures, BRDFs and environment maps for the use in spectral rendering systems.

In virtual environments, selection is typically solved by moving a cursor above a virtual item/object and issuing a selection command. In the context of hand-tracking, the cursor movement is controlled by a certain mapping of the hand pose to the virtual cursor position, allowing the cursor to reach any place in the virtual working space. If the virtual working space is bounded, a linear mapping can be used. This is called a proportional control.

Abstract Driven by the advances in lossy compression of bidirectional texture functions (BTFs), there is a growing need for reliable methods to numerically measure the visual quality of the various compressed representations. Based on the CIE ΔE00 colour difference equation and concepts of its spatio‐temporal extension ST‐CIELab for video quality assessment, this paper presents a numerical quality measure for compressed BTF representations. By analysing the BTF in its full six‐dimensional (6D) space, light and view transition effects are integrated into the measure. In addition to the compressed representation, the method only requires the source BTF images as input and thus aids the objective evaluation of different compression techniques by means of a simple numerical comparison. By separating the spatial and angular components of the difference measure and linearizing each of them, the measure can be incorporated into any linear or multi‐linear compression technique. Using a per‐colour‐channel principal component analysis (PCA), compression rates of about 500:1 can be achieved at excellent visual quality.

Abstract We consider the problem of approximating an arbitrary generic surface with a given set of simple surface primitives. In contrast to previous approaches based on variational surface approximation, which are primarily concerned with finding an optimal partitioning of the input geometry, we propose to integrate a model selection step into the algorithm in order to also optimize the type of primitive for each proxy. Our method is a joint global optimization of both the partitioning of the input surface as well as the types and number of used shape proxies. Thus, our method performs an automatic trade‐off between representation complexity and approximation error without relying on a user supplied predetermined number of shape proxies. This way concise surface representations are found that better exploit the full approximative power of the employed primitive types.

Abstract Photo‐realistic reproduction of material appearance from images has widespread use in applications ranging from movies over advertising to virtual prototyping. A common approach to this task is to reconstruct the small scale geometry of the sample and to capture the reflectance properties using spatially varying BRDFs. For this, multi‐view and photometric stereo reconstruction can be used, both of which are limited regarding the amount of either view or light directions and suffer from either low‐ or high‐frequency artifacts, respectively. In this paper, we propose a new algorithm combining both techniques to recover heightfields and spatially varying BRDFs while at the same time overcoming the above mentioned drawbacks. Our main contribution is a novel objective function which allows for the reconstruction of a heightfield and high quality SVBRDF including view dependent effects. Thereby, our method also avoids both low and high frequency artifacts. Additionally, our algorithm takes inter‐reflections into account allowing for the reconstruction of undisturbed representations of the underlying material. In our experiments, including synthetic and real‐world data, we show that our approach is superior to state‐of‐the‐art methods regarding reconstruction error as well as visual impression. Both the reconstructed geometry and the recovered SVBRDF are highly accurate, resulting in a faithful reproduction of the materials characteristic appearance, which is of paramount importance in the context of material rendering.

Abstract In this paper, we present a novel compression technique for Bidirectional Texture Functions based on a sparse tensor decomposition. We apply the K‐SVD algorithm along two different modes of a tensor to decompose it into a small dictionary and two sparse tensors. This representation is very compact, allowing for considerably better compression ratios at the same RMS error than possible with current compression techniques like PCA, N‐mode SVD and Per Cluster Factorization. In contrast to other tensor decomposition based techniques, the use of a sparse representation achieves a rendering performance that is at high compression ratios similar to PCA based methods.

Abstract We consider the problem of reconstruction from incomplete point‐clouds. To find a closed mesh the reconstruction is guided by a set of primitive shapes which has been detected on the input point‐cloud (e.g. planes, cylinders etc.). With this guidance we not only continue the surrounding structure into the holes but also synthesize plausible edges and corners from the primitives' intersections. To this end we give a surface energy functional that incorporates the primitive shapes in a guiding vector field. The discretized functional can be minimized with an efficient graph‐cut algorithm. A novel greedy optimization strategy is proposed to minimize the functional under the constraint that surface parts corresponding to a given primitive must be connected. From the primitive shapes our method can also reconstruct an idealized model that is suitable for use in a CAD system.

Panoramic maps combine the advantages of both ordinary geographic maps and terrestrial images. While inheriting the familiar perspective of terrestrial images, they provide a good overview and avoid occlusion of important geographical features. The designer achieves this by skillful choice and integration of several views in a single image. As important features on the surface must be carefully rearranged to guarantee their visibility, the manual design of panoramic maps requires many hours of tedious and painstaking work. In this article we take a variational approach to the design of panoramic maps. Starting from conventional elevation data and aerial images, our method fully automatically computes panoramic maps from arbitrary viewpoints. It rearranges geographic structures to maximize the visibility of a specified set of features while minimizing the deformation of the landscape's shape.

In this work a practical approach to photometric acquisition of hair color is presented. Based on a single input photograph of a simple setup we are able to extract physically plausible optical properties of hair and to render virtual hair closely matching the original. Our approach does not require any costly special hardware but a standard consumer camera only.

Abstract To assist wayfinding and navigation, the display of maps and driving directions on mobile devices is nowadays commonplace. While existing system can naturally exploit GPS information to facilitate orientation, the inherently limited screen space is often perceived as a drawback compared to traditional street maps as it constrains the perception of contextual information. Moreover, occlusion issues add to this problem if the environment is shown from the popular egocentric perspective. In this paper we describe an interactive visualization system that addresses these problems by reallocating the available screen space. At the heart of our system are three novel visualization techniques: First, we propose a non‐standard perspective that allows to blend between the familiar pedestrian perspective and a standard map depiction with reduced occlusion. Second, we derive an efficient deformation technique that allows an interactive allocation of screen space to areas of interest like e.g. nearby touristic attractions. Finally, a path adaptive isometric perspective is proposed that reveals otherwise hidden facades in top‐down views. We describe efficient implementations of all techniques and exemplify our interactive system on real world urban models.

Abstract State‐of‐the‐art car paint shows not only interesting and subtle angular dependency but also significant spatial variation. Especially in sunlight these variations remain visible even for distances up to a few meters and give the coating a strong impression of depth which cannot be reproduced by a single BRDF model and the kind of procedural noise textures typically used. Instead of explicitly modeling the responsible effect particles we propose to use image‐based reflectance measurements of real paint samples and represent their spatial varying part by Bidirectional Texture Functions (BTF). We use classical BRDF models like Cook‐Torrance to represent the reflection behavior of the base paint and the highly specular finish and demonstrate how the parameters of these models can be derived from the BTF measurements. For rendering, the image‐based spatially varying part is compressed and efficiently synthesized. This paper introduces the first hybrid analytical and image‐based representation for car paint and enables the photo‐realistic rendering of all significant effects of highly complex coatings.

In this work we develop a new alternative to conventional maps for visualization of relatively short paths as they are frequently encountered in hotels, resorts or museums. Our approach is based on a warped rendering of a 3D model of the environment such that the visualized path appears to be straight even though it may contain several junctions. This has the advantage that the beholder of the image gains a realistic impression of the surroundings along the way which makes it easy to retrace the route in practice. We give an intuitive method for generation of such images and present results from user studies undertaken to evaluate the benefit of the warped images for orientation in unknown environments.

Mesh editing methods based on differential surface representations are known for their efficiency and ease of implementation. For reconstruction from such representations, local frames have to be determined which is a nonlinear problem. In linear approximations frames can either degenerate or become inconsistent with the geometry. Both results in contra-intuitive deformations. Existing nonlinear approaches, however, are comparatively slow and considerably more complex. In this paper we present a differential representation that implicitly enforces orthogonal and geometry consistent frames while allowing a simple and efficient implementation. In particular, it enforces conformal surface deformations preserving local texture features.

In order to compute texture atlases with low stretch and hardly visible texture seams existing texture mapping tools pose high demands on the quality of surface representations like consistent orientation, watertightness, or manifoldness which many models commonly used in day-to-day modeling practice fail to meet. In this paper we propose an approach that bridges the gap between requirements of high quality texture mapping tools and poor mesh connectivity of models used in practice. In the spirit of classical two-part mapping an intermediate proxy surface is created that can be processed by high quality texture mapping tools. The texture signal is transferred to the original geometry using a novel mapping technique. Avoiding a modification of the original geometry typical problems of mesh repairing approaches like approximation errors or feature corruption are circumvented. As our method poses almost no demands on connectivity it can also be applied to point clouds. Contrary to classical two- part mapping, the method requires little user-interaction. Its robustness and quality are demonstrated in several examples.

In this paper we present a novel computer vision based handtracking method, which is capable of simultaneously tracking 6+4 degrees of freedom (DOFs) of each human hand in real-time (25 frames per second) with the help of 3 (or more) off-the-shelf consumer cameras. '6+4 DOF' means that the system can track the global pose (6 continuous parameters for translation and rotation) of 4 different gestures. Different studies discovered the need for two-handed interaction to enable an intuitive 3D Human-Computer-Interaction. Previously, using both hands as at least 6 DOF input devices involved the use of either datagloves or markers. Applying our two-hand-tracking we evaluated the use of both hands as input devices for two applications: fly-through exploration of a virtual world and a mesh editing application.

Abstract In this paper we present a novel computer vision based hand‐tracking technique, which is capable of robustly tracking 6+4DOF of the human hand in real‐time (at least 25 frames per second) with the help of 3 (or more) off‐the‐shelf consumer cameras. ‘6+4DOF’ means that the system can track the global pose (6 continuous parameters for translation and rotation) of 4 different gestures. A key feature of our system is its fully automatic real‐time initialization procedure, which, along with a sound tracking‐lost detector, makes the system fit for real‐world applications. Because of this, our method acts as an enabling technology for uncumbersome hand‐based 3D Human‐Computer‐Interaction (HCI). Previously, using the hand as an at least 6DOF input device involved the use of either datagloves or markers. Using our tracking we evaluated the use of the hand as an input device for two prevalent Virtual Reality applications: fly‐through exploration of a virtual world and a simple digital assembly simulation.

Abstract In this paper we present an automatic algorithm to detect basic shapes in unorganized point clouds. The algorithm decomposes the point cloud into a concise, hybrid structure of inherent shapes and a set of remaining points. Each detected shape serves as a proxy for a set of corresponding points. Our method is based on random sampling and detects planes, spheres, cylinders, cones and tori. For models with surfaces composed of these basic shapes only, for example, CAD models, we automatically obtain a representation solely consisting of shape proxies. We demonstrate that the algorithm is robust even in the presence of many outliers and a high degree of noise. The proposed method scales well with respect to the size of the input point cloud and the number and size of the shapes within the data. Even point sets with several millions of samples are robustly decomposed within less than a minute. Moreover, the algorithm is conceptually simple and easy to implement. Application areas include measurement of physical parameters, scan registration, surface compression, hybrid rendering, shape classification, meshing, simplification, approximation and reverse engineering .

Abstract Based on the potential of current programmable GPUs, recently several approaches were developed that use the GPU to calculate deformations of surfaces like the folding of cloth or to convert higher level geometry to renderable primitives like NURBS or subdivision surfaces. These algorithms are realized as a per‐frame operation and take advantage of the parallel processing power of the GPU. Unfortunately, an efficient accurate collision detection, that is necessary for the simulation itself or for the interaction with and editing of the objects, can currently not be integrated seamlessly into these GPU‐based approaches without switching back to the CPU. In this paper we describe a novel GPU‐based collision detection method for deformable parameterized surfaces that can easily be combined with the aforementioned approaches. Representing the individual parameterized surfaces by stenciled geometry images allows to generate GPU‐optimized bounding volume hierarchies in real‐time that serve as a basis for an optimized GPU‐based hierarchical collision detection algorithm. As a test case we applied our algorithm to the collision detection of deformable trimmed NURBS models, which is an important problem in industry. For the trimming and tessellation of the NURBS on the GPU we used a recent approach [GBK05] and combined it with our collision detection algorithm. This way we are able to render and check collisions for deformable models consisting of several thousands of trimmed NURBS patches in real‐time. Categories and Subject Descriptors (according to ACM CCS): I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling—Geometric algorithms, languages, and systems; Splines; I.3.7 [Computer Graphics]: Three‐Dimensional Graphics and Realism—Virtual reality

Abstract Image‐based representations of an object profit from known geometry. The more accurate this geometry is known, the better corresponding pixels in the different images can be aligned, which leads to less artifacts and better compression performance. For opaque objects the per‐pixel data can then be interpreted as a sampling of the BRDF at the respective surface point. In order to parameterize this sampled data a coordinate frame has to be defined. In previous work this coordinate frame was either the global frame or a local frame derived from the base geometry. Both approaches lead to misalignments between sample vectors: Features of basically very similar BRDFs will be shifted to different regions in the sample vector leading to poor compression performance. In order to improve alignment between the sampled BRDFs in image‐based rendering, we propose an optimization algorithm which determines consistent coordinate frames for every sample point on the object surface. This way we efficiently align the features even of anisotropic reflection functions and reconstruct approximate local coordinate frames without performing an explicit 3D‐reconstruction. The optimization is calculated efficiently by exploiting the Fourier‐shift theorem for spherical harmonics. In order to deal with different materials in a scene, the technique is combined with a clustering algorithm. We demonstrate the utility of our method by applying it to BTFs and 6D surface reflectance fields. Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Picture/Image Generation]: Digitizing and scanning I.3.7 [Three‐Dimensional Graphics and Realism]: Color, shading, shadowing, and texture

Abstract One of the main challenges in computer graphics is still the realistic rendering of complex materials such as fabric or skin. The difficulty arises from the complex meso structure and reflectance behavior defining the unique look‐and‐feel of a material. A wide class of such realistic materials can be described as 2D‐texture under varying light‐ and view direction, namely, the Bidirectional Texture Function (BTF). Since an easy and general method for modeling BTFs is not available, current research concentrates on image‐based methods, which rely on measured BTFs (acquired real‐world data) in combination with appropriate synthesis methods. Recent results have shown that this approach greatly improves the visual quality of rendered surfaces and therefore the quality of applications such as virtual prototyping. This state‐of‐the‐art report (STAR) will present the techniques for the main tasks involved in producing photo‐realistic renderings using measured BTFs in details.

As there is no hardware support neither for rendering trimmed NURBS -- the standard surface representation in CAD -- nor for T-Spline surfaces the usability of existing rendering APIs like OpenGL, where a run-time tessellation is performed on the CPU, is limited to simple scenes. Due to the irregular mesh data structures required for trimming no algorithms exists that exploit the GPU for tessellation. Therefore, recent approaches perform a pretessellation and use level-of-detail techniques. In contrast to a simple API these methods require tedious preparation of the models before rendering and hinder interactive editing. Furthermore, due to the tremendous amount of triangle data smooth zoom-ins from long shot to close-up are not possible, In this paper we show how the trimming region can be defined by a trim-texture that is dynamically adapted to the required resolution and allows for an efficient trimming of surfaces on the GPU. Combining this new method with GPU-based tessellation of cubic rational surfaces allows a new rendering algorithm for arbitrary trimmed NURBS and T-Spline surfaces with prescribed error in screen space on the GPU. The performance exceeds current CPU-based techniques by a factor of up to 1000 and makes real-time visualization of real-world trimmed NURBS and T-Spline models possible on consumer-level graphics cards.

In this paper we present a novel integrated 3D editing environment that combines recent advantages in various fields of computer graphics, such as shape modelling, video-based Human Computer Interaction, force feedback and VR fine-manipulation techniques. This integration allows us to create a new compelling form of 3D object creation and manipulation preserving the metaphors designers, artists and painters have accustomed to during their day to day practice. Our system comprises a novel augmented reality workbench and enables users to simultaneously perform natural fine pose determination of the edited object with one hand and model or paint the object with the other hand. The hardware setup features a non-intrusive, video-based hand tracking subsystem, see-through glasses and a 3D 6-degree of freedom input device. The possibilities delivered by our AR workbench enable us to implement traditional and recent editing metaphors in an immersive and fully three-dimensional environment, as well as to develop novel approaches to 3D object interaction.

In this paper, we propose a new method that can consistently orient all normals of any mesh (if at all possible), while ensuring that most polygons are seen with their front-faces from most viewpoints. Our algorithm combines the proximity-based with a new visibility-based approach. Thus, it virtually eliminates the problems of proximity-based approaches, while avoiding the limitations of previous solid-based approaches. Our new method builds a connectivity graph of the patches of the model, which encodes the "proximity" of neighboring patches. In addition, it augments this graph with two visibility coefficients for each patch. Based on this graph, a global consistent orientation of all patches is quickly found by a greedy optimization. We have tested our new method with a large suite of models, many of which from the automotive industry. The results show that almost all models can be oriented consistently and sensibly using our new algorithm

This work presents a novel calibration method for data-gloves with many degrees of freedom. The goal of our method is to establish a mapping from the sensor values of the glove to the joint angles of an articulated hand that is of 'high visual' fidelity. This is in contrast to previous methods that aim at determining the absolute values of the real joint angles with high accuracy. The advantage of our method is that it can be simply carried through without the need for auxiliary calibration hardware (such as cameras), while still producing visually correct mappings. To achieve this, we developed a method that explicitly models the cross-couplings of the abduction sensors with the neighboring flex sensors. The results show that our method performs superior to linear calibration in most cases.

Rendering geometric models with complex surface materials in arbitrary lighting environments is a challenging problem. In order to relight and render geometries covered with complex, measured BTFs two problems have to be addressed: the memory problem resulting from the large size of the measured BTF data and the light integration problem resulting from summing up the contributions from all measured light-sources. In this paper we describe how highly efficient BTF compression methods like local-PCA and suitable representations of environmental light based on spherical harmonics can be combined leading to fast environmental lighting for efficiently encoded BTFs. As a side effect the method supports precomputed radiance transfer

Creating long animation sequences with nontrivial repetitions is a time consuming and often difficult task. This is true for 2D images and even more true for 3D sequences. Based upon the idea of video textures we propose a simple algorithm to create new user controlled animation sequences based only on a few key frames by the analysis of velocity and position coherence. The simplicity of the method is achieved by carrying out the calculations on the main principal components of the reference animation, hence reducing the dimensionality of the input data. This also leads to significant compression. Smooth animations are ensured, using one of the proposed blending schemes.

In the last decade, Fourier volume rendering (FVR) has obtained considerable attention due to its O(N/sup 2/logN) rendering complexity, where O(N/sup 3/) is the volume size. Although ordinary volume rendering has O(N/sup 3/) rendering complexity, it is still preferred over FVR for the main reason, that FVR offers bad localization of spatial structures. As a consequence, it was assumed, that it is hardly possible to apply ID transfer functions, which arbitrarily modify voxel values not only in dependence of the position, but also the voxel value. We show that this assumption is not true for threshold operators. Based on the theory of Fourier series, we derive a FVR method, which is capable of integrating all sample points greater (or alternatively, lower) than an iso-value T during rendering, where T can be modified interactively during the rendering session. We compare our method with other approaches and we show examples on well-known datasets to illustrate the quality of the renderings.

In this paper, we propose the utilization of a kd-tree based hierarchy as an implicit object representation. Compared to an octree, the kd-tree based hierarchy is superior in terms of adaptation to the object surface. In consequence, we obtain considerably more compact implicit representations especially in case of thin object structures. We describe a new isosurface extraction algorithm for this kind of implicit representation. In contrast to related algorithms for octrees, it generates 2-manifold meshes even for kd-trees with cells containing multiple surface components. The algorithm retains all the good properties of the dual contouring approach [10] like feature preservation, computational efficiency, etc. In addition, we present a simplification framework for the surfaces represented by the kd-tree based on quadric error metrics. We adapt this framework to quantify the influence of topological changes, thereby allowing controlled topological simplification of the object. The advantages of the new algorithm are demonstrated by several examples.

We present the concept of volumetric depth-peeling. The proposed method is conceived to render interior and exterior iso-surfaces for a fixed iso-value and to blend them without the need to render the volume multiple times. The main advantage of our method over pre-integrated volume rendering is the ability to extract arbitrarily many iso-layers for the given iso-value. Up to now, pre-integrated volume rendering is only capable of visualizing the nearest two (front and back-faced) iso-surfaces. A further gain of our algorithm is the rendering speed, since it does not depend on the number of layers to be extracted, as for previous depth-peeling methods. We rather exploit the natural slicing order of 3D texturing to circumvent the handicap of storing intermediate layers in textures, as done in polygonal-based depth-peeling approaches. We are further capable of rapidly previewing the volume data, when only few context information about the concerning dataset is available. An important example of use in the area of non-photorealistic rendering is given, where we can distinguish between visible and hidden silhouettes, which are important elements in stylization. By using standard OpenGL extensions, we allow the exploration of spatial relationships in the volume -at interactive rates- in hardware.

Abstract A Texture Atlas is a two dimensional representation of a 3D model usable for paint systems or as a sewing pattern.The field of texture atlas generation from polygonal models has been well exploited in the recent years. The developedalgorithms work on piecewise linear surface representations, but not on parametric surfaces like NURBS,that are still the main surface representation in CAD systems. If a texture atlas is generated from a triangulatedNURBS model, the result cannot be edited further in a CAD system, since the separation into charts is not basedon the separate NURBS patches of the original model. We present a method for automatic generation of a textureatlas directly from trimmed NURBS models, while preserving the original NURBS representation. The resultingtexture atlas is build of several charts, each consisting of the original NURBS patches sewn together. Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Picture/Image Generation;I.3.5 [Computer Graphics]: Computational Geometry and Object Modeling; I.3.7 [Computer Graphics]: Three‐DimensionalGraphics and Realism — Color, Shading and Texture; J.6 [Computer‐aided Engineering]: Computer‐aideddesign (CAD)

The problem of rendering large trimmed NURBS models at interactive frame rates is of great interest for industry, since nearly all their models are designed on the basis of this surface type. Most existing approaches first transform the NURBS surfaces into polygonal representation and subsequently build static levels of detail upon them, as current graphics hardware is optimized for rendering triangles. lit this work, we present a method for memory efficient, view-dependent rendering of trimmed NURBS surfaces that yields high-quality results at interactive frame rates. In contrast to existing algorithms, our approach needs not store hierarchies of triangles, since utilizing our special multiresolution seam graph data structure, we are able to generate required triangulations on the fly.

Along with the development of 3D acquisition devices and methods and the increasing number of available 3D objects, new tools are necessary to automatically analyze, search and interpret these models. In this paper, we describe a novel geometric approach to 3D object comparison and analysis. To compare two objects geometrically, we first properly position and align the objects. After solving this pose estimation problem, we generate specific distance histograms that define a measure of the geometric similarity of the inspected objects. The geometric approach is very well-suited for structures with moderate variance, e.g. bones, fruits, etc. The strength of the approach is proven through the results of several tests that we performed on different data sets.

In this paper we consider the problem of reconstructing triangular surfaces from given contours. An algorithm solving this problem has to decide which contours of two successive slices should be connected by the surface (branching problem), and, given that, which vertices of the assigned contours should be connected for the triangular mesh (correspondence problem). We present a new approach that solves both tasks in an elegant way. The main idea is to employ discrete distance fields enhanced with correspondence information. This allows us not only to connect vertices from successive slices in a reasonable way but also to solve the branching problem by creating intermediate contours where adjacent contours differ too much. Last but not least we show how the 2D-distance fields used in the reconstruction step can be converted to a 3D-distance field that can be advantageously exploited for distance calculations during a subsequent simplification step.

State of the art multiresolution modeling allows to selectively refine a coarse mesh of an object on the visually important parts. In this way it is possible to render the geometry of a given object accurately with a minimum number of triangles. There are also approaches that use cones of normal vectors to perform illumination dependent selective refinement. This includes the correct rendering of highlights and transitions between lit and unlit areas of an object. The drawback of these approaches is that using only the normal cone for the detection gives only a very poor criteria to detect the problematic cases. Therefore, too much refinement has to be performed which in most cases prevents real time update rates between successive frames. The authors describe a new approach for better estimation of normal deviations between different levels of detail. This allows for accurate lighting with a minimum number of triangles.

A view-dependent multiresolution triangulation algorithm is presented for a real-time fly-through. The triangulation of the terrain is generated incrementally on-the-fly during the rendering time. We show that since the view changes smoothly, only a few incremental modifications are required to update the triangulation to a new view. The resulting triangles form a multiresolution Delaunay triangulation which satisfies a predetermined view-dependent error tolerance. The presented method provides a guaranteed-quality mesh since it has control over the global geometric approximation error of the multiresolution view-dependent triangulation. © 1998 John Wiley & Sons, Ltd.

A view-dependent multiresolution triangulation algorithm is presented for a real-time flythrough. The triangulation of the terrain is generated incrementally on-the-fly during the rendering time. We show that since the view changes smoothly only a few incremental modifications are required to update the triangulation to a new view. The resulting triangles form a multiresolution Delaunay triangulation which satisfies a predetermined view-dependent error tolerance. The presented method provides a guaranteed quality mesh since it has control over the global geometric approximation error of the multiresolution view-dependent triangulation.

In many cases the surfaces of geometric models consist of a large number of triangles. Several algorithms were developed to reduce the number of triangles required to approximate such objects. Algorithms that measure the deviation between the approximated object and the original object are only available for special cases. We use the Hausdorff distance between the original and the simplified mesh as a geometrically meaningful error value which can be applied to arbitrary triangle meshes. We present a new algorithm to reduce the number of triangles of a mesh without exceeding a user defined Hausdorff distance between the original and simplified mesh. As this distance is parameterization independent, its use as error measure is superior to the use of the L/sup /spl infin//-Norm between parameterized surfaces. Furthermore the Hausdorff distance is always less than the distance induced by the L/sup /spl infin//-Norm. This results in higher reduction rates. Excellent results were achieved by the new decimation algorithm for triangle meshes that has been used in different application areas such as volume rendering, terrain modeling and the approximations of parameterized surfaces. The key advantages of the new algorithm are: it guarantees a user defined position dependent approximation error; it allows one to generate a hierarchical geometric representation in a canonical way; it automatically preserves sharp edges.