Abstract We present a novel approach to create plausible 3D face models from vague recollections or incomplete descriptions. This task plays an important role in police work, where composite facial images of suspects need to be created from vague descriptions given by the eyewitnesses of an incident. Our approach is based on a morphable model of 3D faces and takes into account correlations among facial features based on human anatomy and ethnicity. Using these correlations, unspecified parts of the target face are automatically completed to yield a coherent face model. The system uses a novel paradigm for navigating face space and provides high‐level control of facial attributes as well as the possibility to import facial features from a database. In addition, the user can specify a set of attribute constraints that are used to restrict the target face to a residual subspace. These constraints can also be enforced on the example faces in the database, bringing their appearance closer to the mental image of the user, and thus avoiding confusing exposure to entirely different faces. We also propose a novel approach for adapting the system to local populations based on additional image databases that are converted into our 3D representation by automated shape reconstruction. We demonstrate the applicability of our system in a simulated forensic scenario and compare our results with those obtained by a professional forensic artist using state‐of‐the‐art software for creating composite images in police work. Categories and Subject Descriptors (according to ACM CCS): I.3.6 [Computer Graphics]: Methodology and Techniques—Interaction techniques I.4.10 [Image Processing and Computer Vision]: Image Representation—Hierarchical, Multidimensional, Statistical J.m [Computer Applications]: Miscellaneous—Forensic Sciences

We present a physically-based approach to compute the colors of the sky during the twilight period before sunrise and after sunset. The simulation is based on the theory of light scattering by small particles. A realistic atmosphere model is assumed, consisting of air molecules, aerosols, and water. Air density, aerosols, and relative humidity vary with altitude. In addition, the aerosol component varies in composition and particle-size distribution. This allows us to realistically simulate twilight phenomena for a wide range of different climate conditions. Besides considering multiple Rayleigh and Mie scattering, we take into account wavelength-dependent refraction of direct sunlight as well as the shadow of the Earth. Incorporating several optimizations into the radiative transfer simulation, a photo-realistic hemispherical twilight sky is computed in less than two hours on a conventional PC. The resulting radiometric data is useful, for instance, for high-dynamic range environment mapping, outdoor global illumination calculations, mesopic vision research and optical aerosol load probing.

We present a hair model together with rendering algorithms suitable for real-time rendering. In our approach, we take into account the major lighting factors contributing to a realistic appearance of human hair: anisotropic reflection and self-shadowing. To deal with the geometric complexity of human hair, we combine single hair fibers into hair wisps, which are represented by textured triangle strips. Our rendering algorithms use OpenGL extensions to achieve real-time performance on recent commodity graphics boards. We demonstrate the applicability of our hair model for a variety of different hairstyles.

Athletes and coaches in most professional sports make use of high-tech equipment to analyze and, subsequently, improve the athlete's performance. High-speed video cameras are employed, for instance, to record the swing of a golf club or a tennis racket, the movement of the feet while running, and the body motion in apparatus gymnastics. High-tech and high-speed equipment, however, usually implies high-cost as well. In this paper, we present a passive optical approach to capture high-speed motion using multi-exposure images obtained with low-cost commodity still cameras and a stroboscope. The recorded motion remains completely undisturbed by the motion capture process. We apply our approach to capture the motion of hand and ball for a variety of baseball pitches and present algorithms to automatically track the position, velocity, rotation axis, and spin of the ball along its trajectory. To demonstrate the validity of our setup and algorithms, we analyze the consistency of our measurements with a physically based model that predicts the trajectory of a spinning baseball. Our approach can be applied to capture a wide variety of other high-speed objects and activities such as golfing, bowling, or tennis for visualization as well as analysis purposes.

We present a new method for the restoration of digitized photographs. Restoration refers to removal of image defects such as scratches and blotches as well as to removal of disturbing objects as, for instance, subtitles, logos, wires, and microphones. Our method combines techniques from texture synthesis and image inpainting, bridging the gap between these two approaches that have recently attracted strong research interest. Combining image inpainting and texture synthesis in a multiresolution approach gives us the best of both worlds and enables us to overcome the limitations of each of those individual approaches. The restored images obtained with our method look plausible in general and surprisingly good in some cases. This is demonstrated for a variety of input images that exhibit different kinds of defects.

Facial reconstruction for postmortem identification of humans from their skeletal remains is a challenging and fascinating part of forensic art. The former look of a face can be approximated by predicting and modeling the layers of tissue on the skull. This work is as of today carried out solely by physical sculpting with clay, where experienced artists invest up to hundreds of hours to craft a reconstructed face model. Remarkably, one of the most popular tissue reconstruction methods bears many resemblances with surface fitting techniques used in computer graphics, thus suggesting the possibility of a transfer of the manual approach to the computer. In this paper, we present a facial reconstruction approach that fits an anatomy-based virtual head model, incorporating skin and muscles, to a scanned skull using statistical data on skull / tissue relationships. The approach has many advantages over the traditional process: a reconstruction can be completed in about an hour from acquired skull data; also, variations such as a slender or a more obese build of the modeled individual are easily created. Last not least, by matching not only skin geometry but also virtual muscle layers, an animatable head model is generated that can be used to form facial expressions beyond the neutral face typically used in physical reconstructions.

We introduce a facial animation system that produces real-time animation sequences including speech synchronization and non-verbal speech-related facial expressions from plain text input. A state-of-the-art text-to-speech synthesis component performs linguistic analysis of the text input and creates a speech signal from phonetic and intonation information. The phonetic transcription is additionally used to drive a speech synchronization method for the physically based facial animation. Further high-level information from the linguistic analysis such as different types of accents or pauses as well as the type of the sentence is used to generate non-verbal speech-related facial expressions such as movement of head, eyes, and eyebrows or voluntary eye blinks. Moreover, emotions are translated into XML markup that triggers emotional facial expressions.

We present a method to efficiently construct and render a smooth surface for approximation of large functional scattered data. Using a subdivision surface framework and techniques from terrain rendering, the resulting surface can be explored from any viewpoint while maintaining high surface fairness and interactive frame rates. We show the approximation error to be sufficiently small for several large data sets. Our system allows for adaptive simplification and provides continuous levels of detail, taking into account the local variation and distribution of the data. Categories and Subject Descriptors (according to ACM CCS): G.1.2 [Approximation]: Approximation of surfaces, Least squares approximation, Piecewise polynomial approximation; I.3.3 [Picture/Image Generation]: Display algorithms, Viewing algorithms; I.3.5 [Computational Geometry and Object Modeling]: Surface representation.

We present a method to adaptively refine an irregular triangle mesh as it deforms in real-time. The method increases surface smoothness in regions of high deformation by splitting triangles in a fashion similar to one or two steps of loop subdivision. The refinement is computed for an arbitrary triangle mesh and the subdivided triangles are simply passed to the rendering engine, leaving the mesh itself unchanged. The algorithm can thus be easily plugged into existing systems to enhance the visual appearance of animated meshes. The refinement step has a very low computational overhead and is easy to implement. We demonstrate the use of the algorithm in our physics-based facial animation system.

Presents an efficient method to automatically compute a smooth approximation of large functional scattered data sets given over arbitrarily shaped planar domains. Our approach is based on the construction of a C/sup 1/-continuous bivariate cubic spline and our method offers optimal approximation order. Both local variation and nonuniform distribution of the data are taken into account by using local polynomial least squares approximations of varying degree. Since we only need to solve small linear systems and no triangulation of the scattered data points is required, the overall complexity of the algorithm is linear in the total number of points. Numerical examples dealing with several real-world scattered data sets with up to millions of points demonstrate the efficiency of our method. The resulting spline surface is of high visual quality and can be efficiently evaluated for rendering and modeling. In our implementation we achieve real-time frame rates for typical fly-through sequences and interactive frame rates for recomputing and rendering a locally modified spline surface.

One of the basic difficulties with interactive walkthroughs is the high quality rendering of object surfaces with non-diffuse light scattering characteristics. Since full ray tracing at interactive rates is usually impossible, we render a precomputed global illumination solution using graphics hardware and use remaining computational power to correct the appearance of non-diffuse objects on-the-fly. The question arises, how to obtain the best image quality as perceived by a human observer within a limited amount of time for each frame. We address this problem by enforcing corrective computation for those non-diffuse objects that are selected using a computational model of visual attention. We consider both the saliency- and task-driven selection of those objects and benefit from the fact that shading artifacts of “unattended” objects are likely to remain unnoticed. We use a hierarchical image-space sampling scheme to control ray tracing and splat the generated point samples. The resulting image converges progressively to a ray traced solution if the viewing parameters remain unchanged. Moreover, we use a sample cache to enhance visual appearance if the time budget for correction has been too low for some frame. We check the validity of the cached samples using a novel criterion suited for non-diffuse surfaces and reproject valid samples into the current view.

Presents a modification of the well-known LBG (Linde, Buzo and Gray, 1980) algorithm for the generation of codebooks in vector quantization. Our algorithm, which we call the ILBG (iterated LBG) algorithm, reduces the codebook error of the LBG algorithm drastically in typical applications. In our experiments, we were able to achieve up to a 75% reduction of the codebook error in only a few additional iteration steps. In the context of lossy image compression, this error reduction in turn leads to an increase of 2-3 dB in the peak signal-to-noise ratio (PSNR).

Using well-adapted object hierarchies can support the rendering of large scenes in different ways. For instance, the quality of the lighting simulation may be improved, or the computational cost for rendering may be reduced. However the meaning of "well-adapted" depends heavily on the criterion due to which the hierarchy has been constructed. Different applications typically have different demands like low average intersection cost for a ray tracer or grouping objects with similar material properties or surface orientation for hierarchical radiosity. The authors propose an algorithm for the automatic creation of object hierarchies. The hierarchies are constructed by sequentially inserting all scene objects into the hierarchy created so far. By basing the insertion decision on a cost function defined by the user, the method can be guided to create hierarchies tailored to the desired application. The results can be improved significantly by running a global optimization on the completed hierarchy. During this optimization step, we perform a re-grouping of the objects in the hierarchy. Any ill-formed groups that were created during the initial algorithm are subject to being eliminated by our global optimization.