The creation of a 3D model is only the first stage of the 3D character animation pipeline. Once a model has been created, and before it can be animated, it must be rigged . Manual rigging is laborious, and automatic rigging approaches are far from real-time and do not allow for incremental updates. This is a hindrance in the real world, where the shape of a model is often revised after rigging has been performed. In this paper, we introduce algorithms and a user-interface for sketch-based 3D modeling that unify the modeling and rigging stages of the 3D character animation pipeline. Our algorithms create a rig for each sketched part in real-time, and update the rig as parts are merged or cut. As a result, users can freely pose and animate their shapes and characters while rapidly iterating on the base shape. The rigs are compatible with the state-of-the-art character animation pipeline; they consist of a low-dimensional skeleton along with skin weights identifying the surface with bones of the skeleton.

Abstract In the style of binary shading, shape and illumination are depicted using two colours, typically black and white, which form coherent lines and regions in the image. We formulate the problem of assigning colours in the rendered image as an energy minimization, computed using graph cut on the image grid. The terms of this energy come from two sources: appearance (shading) and geometry (depth and curvature). Our contributions are in the use of geometric information in determining colours, and how this information is incorporated into a graph cut approach. This optimization yields boundaries between black and white regions that tend towards being shorter and to run along geometric features like creases. We show a range of results, and demonstrate that this approach produces more coherent images than simpler approaches that make local decisions when assigning colours, or that do not use geometry.

This paper investigates the ability of sparse line drawings to depict 3D shape. We perform a study in which people are shown an image of one of twelve 3D objects depicted with one of six styles and asked to orient a gauge to coincide with the surface normal at many positions on the object's surface. The normal estimates are compared with each other and with ground truth data provided by a registered 3D surface model to analyze accuracy and precision. The paper describes the design decisions made in collecting a large data set (275,000 gauge measurements) and provides analysis to answer questions about how well people interpret shapes from drawings. Our findings suggest that people interpret certain shapes almost as well from a line drawing as from a shaded image, that current computer graphics line drawing techniques can effectively depict shape and even match the effectiveness of artist's drawings, and that errors in depiction are often localized and can be traced to particular properties of the lines used. The data collected for this study will become a publicly available resource for further studies of this type.

It's often important to analyze shapes as made up of parts. But there are two ways to think of how parts fit together. We can characterize the remainder of a shape after apart is removed; here we want to cut the shape so what remains has the simplest possible structure. Alternatively, we can cut out the part so that the part itself takes on a simple shape. These cuts do not directly give rise to a segmentation of the shape; a point inside the shape may associate with the part, the remainder, neither, or both. We present a new model for reconstructing these cuts based on the differential geometry of smoothed local symmetries. The model takes into account relatability (which characterizes clean cuts) to determine part boundaries. Our approach complements and unifies existing work on part- based segmentation of shape, and can be used to construct interesting simplifications of shapes.

Summary form only given. The last few years have brought a more concrete understanding of the mathematical relationship between strokes in drawings, linear features in images, and lines on 3D shapes. However, fundamental questions remain unanswered about how our perceptual system resolves these lines as giving evidence about shape. These questions need to be addressed to assemble mathematically defined lines into clear and compelling drawings. I'll discuss my ongoing research on suggestive contours and highlight lines from this perspective.

We present an interactive method for cropping photographs given minimal information about important content location, provided by eye tracking. Cropping is formulated in a general optimization framework that facilitates adding new composition rules, and adapting the system to particular applications. Our system uses fixation data</ to identify important image content and compute the best crop for any given aspect ratio or size, enabling applications such as automatic snapshot recomposition, adaptive documents, and thumbnailing. We validate our approach with studies in which users compare our crops to ones produced by hand and by a completely automatic approach. Experiments show that viewers prefer our gaze-based crops to uncropped images and fully automatic crops.

In fields ranging from technical illustration to mapmaking, artists have developed distinctive visual styles designed to convey both detail and overall shape as clearly as possible. We investigate a non-photorealistic shading model, inspired by techniques for carto-graphic terrain relief, based on dynamically adjusting the effective light position for different areas of the surface. It reveals detail regardless of surface orientation and, by operating at multiple scales, is designed to convey detail at all frequencies simultaneously.

Renderings of volumetric data have become an important data analysis tool for applications ranging from medicine to scientific simulation. We propose a volumetric drawing system that directly extracts sparse linear features, such as silhouettes and suggestive contours, using a temporally coherent seed-and-traverse framework. In contrast to previous methods based on isosurfaces or nonrefractive transparency, producing these drawings requires examining an asymptotically smaller subset of the data, leading to efficiency on large data sets. In addition, the resulting imagery is often more comprehensible than standard rendering styles, since it focuses attention on important features in the data. We test our algorithms on datasets up to 512 3 , demonstrating interactive extraction and rendering of line drawings in a variety of drawing styles.

Characterizing the location and extent of a viewer's interest, in terms of eye movement recordings, informs a range of investigations in image and scene viewing. We present an automatic data-driven method for accomplishing this, which clusters visual point-of-regard (POR) measurements into gazes and regions-of-interest using the mean shift procedure. Clusters produced using this method form a structured representation of viewer interest, and at the same time are replicable and not heavily influenced by noise or outliers. Thus, they are useful in answering fine-grained questions about where and how a viewer examined an image.

Abstract People highlight the intended interpretation of their utterances within a larger discourse by a diverse set of non‐verbal signals. These signals represent a key challenge for animated conversational agents because they are pervasive, variable, and need to be coordinated judiciously in an effective contribution to conversation. In this paper, we describe a freely available cross‐platform real‐time facial animation system,RUTH, that animates such high‐level signals in synchrony with speech and lip movements.RUTHadopts an open, layered architecture in which fine‐grained features of the animation can be derived by rule from inferred linguistic structure, allowing us to useRUTH, in conjunction with annotation of observed discourse, to investigate the meaningful high‐level elements of conversational facial movement for American English speakers. Copyright © 2004 John Wiley & Sons, Ltd.

We describe a method for using a database of recorded speech and captured motion to create an animated conversational character. People's utterances are composed of short, clearly-delimited phrases; in each phrase, gesture and speech go together meaningfully and synchronize at a common point of maximum emphasis. We develop tools for collecting and managing performance data that exploit this structure. The tools help create scripts for performers, help annotate and segment performance data, and structure specific messages for characters to use within application contexts. Our animations then reproduce this structure. They recombine motion samples with new speech samples to recreate coherent phrases, and blend segments of speech and motion together phrase-by-phrase into extended utterances. By framing problems for utterance generation and synthesis so that they can draw closely on a talented performance, our techniques support the rapid construction of animated characters with rich and appropriate expression.

Templates are a widespread natural language technology that achieves believability within a narrow range of interaction and coverage. We consider templates for embodied conversational behavior. Such templates combine a specific pattern of marked-up text, specifying prosody and conversational signals as well as words, with similarly annotated gaps that can be filled in by rule to yield a coherent contribution to a dialogue with a user. In this paper we argue that templates can give a designer substantial freedom to realize specific combinations of behaviors in interactions with users and thereby to explore the relationships among such factors as emotion, personality, individuality and social role.

In this paper, we describe a non-photorealistic rendering system that conveys shape using lines. We go beyond contours and creases by developing a new type of line to draw: the suggestive contour . Suggestive contours are lines drawn on clearly visible parts of the surface, where a true contour would first appear with a minimal change in viewpoint. We provide two methods for calculating suggestive contours, including an algorithm that finds the zero crossings of the radial curvature. We show that suggestive contours can be drawn consistently with true contours, because they anticipate and extend them. We present a variety of results, arguing that these images convey shape more effectively than contour alone.

People highlight the intended interpretation of their utterances within a larger discourse by a diverse set of nonverbal signals. These signals represent a key challenge for animated conversational agents because they are pervasive, variable, and need to be coordinated judiciously in an effective contribution to conversation. In this paper we describe a freely-available cross-platform real-time facial animation system, RUTH, that animates such high-level signals in synchrony with speech and lip movements. RUTH adopts an open, layered architecture in which fine-grained features of the animation can be derived by rule from inferred linguistic structure, allowing us to use RUTH, in conjunction with annotation of observed discourse, to investigate the meaningful high-level elements of conversational facial movement for American English speakers.

We present a method for estimating the shape of a deformable model using the least-squares residuals from a model-based optical flow computation. This method is built on top of an estimation framework using optical flow and image features, where optical flow affects only the motion parameters of the model. Using the results of this computation, our new method adjusts all of the parameters so that the residuals from the flow computation are minimized. We present face tracking experiments that demonstrate that this method obtains a better estimate of shape compared to related frameworks.

Good information design depends on clarifying the meaningful structure in an image. We describe a computational approach to stylizing and abstracting photographs that explicitly responds to this design goal. Our system transforms images into a line-drawing style using bold edges and large regions of constant color. To do this, it represents images as a hierarchical structure of parts and boundaries computed using state-of-the-art computer vision. Our system identifies the meaningful elements of this structure using a model of human perception and a record of a user's eye movements in looking at the photo; the system renders a new image using transformations that preserve and highlight these visual elements. Our method thus represents a new alternative for non-photorealistic rendering both in its visual style, in its approach to visual form, and in its techniques for interaction.

In this paper we show how the use of hard constraints in solving estimation problems, by allowing multiple sources of information to be taken into account during optimization, increases robustness and improves efficiency over alternative methods such as the statistical combination of separate optimization results. Our argument is based on an empirical evaluation of the technique which uses a model-based optical flow constraint in a deformable model framework for tracking a face. The flow constraint makes the model-to-edge alignment optimization problem easier by projecting away the portion of the search space that optical flow makes unlikely, while a Kalman filter is used to reconcile hard constraints with the uncertainty in the optical flow data. Using these hard constraints, the system converges more quickly at each iteration and avoids local minima in solutions that cause other methods to lose track. We conjecture that this use of constraints will be effective in any integration application where there are disparities in the difficulty of computational problems associated with the we of different information sources.

We present a novel method for the shape and motion estimation of a deformable model using error residuals from model-based motion analysis. The motion of the model is first estimated using a model-based least squares method. Using the residuals from the least squares solution, the non-rigid structure of the model can be better estimated by computing how changes in the shape of the model affect its motion parameterization. This method is implemented as a component in a deformable model-based framework that uses optical flow information and edges. This general model-based framework is applied to human face shape and motion estimation. We present experiments that demonstrate that this framework is a considerable improvement over a framework that uses only optical flow information and edges.

We describe a system that automatically generates varied geometric models of human faces. A collection of random measurements of the face is generated according to anthropometric statistics for likely face measurements in a population. These measurements are then treated as constraints on a parameterized surface. Variational modeling is used to find a smooth surface that satisfies these constraints while using a prototype shape as a reference.

This paper describes a framework for the estimation of shape from sparse or incomplete range data. It uses a shape representation called blending, which allows for the geometric combination of shapes into a unified model - selected regions of the component shapes are cut-out and glued together. Estimation of shape by this representation is realized using a physics-based framework, and it also includes a process for deciding how to adapt the structure and topology of the model to improve the fit. The blending representation helps avoid abrupt changes in model geometry during fitting by allowing the smooth evolution of the shape, which improves the robustness of the technique. We demonstrate this framework with a series of experiments showing its ability to automatically extract structured representations from range data given both structurally and topologically complex objects.

We present a formal methodology for the integration of optical flow and deformable models. The optical flow constraint equation provides a non-holonomic constraint on the motion of the deformable model. In this augmented system, forces computed from edges and optical flow are used simultaneously. When this dynamic system is solved, a model-based least-squares solution for the optical flow is obtained and improved estimation results are achieved. The use of a 3-D model reduces or eliminates problems associated with optical flow computation. This approach instantiates a general methodology for treating visual cues as constraints on deformable models. We apply this framework to human face shape and motion estimation. Our 3-D deformable face model uses a small number of parameters to describe a rich variety of face shapes and facial expressions. We present experiments in extracting the shape and motion of a face from image sequences.

We describe a model-based approach to human face shape and motion estimation using a deformable model framework in which optical flow information has been integrated. Our 3-D deformable face model uses a small number of parameters to describe a rich variety of face shapes and facial expressions. A detailed face model allows for the handling of self-occlusion and the correction of error accumulation in tracking by aligning the expected location of face features with those found in the input images. We present experiments in which we employ Kalman filtering to estimate the shape and motion of a face from image sequences of four subjects. The face is successfully tracked even in the presence of large head rotations.

This paper develops a new class of parameterized models based on the linear interpolation of two parameterized shapes along their main axes, using a blending function. This blending function specifies the relative contribution of each component shape on the resulting blended shape. The resulting blended shape can have aspects of each of the component shapes. Using a small number of additional parameters, blending extends the coverage of shape primitives while also providing abstraction of shape. In particular, it offers the ability to construct shapes whose genus can change. Blended models are incorporated into a physics-based shape estimation framework which uses dynamic deformable models. Finally, we present experiments involving the extraction of complex shapes from range data including examples of dynamic genus change.

We propose a shape representation scheme which allows two shapes to be combined into a single model. The desired regions of the two shapes are selected, and then merged together forming a blended shape. For reconstruction, blending is incorporated into a deformable model framework. The model automatically adapts to the data, blending when necessary. Hierarchical blending allows multiple blends of a shape to occur forming an evolution from the initial shape of a sphere to the final shape. Blending also allows the insertion of a hole between arbitrary locations. The models used are globally defined, making the recovered shape a natural symbolic description. We present reconstruction experiments involving shapes of various topologies.>

We develop a new class of parameterized models based on the linear interpolation of two parameterized shapes using a blending function. Using a small number of additional parameters, blending extends the coverage of shape primitives. In particular, it offers the ability to construct shapes whose genus can change. Blended models are incorporated into a physics-based shape estimation framework which uses dynamic deformable models with global and local deformations. We present experiments involving the extraction of complex shapes, including an example of dynamic genus change.>