We describe a novel computer vision application: vision-based human sensing for a Smart Kiosk interface. A Smart Kiosk is a free-standing information dispensing computer appliance capable of engaging in public interactions with multiple people. Vision sensing is a critical component of the kiosk interface, where it is used to determine the context for the interaction. We present a taxonomy of vision problems for a kiosk interface and describe a prototype kiosk which uses color stereo tracking and graphical output to interact with several users.

The goal of this work is to use phonetic recognition to drive a synthetic image with speech. Phonetic units are identified by the phonetic recognition engine and mapped to mouth gestures, known as visemes, the visual counterpart of phonemes. The acoustic waveform and visemes are then sent to a synthetic image player, called FaceMe! where they are rendered synchronously. This paper provides background for the core technologies involved in this process and describes asynchronous and synchronous prototypes of a combined phonetic recognition/FaceMe! system which we use to render mouth gestures on an animated face. 1. Introduction This paper addresses the problem of driving an animated face using audio data. We present a phonetic recognition system as a front-end process to generate visemes, the visual analog of phonemes. The paper provides background for the core phonetic recognition technology, based on Statistical Trajectory Modeling [1] and the core FaceMe! animation technology. It th...

Panel OverviewFacial animation is now attracting more attention than ever before in its 25 years as an identifiable area of computer graphics.Imaginative applications of animated graphical faces are found in sophisticated human-computer interfaces, interactive games, multimedia titles, VR telepresence experiences, and, as always, in a broad variety of production animations.Graphics technologies underlying facial animation now run the gamut from keyframing to image morphing, video tracking, geometric and physical modeling, and behavioral animation.Supporting technologies include speech synthesis and artificial intelligence.Whether the goal is to synthesize realistic faces or fantastic ones, representing the dynamic facial likeness of humans and other creatures is giving impetus to a diverse and rapidly growing body of cross-disciplinary research.The panel will present a historical perspective, assess the state of the art, and speculate on the exciting future of facial animation.

We investigated how cooperation with a computer agent was affected by the agent's pictorial realism, humanlikeness, and likability.Participants played a social dilemma game with a talking computer agent that resembled a person, a dog, or a cartoon dog, or with a confederate interacting through a video link.Participants cooperated highly with the person computer agent and with the confederate.They loved the dog and dog cartoon agents, but (excepting dog owners), significantly less with the dog agents, questionnaire results suggest likability than respect in prompting cooperation agent.

People behave differently in the presence of other people than they do when they are alone. People also may behave differently when designers introduce more human-like qualities into computer interfaces. In an experimental study we demonstrate that people's responses to a talking-face interface differ from their responses to a text-display interface. They attribute some personality traits to it; they are more aroused by it; they present themselves in a more positive light. We use theories of person perception, social facilitation, and self-presentation to predict and interpret these results. We suggest that as computer interfaces become more "human-like," people who use those interfaces may change their own personas in response to them.

A major unsolved problem in computer graphics is the construction and animation of realistic human facial models. Traditionally, facial models have been built painstakingly by manual digitization and animated by ad hoc parametrically controlled facial mesh deformations or kinematic approximation of muscle actions. Fortunately, animators are now able to digitize facial geometries through the use of scanning range sensors and animate them through the dynamic simulation of facial tissues and muscles. However, these techniques require considerableuser input to construct facial models of individuals suitable for animation. In this paper, we present a methodology for automating this challenging task. Starting with a structured facial mesh, we develop algorithms that automatically construct functional models of the heads of human subjects from laser-scanned range and reflectance data. These algorithms automatically insert contractile muscles at anatomically correct positions within a dynamic skin model and root them in an estimated skull structure with a hinged jaw. They also synthesize functional eyes, eyelids, teeth, and a neck and fit them to the final model. The constructed face may be animated via muscle actuations. In this way, we create the most authentic and functional facial models of individuals available to date and demonstrate their use in facial animation.

This paper addresses the problem of automatically synchronizing computer-generated faces with synthetic speech. The complete process provides a novel form of face-to-face communication and the ability to create a new range of talking personable synthetic characters. Based on plain ASCII text input, a synthetic speech segment is generated and synchronized in real-time to a graphical display of an articulating mouth and face. The key component of the algorithm is the run-time facility that adaptively synchronizes the graphical display of the face to the audio.

An approach to the analysis of dynamic facial images for the purposes of estimating and resynthesizing dynamic facial expressions is presented. The approach exploits a sophisticated generative model of the human face originally developed for realistic facial animation. The face model which may be simulated and rendered at interactive rates on a graphics workstation, incorporates a physics-based synthetic facial tissue and a set of anatomically motivated facial muscle actuators. The estimation of dynamical facial muscle contractions from video sequences of expressive human faces is considered. An estimation technique that uses deformable contour models (snakes) to track the nonrigid motions of facial features in video images is developed. The technique estimates muscle actuator controls with sufficient accuracy to permit the face model to resynthesize transient expressions.>

Abstract This paper extends a physically‐based approach to the realistic modelling and animation of human heads using scanned data. A three‐step procedure is proposed which begins with geometry and colour data acquired from a subject's head using a radial laser scanner, continues with the construction of a 3D model of the head with complete colour texture mapping, and culminates in real‐time, interactive facial animation. First, an adaptive meshing technique is described that creates non‐uniform facial meshes from the densely sampled data. These geometrically accurate facial meshes adapt to features of interest in the data, increasing polygon density over articulate face regions and areas of high curvature. Secondly, the mesh is transformed into a physically‐based head model of the subject. This incorporates a physical approximation to facial tissue and a set of anatomically‐motivated facial muscle actuators. Finally, these components are integrated in a facial modelling and animation system.

A novel approach is presented to the analysis of dynamic facial images. The approach exploits a realistic model of the human face. The face model, which may be simulated and rendered at interactive rates on a graphics workstation, incorporates a physically-based approximation to facial tissue and a set of anatomically-motivated facial muscle actuators. The authors consider the estimation of dynamic facial muscle contractions from video sequences of expressive human faces. They develop an estimation technique that uses deformable contour models to track the nonrigid motions of facial features in images. The technique computes robust muscle actuator controls, enabling the face model to resynthesize transient expressions accurately.>

Abstract We develop a new 3D hierarchical model of the human face. The model incorporates a physically‐based approximation to facial tissue and a set of anatomically‐motivated facial muscle actuators. Despite its sophistication, the model is efficient enough to produce facial animation at interactive rates on a high‐end graphics workstation. A second contribution of this paper is a technique for estimating muscle contractions from video sequences of human faces performing expressive articulations. These estimates may be input as dynamic control parameters to the face model in order to produce realistic animation. Using an example, we demonstrate that our technique yields sufficiently accurate muscle contraction estimates for the model to reconstruct expressions from dynamic images of faces.

The development of a parameterized facial muscle process, that incorporates the use of a model to create realistic facial animation is described.Existing methods of facial parameterization have the inherent problem of hard-wiring performable actions. The development of a muscle process that is controllable by a limited number of parameters and is non-specific to facial topology allows a richer vocabulary and a more general approach to the modelling of the primary facial expressions.A brief discussion of facial structure is given, from which a method for a simple modelling of a muscle process that is suitable for the animation of a number of divergent facial types is described.