There has recently been an explosion of interest in creating large-scale shared virtual spaces for multiplayer content. However, rendering player-controllable avatars in real-time creates latency issues when scaling to thousands of players. We introduce a human audience video dataset to support applications in deep learning-based 2D video audience simulation, bypassing the need for background 3D virtual humans. This dataset consists of YouTube videos that depict audiences with diverse lighting conditions, color, dress, and movement patterns. We describe the dataset statistics, our implicit data collection strategy, and audience video extraction pipeline. We apply deep learning tasks on this data based on video prediction techniques, and propose a novel method for 2D audience simulations.

Crowds arise in a variety of situations, such as public concerts and sporting matches. In typical conditions, the crowd moves in an orderly manner, but panic situations may lead to catastrophic results. We propose a computer vision method to identify motion pattern changes in human crowds that can be related to an unusual event. The proposed approach can identify global changes, by evaluating 2D motion histograms in time, and also local effects, by identifying clusters that present similar spatial locations and velocity vectors. The method is tested both on publicly available data sets involving crowded scenarios and on synthetic data produced by a crowd simulation algorithm, which allows the creation of controlled environments with known motion patterns that are particularly suitable for multicamera scenarios.

A major goal of research on virtual humans is the animation of expressive characters that display distinct psychological attributes. Body motion is an effective way of portraying different personalities and differentiating characters. The purpose and contribution of this work is to describe a formal, broadly applicable, procedural, and empirically grounded association between personality and body motion and apply this association to modify a given virtual human body animation that can be represented by these formal concepts. Because the body movement of virtual characters may involve different choices of parameter sets depending on the context, situation, or application, formulating a link from personality to body motion requires an intermediate step to assist generalization. For this intermediate step, we refer to Laban Movement Analysis, which is a movement analysis technique for systematically describing and evaluating human motion. We have developed an expressive human motion generation system with the help of movement experts and conducted a user study to explore how the psychologically validated OCEAN personality factors were perceived in motions with various Laban parameters. We have then applied our findings to procedurally animate expressive characters with personality, and validated the generalizability of our approach across different models and animations via another perception study.

Recognizing and captioning the occurrence of virtual episodes can add descriptive capabilities to games and simulations featuring human-like agents. We introduce a captioning heuristic for multi-character animations. The input of our algorithm consists of the traces (lowest-level procedure names) of each character's animation, such as walking, running, talking, reaching, etc. To identify virtual episodes from these traces we pre-authored episode-centric trees called Core Components Trees (CCT). We compute a vagueness measure over each possible match between episode CCTs and the given trace inputs using fuzzy logic, and derive the best match to describe and caption the perceived episodes.

In the social psychology literature, crowds are classified as audiences and mobs. Audiences are passive crowds, whereas mobs are active crowds with emotional, irrational and seemingly homogeneous behavior. In this study, we aim to create a system that enables the specification of different crowd types ranging from audiences to mobs. In order to achieve this goal we parametrize the common properties of mobs to create collective misbehavior. Because mobs are characterized by emotionality, we describe a framework that associates psychological components with individual agents comprising a crowd and yields emergent behaviors in the crowd as a whole. To explore the effectiveness of our framework we demonstrate two scenarios simulating the behavior of distinct mob types.

Many interactive 3D games utilize motion capture for both character animation and user input. These applications require short, meaningful sequences of data. Manually producing these segments of motion capture data is a laborious, time-consuming process that is impractical for real-time applications. We present a method to automatically produce semantic segmentations of general motion capture data by examining the qualitative properties that are intrinsic to all motions, using Laban Movement Analysis (LMA). LMA provides a good compromise between high-level semantic features, which are difficult to extract for general motions, and low-level kinematic features, which often yield unsophisticated segmentations. Our method finds motion sequences which exhibit high output similarity from a collection of neural networks trained with temporal variance. We show that segmentations produced using LMA features are more similar to manual segmentations, both at the frame and the segment level, than several other automatic segmentation methods.

Abstract A person's emotions and state of mind are apparent in their face and eyes. As a Latin proverb states: ‘The face is the portrait of the mind; the eyes, its informers’. This presents a significant challenge for Computer Graphics researchers who generate artificial entities that aim to replicate the movement and appearance of the human eye, which is so important in human–human interactions. This review article provides an overview of the efforts made on tackling this demanding task. As with many topics in computer graphics, a cross‐disciplinary approach is required to fully understand the workings of the eye in the transmission of information to the user. We begin with a discussion of the movement of the eyeballs, eyelids and the head from a physiological perspective and how these movements can be modelled, rendered and animated in computer graphics applications. Furthermore, we present recent research from psychology and sociology that seeks to understand higher level behaviours, such as attention and eye gaze, during the expression of emotion or during conversation. We discuss how these findings are synthesized in computer graphics and can be utilized in the domains of Human–Robot Interaction and Human–Computer Interaction for allowing humans to interact with virtual agents and other artificial entities. We conclude with a summary of guidelines for animating the eye and head from the perspective of a character animator.

Abstract Pedestrian steering algorithms range from completely procedural to entirely data‐driven, but the former grossly generalize across possible human behaviors and suffer computationally, whereas the latter are limited by the burden of ever‐increasing data samples. Our approach seeks the balanced middle ground by deriving a collection of machine‐learned policies based on the behavior of a procedural steering algorithm through the decomposition of the space of possible steering scenarios into steering contexts. The resulting algorithm scales well in the number of contexts, the use of new data sets to create new policies, and in the number of controlled agents as the policies become a simple evaluation of the rules asserted by the machine‐learning process. We also explore the use of synthetic data from an “oracle algorithm” that serves as an as‐needed source of samples, which can be stochastically polled for effective coverage. We observe that our approach produces pedestrian steering similar to that of the oracle steering algorithm, but with a significant performance boost. Runtime was reduced from hours under the oracle algorithm with 10 agents to on the order of 10 frames per second (FPS) with 3000 agents. We also analyze the nature of collisions in such a framework with no explicit collision avoidance. Copyright © 2014 John Wiley & Sons, Ltd.

Abstract Path planning is a fundamental problem in many areas, ranging from robotics and artificial intelligence to computer graphics and animation. Although there is extensive literature for computing optimal, collision‐free paths, there is relatively little work that explores the satisfaction of spatial constraints between objects and agents at the global navigation layer. This paper presents a planning framework that satisfies multiple spatial constraints imposed on the path. The type of constraints specified can include staying behind a building, walking along walls, or avoiding the line of sight of patrolling agents. We introduce two hybrid environment representations that balance computational efficiency and search space density to provide a minimal, yet sufficient, discretization of the search graph for constraint‐aware navigation. An extended anytime dynamic planner is used to compute constraint‐aware paths, while efficiently repairing solutions to account for varying dynamic constraints or an updating world model. We demonstrate the benefits of our method on challenging navigation problems in complex environments for dynamic agents using combinations of hard and soft, attracting and repelling constraints, defined by both static obstacles and moving obstacles. Copyright © 2014 John Wiley & Sons, Ltd.

This paper presents a GPU-based wave-front propagation technique for multi-agent path planning in extremely large, complex, dynamic environments. Our work proposes an adaptive subdivision of the environment with efficient indexing, update, and neighbor-finding operations on the GPU to address several known limitations in prior work. In particular, an adaptive environment representation reduces the device memory requirements by an order of magnitude which enables for the first time, GPU-based goal path planning in truly large-scale environments (> 2048 m2) for hundreds of agents with different targets. We compare our approach to prior work that uses an uniform grid on several challenging navigation benchmarks and report significant memory savings, and up to a 1000X computational speedup.

This paper addresses the problem of group path planning while maintaining group coherence and persistence. Group coherence ensures that a group minimizes both longitudinal and lateral dispersion, and is achieved with the introduction of a deformation penalty to the cost formulation. When the deformation penalty is significantly high, a group may split and later merge. Group persistence is modeled by introducing split and merge actions in the action space, and adding a split penalty to the cost measure. We formulate the problem domain (state, action space, and cost formulation), present our path planning approach for coherent and persistent groups, and provide empirical results demonstrating the capabilities of our method on a variety of challenging scenarios.

Viewers effortlessly decouple action from style for human motion. Regardless of whether style refers to the subtle differences between individuals (John Wayne's walk versus Charlie Chaplin's walk) or to the manner in which the same action is expressed (such as a sad walk versus a nervous walk), the core intent of an action is readily recognizable.

With the increasing realism of interactive applications, there is a growing need for harnessing additional sensory modalities such as hearing. While the synthesis and propagation of sounds in virtual environments has been explored, there has been little work that addresses sound localization and its integration into behaviors for autonomous virtual agents. This paper develops a framework that enables autonomous virtual agents to localize sounds in dynamic virtual environments, subject to distortion effects due to attenuation, reflection and diffraction from obstacles, as well as interference between multiple audio signals. We additionally integrate hearing into standard predictive collision avoidance techniques and couple it with vision to allow agents to react to what they see and hear, while navigating in virtual environments.

We present ADAPT, a flexible platform for designing and authoring functional, purposeful human characters in a rich virtual environment. Our framework incorporates character animation, navigation, and behavior with modular interchangeable components to produce narrative scenes. The animation system provides locomotion, reaching, gaze tracking, gesturing, sitting, and reactions to external physical forces, and can easily be extended with more functionality due to a decoupled, modular structure. The navigation component allows characters to maneuver through a complex environment with predictive steering for dynamic obstacle avoidance. Finally, our behavior framework allows a user to fully leverage a character's animation and navigation capabilities when authoring both individual decision-making and complex interactions between actors using a centralized, event-driven model.

Motion capture remains a popular and widely-used method for animating virtual characters. However, all practical applications of motion capture rely on motion editing techniques to increase the reusability and flexibility of captured motions. Because humans are proficient in detecting and interpreting subtle details in human motion, understanding the perceptual consequences of motion editing is essential. Thus in this work, we perform three experiments to gain a better understanding of how motion editing might affect the emotional content of a captured performance, particularly changes in posture and dynamics, two factors shown to be important perceptual indicators of bodily emotions. In these studies, we analyse the properties (angles and velocities) and perception (recognition rates and perceived intensities) of a varied set of full-body motion clips representing the six emotions anger, disgust, fear, happiness, sadness, and surprise. We have found that emotions are mostly conveyed through the upper body, that the perceived intensity of an emotion can be reduced by blending with a neutral motion, and that posture changes can alter the perceived emotion but subtle changes in dynamics only alter the intensity.

Path finding is a fundamental, yet computationally expensive problem in robotics navigation. Often times, it is necessary to sacrifice optimality to find a feasible plan given a time constraint due to the search complexity. Dynamic environments may further invalidate current computed plans, requiring an efficient planning strategy that can repair existing solutions. This paper presents a massively parallelized wavefront-based approach to path planning, running on the GPU, that can efficiently repair plans to accommodate world changes and agent movement, without having to restart the wavefront propagation process. In addition, we introduce a termination condition which ensures the minimum number of GPU iterations while maintaining strict optimality constraints on search graphs with non-uniform costs.

Path planning is a fundamental problem in many areas ranging from robotics and artificial intelligence to computer graphics and animation. While there is extensive literature for computing optimal, collision-free paths, there is little work that explores the satisfaction of spatial constraints between objects and agents at the global navigation layer. This paper presents a planning framework that satisfies multiple spatial constraints imposed on the path. The type of constraints specified could include staying behind a building, walking along walls, or avoiding the line of sight of patrolling agents. We introduce a hybrid environment representation that balances computational efficiency and discretization resolution, to provide a minimal, yet sufficient discretization of the search graph for constraint-aware navigation. An extended anytime-dynamic planner is used to compute constraint-aware paths, while efficiently repairing solutions to account for dynamic constraints. We demonstrate the benefits of our method on challenging navigation problems in complex environments for dynamic agents using combinations of hard and soft constraints, attracting and repelling constraints, on static obstacles and moving obstacles.

Adventure role playing games (RPGs) provide players with increasingly expansive worlds, compelling storylines, and meaningful fictional character interactions. Despite the fast-growing richness of these worlds, the majority of interactions between the player and non-player characters (NPCs) still remain scripted. In this paper we propose using an NPC's animations to reflect how they feel towards the player and as a proof of concept, investigate the potential for a straightforward gaze model to convey trust. Through two perceptual experiments, we find that viewers can distinguish between high and low trust animations, that viewers associate the gaze differences specifically with trust and not with an unrelated attitude (aggression), and that the effect can hold for different facial expressions and scene contexts, even when viewed by participants for a short (five second) clip length. With an additional experiment, we explore the extent that trust is uniquely conveyed over other attitudes associated with gaze, such as interest, unfriendliness, and admiration.

In this paper, we propose an event-centric planning framework for directing interactive narratives in complex 3D environments populated by virtual humans. Events facilitate precise authorial control over complex interactions involving groups of actors and objects, while planning allows the simulation of causally consistent character actions that conform to an overarching global narrative. Events are defined by preconditions, postconditions, costs, and a centralized behavior structure that simultaneously manages multiple participating actors and objects. By planning in the space of events rather than in the space of individual character capabilities, we allow virtual actors to exhibit a rich repertoire of individual actions without causing combinatorial growth in the planning branching factor. Our system produces long, cohesive narratives at interactive rates, allowing a user to take part in a dynamic story that, despite intervention, conforms to an authored structure and accomplishes a predetermined goal.

There has been a recent paradigm shift in the computer animation industry with an increasing use of pre-recorded motion for animating virtual characters. A fundamental requirement to using motion capture data is an efficient method for indexing and retrieving motions. In this paper, we propose a flexible, efficient method for searching arbitrarily complex motions in large motion databases. Motions are encoded using keys which represent a wide array of structural, geometric and, dynamic features of human motion. Keys provide a representative search space for indexing motions and users can specify sequences of key values as well as multiple combination of key sequences to search for complex motions. We use a trie-based data structure to provide an efficient mapping from key sequences to motions. The search times (even on a single CPU) are very fast, opening the possibility of using large motion data sets in real-time applications.

We present ADAPT, a flexible platform for designing and authoring functional, purposeful human characters in a rich virtual environment. Our framework incorporates character animation, navigation, and behavior with modular interchangeable components to produce narrative scenes. Our animation system provides locomotion, reaching, gaze tracking, gesturing, sitting, and reactions to external physical forces, and can easily be extended with more functionality due to a decoupled, modular structure. Additionally, our navigation component allows characters to maneuver through a complex environment with predictive steering for dynamic obstacle avoidance. Finally, our behavior framework allows a user to fully leverage a character's animation and navigation capabilities when authoring both individual decision-making and complex interactions between actors using a centralized, event-driven model.

ABSTRACT Conversations between two people are ubiquitous in many inhabited contexts. The kinds of conversations that occur depend on several factors, including the time, the location of the participating agents, the spatial relationship between the agents, and the type of conversation in which they are engaged. The statistical distribution of dyadic conversations among a population of agents will therefore depend on these factors. In addition, the conversation types, flow, and duration will depend on agent attributes such as interpersonal relationships, emotional state, personal priorities, and socio‐cultural proxemics. We present a framework for distributing conversations among virtual embodied agents in a real‐time simulation. To avoid generating actual language dialogues, we express variations in the conversational flow by using behavior trees implementing a set of conversation archetypes. The flow of these behavior trees depends in part on the agents' attributes and progresses based on parametrically estimated transitional probabilities. With the participating agents' state, a ‘smart event’ model steers the interchange to different possible outcomes as it executes. Example behavior trees are developed for two conversation archetypes: buyer–seller negotiations and simple asking–answering; the model can be readily extended to others. Because the conversation archetype is known to participating agents, they can animate their gestures appropriate to their conversational state. The resulting animated conversations demonstrate reasonable variety and variability within the environmental context. Copyright © 2012 John Wiley & Sons, Ltd.

In this paper we investigate the utility of an interactive, desktop-based virtual reality (VR) system for training personnel in hazardous working environments. Employing a novel software model, CRAM (Course Resource with Active Materials), we asked participants to learn a specific aircraft maintenance task. The evaluation sought to identify the type of familiarization training that would be most useful prior to hands on training, as well as after, as skill maintenance. We found that participants develop an increased awareness of hazards when training with stimulating technology - in particular (1) interactive, virtual simulations and (2) videos of an instructor demonstrating a task - versus simply studying (3) a set of written instructions. The results also indicate participants desire to train with these technologies over the standard written instructions. Finally, demographic data collected during the evaluation elucidates future directions for VR systems to develop a more robust and stimulating hazard training environment.

Abstract Aging and imperfections provide important visual cues for realism. We present a novel physically‐based approach for simulating the biological aging and decay process in fruits. This method simulates interactions between multiple processes. Our biologically‐derived, reaction‐diffusion model generates growth patterns for areas of fungal and bacterial infection. Fungal colony spread and propagation is affected by both bacterial growth and nutrient depletion. This process changes the physical properties of the surface of the fruit as well as its internal volume substrate. The fruit is physically simulated with parameters such as skin thickness and porosity, water content, flesh rigidity, ambient temperature, humidity, and proximity to other surfaces. Our model produces a simulation that closely mirrors the progression of decay in real fruits under similar parameterized conditions. Additionally, we provide a tool that allows artists to customize the input of the program to produce generalized fruit simulations.

What is often missing from many virtual worlds is a physical sense of the confinement and constraint of the virtual environment. To address this issue, we present a method for providing localized cutaneous vibratory feedback to the user's right arm. We created a sleeve of tactors linked to a real-time human model that activates when the corresponding body area collides with an object. The hypothesis is that vibrotactile feedback to body areas provides the wearer sufficient guidance to ascertain the existence and physical realism of access paths and body configurations. The results of human subject experiments clearly show that the use of full arm vibrotactile feedback improves performance over purely visual feedback in navigating the virtual environment. These results validate the empirical performance of this concept

Abstract This paper presents a new region‐based method for automatically synthesizing varied, natural looking 3D human face models by morphing local facial features according to intuitive control parameters. We automatically compute a one‐to‐one vertex correspondence among the unregistered face scans in a large database by deforming a generic mesh to fit the specific person's face geometry in a global‐to‐local fashion. With the obtained correspondence, we transform the generated data sets of feature shapes into vector space representations. We parameterize the example models using the face anthropometric measurements that reflect the facial physical structure, and predefine the interpolation functions for the parameterized example models based on radial basis functions. At runtime, the interpolation functions are evaluated to efficiently generate the appropriate feature shape by taking the anthropometric parameters as input. We use a shape blending approach for generating a seamlessly deformed mesh around the feature region boundary. The correspondence among all example textures is established by parameterizing the 3D generic mesh over a 2D image domain. The new feature texture with desired attributes is synthesized by interpolating the example textures. Our 3D face synthesis method has several advantages: (1) it regulates the naturalness of synthesized faces, maintaining the quality existing in the real face examples; (2) the region‐based morphing and comprehensive face shape and texture control parameters allow more diverse faces to be generated readily; and (3) the automatic runtime face synthesis is efficient in time complexity and performs fast. Copyright © 2006 John Wiley & Sons, Ltd.

We present a heuristic-based real-time reach planning algorithm for virtual human figures. Given the start and goal positions in a 3D workspace, our problem is to compute a collision-free path that specifies all the configurations for a human arm to move from the start to the goal. Our algorithm consists of three modules: spatial search, inverse kinematics, and collision detection. For the search module, instead of searching in joint configuration space like most existing motion planning methods do, we run a direct search in the workspace, guided by a heuristic distance-to-goal evaluation function. The inverse kinematics module attempts to select natural posture configurations for the arm along the path found in the workspace. During the search, candidate configurations will be checked for collisions taking advantage of the graphics hardware-depth buffer. The algorithm is fast and easy to implement. It allows real-time planning not only in static, structured environments, but also in dynamic, unstructured environments. No preprocessing and prior knowledge about the environment is required. Several examples are shown illustrating the competence of the planner at generating motion plans for a typical human arm model with seven degrees of freedom.

The usefulness of modern day haptics equipment for virtual simulations of actual maintenance actions is examined. In an effort to categorize which areas haptic simulations may be useful, we have developed a taxonomy for haptic actions. This classification has two major dimensions: the general type of action performed and the type of force or torque required. Building upon this taxonomy, we selected three representative tasks from the taxonomy to evaluate in a virtual reality simulation. We conducted a series of human subject experiments to compare user performance and preference on a disassembly task with and without haptic feedback using CyberGlove, Phantom, and SpaceMouse interfaces. Analysis of the simulation runs shows Phantom users learned to accomplish the simulated actions significantly more quickly than did users of the CyberGlove or the SpaceMouse. Moreover, a lack of differences in the post-experiment questionnaire suggests that haptics research should include a measure of actual performance speed or accuracy rather than relying solely on subjective reports of a device's ease of use.

The last few years have seen great maturation in understanding how to use computer graphics technology to portray 3D embodied characters or virtual humans. Unlike the off-line, animator-intensive methods used in the special effects industry, real-time embodied agents are expected to exist and interact with us "live." They can be represent other people or function as autonomous helpers, teammates, or tutors enabling novel interactive educational and training applications. We should be able to interact and communicate with them through modalities we already use, such as language, facial expressions, and gesture. Various aspects and issues in real-time virtual humans will be discussed, including consistent parameterizations for gesture and facial actions using movement observation principles, and the representational basis for character believability, personality, and affect. We also describe a Parameterized Action Representation (PAR) that allows an agent to act, plan, and reason about its actions or actions of others. Besides embodying the semantics of human action, the PAR is designed for building future behaviors into autonomous agents and controlling the animation parameters that portray personality, mood, and affect in an embodied agent.

For an animated human face model to appear natural it should produce eye movements consistent with human ocular behavior. During face-to-face conversational interactions, eyes exhibit conversational turn-taking and agent thought processes through gaze direction, saccades, and scan patterns. We have implemented an eye movement model based on empirical models of saccades and statistical models of eye-tracking data. Face animations using stationary eyes, eyes with random saccades only, and eyes with statistically derived saccades are compared, to evaluate whether they appear natural and effective while communicating.

Creating a complex virtual environment with human inhabitants that behave as we would expect real humans to behave is a difficult and time consuming task. Time must be spent to construct the environment, to create human figures, to create animations for the agents' actions, and to create controls for the agents' behaviors, such as scripts, plans, and decision-makers. Often work done for one virtual environment must be completely replicated for another. The creation of robust, procedural actions that can be ported from one simulation to another would ease the creation of new virtual environments. As walking is useful in many different virtual environments, the creation of natural looking walking is important. In this paper we present a system for producing more natural looking walking by incorporating actions for the upper body. We aim to provide a tool that authors of virtual environments can use to add realism to their characters without effort.

We present a new collision resolution scheme for cloth collisions. Our main concern is to find dynamically convincing resolutions, i.e. positions and velocities of cloth elements, for any kinds of collisions occuring in cloth simulation (cloth-cloth, cloth-rigid, anti cloth-cloth-rigid). We define our cloth surface as connected faces of mass particles, where each particle is controlled by its internal energy functions. Our collision resolution method finds appropriate next positions and velocities of particles by conserving the particles' momentums as accurately as possible. Cloth-cloth collision resolution is a special case of deformable N-body collision resolution. So to solve deformable N-body collision resolutions, we propose a new collision resolution method which groups cloth particles into parts and resolves collisions between parts using the law of momentum conservation. To resolve collisions, we solve a system of linear equations derived from the collision relationships. A system of linear equations is built using a scheme adapted from the simultaneous resolution method for rigid N-body collisions. For the special case where we can,find cyclic relationships in collisions, we solve a system of linear inequalities derived from the collision relationships.

A list of representative directional prepositions of the English language is investigated to develop computation models that output some general motion trajectory or goal direction, given instructions involving prepositional phrases. Computation models are implemented through geometric definitions and procedures such as: centroid, quasi-centroid, convex-hull, closest, nearest-neighbour, and next-to. All algorithms are defined by or derived from standard computational geometry concepts.

We describe a new paradigm in which a user can produce a wide range of expressive, natural-looking movements of animated characters by specifying their manners and attitudes with natural language verbs and adverbs. A natural language interpreter, a parameterized action representation (PAR), and an expressive motion engine (EMOTE) are designed to bridge the gap between natural language instructions issued by the user and expressive movements carried out by the animated characters. By allowing users to customize basic movements with natural language terms to support individualized expressions, our approach may eventually lead to the automatic generation of expressive movements from speech text, a storyboard script, or a behavioral simulation.

Computer synthesized characters are expected to make appropriate face, limb, and body gestures during communicative acts. We focus on non-facial movements and try to elucidate what is intended with the notions of "gesture" and "naturalness". We argue that looking only at the psychological notion of gesture and gesture type is insufficient to capture movement qualities needed by an animated character. Movement observation science, specifically Laban Movement Analysis and its Effort and Shape components with motion phrasing provide essential gesture components (I. Bartenieff and D. Lewis, 1980). We assert that the expression of movement qualities from the Effort dimensions are needed to make a gesture naturally crystallize out of abstract movements. Finally, we point out that nonfacial gestures must involve the rest of the body to appear natural and convincing. A system called EMOTE has been implemented which applies parameterized Effort and Shape qualities to movements and thereby forms improved synthetic gestures.

Human movements include limb gestures and postural attitude. Although many computer animation researchers have studied these classes of movements, procedurally generated movements still lack naturalness. We argue that looking only at the psychological notion of gesture is insufficient to capture movement qualities needed by animated characters. We advocate that the domain of movement observation science, specifically Laban Movement Analysis (LMA) and its Effort and Shape components, provides us with valuable parameters for the form and execution of qualitative aspects of movements. Inspired by some tenets shared among LMA proponents, we also point out that Effort and Shape phrasing across movements and the engagement of the whole body are essential aspects to be considered in the search for naturalness in procedurally generated gestures. Finally, we present EMOTE (Expressive MOTion Engine), a 3D character animation system that applies Effort and Shape qualities to independently defined underlying movements and thereby generates more natural synthetic gestures.

In this paper we develop a set of inverse kinematics algorithms suitable for an anthropomorphic arm or leg. We use a combination of analytical and numerical methods to solve generalized inverse kinematics problems including position, orientation, and aiming constraints. Our combination of analytical and numerical methods results in faster and more reliable algorithms than conventional inverse Jacobian and optimization-based techniques. Additionally, unlike conventional numerical algorithms, our methods allow the user to interactively explore all possible solutions using an intuitive set of parameters that define the redundancy of the system.

Automatically animating virtual humans with actions that reflect real human motions is still a challenge. We present a framework for animation that is based on utilizing empirical and validated data from movement observation and cognitive psychology. To illustrate these, we demonstrate a mapping from effort motion factors onto expressive arm movements, and from cognitive data to autonomous attention behaviors. We conclude with a discussion on the implications of this approach for the future of real time virtual human animation.

Creation of compelling 3-dimensional, multi-user virtual worlds for education and training applications requires a high degree of realism in the appearance, interaction, and behavior of avatars within the scene. Our goal is to develop and/or adapt existing 3-dimensional technologies to provide training scenarios across the Internet in a form as close as possible to the appearance and interaction expected of live situations with human participants. We have produced a prototype system, JackMOO, which combines Jack, a virtual human system, and LambdaMOO, a multiuser; network-accessible, programmable, interactive server: Jack provides the visual realization of avatars and other objects. LambdaMOO provides the Web-accessible communication, programmability, and persistent object database. The combined JackMOO allows us to store the richer semantic information necessitated by the scope and range of human actions that an avatar must portray, and to express those actions in the form of imperative sentences. We describe JackMOO, its components, and a prototype application with five virtual human agents.

We analyzed Neal Stephenson's novel "Snow Crash" for all references to avatars. This analysis is documented in this paper, including a comparison of the Snow Crash avatars to the current state of real-time virtual human research. The avatar characteristics discussed include appearance, clothing and attachments, ethnicity, locomotion, body actions, forms of communication, emotion and personality.

Gesture and speech are two very important behaviors for virtual humans. They are not isolated from each other but generally employed simultaneously in the service of the same intention. An underlying PaT-Net parallel finite state machine may be used to coordinate them both. Gesture selection is not arbitrary. Typical movements correlated with specific textual elements are used to select and produce gesticulation online. This enhances the expressiveness of speaking virtual humans.

The last few years have seen great maturation in the computation speed and control methods needed to portray 3D virtual humans that are suitable for real interactive applications. We first describe the state of the art, then focus on the particular approach taken at the University of Pennsylvania with the Jack system. Various aspects of real-time virtual humans are considered, such as their appearance and motion, interactive control, autonomous actions, gestures, attention, locomotion and multiple individuals. The underlying architecture consists of a sense-control-act structure that permits reactive behaviors to be locally adaptive to the environment, and a PaT-Net (parallel transition network) parallel-finite-state machine controller that can be used to drive virtual humans through complex tasks. We then argue for a deep connection between language and animation and describe current efforts in linking them through two systems: the Jack Presenter and the JackMOO extension to lambdaMOO. Finally, we outline a parameterized action representation for mediating between language instructions and animated actions.

Article Free Access Share on Multi-pass pipeline rendering: realism for dynamic environments Authors: Paul J. Diefenbach DRaW Computing Associates Inc., 3508 Market St., Philadelphia, PA, and Center for Human Modeling and Simulation, University of Pennsylvania, Philadelphia, PA DRaW Computing Associates Inc., 3508 Market St., Philadelphia, PA, and Center for Human Modeling and Simulation, University of Pennsylvania, Philadelphia, PAView Profile , Norman I. Badler Center for Human Modeling and Simulation, University of Pennsylvania, Philadelphia PA Center for Human Modeling and Simulation, University of Pennsylvania, Philadelphia PAView Profile Authors Info & Claims I3D '97: Proceedings of the 1997 symposium on Interactive 3D graphicsApril 1997 Pages 59–ff.https://doi.org/10.1145/253284.253308Online:30 April 1997Publication History 40citation899DownloadsMetricsTotal Citations40Total Downloads899Last 12 Months22Last 6 weeks5 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteeReaderPDF

We present a physics based system for the guided animation of articulated figures. Based on an efficient forward dynamics simulator we introduce a robust feedback control scheme and a fast two stage collision response algorithm. A user of our system provides kinematic trajectories for those degrees of freedom (DOFs) of the figure they want direct control over. The output motion is fully generated using forward dynamics. The specified motion trajectories are the input to a control system which computes the forces and torques that should be exerted to achieve the desired motion. The dynamic controllers, designed based on the Model Reference Adaptive Control paradigm, continuously self adjust for optimal performance in trajectory following. Moreover, the user is given a handle on the type and speed of reaction of the figure's controlled DOFs to sudden changes in their desired motion. The overall goal of our system is to provide a platform for generating and studying realistic, user controlled motion at interactive rates. We require minimal user involvement in specifying non intuitive parameters.

A system for controlling the behaviors of an interactive human-like agent, and executing them in real-time, is presented. It relies on an underlying model of continuous behavior as well as a discrete scheduling mechanism for changing behavior over time. A multiprocessing framework executes the behaviors and renders the motion of the agents in real-time. Finally we discuss the current state of our implementation and some areas of future work.

Based on an ongoing attempt to integrate Natural Language instructions with human figure animation, we demonstrate that agents' understanding and use of instructions can complement what they can derive from the environment in which they act. We focus on two attitudes that contribute to agents' behavior—their intentions and their expectations—and shown how Natural Language instructions contribute to such attitudes in ways that complement the environment. We also show that instructions can require more than one context of interpretation and thus that agents' understanding of instructions can evolve as their activity progresses. A significant consequence is that Natural Language understanding in the context of behavior cannot simply be treated as “front end” processing, but rather must be integrated more deeply into the processes that guide an agent's behavior and respond to its perceptions.

While interpolating between successive postures of an articulated figure is not mathematically difficult, it is much more useful to provide postural transitions that are behaviorally reasonable and that avoid collisions with nearby objects. We describe such a posture interpolator which begins with a number of pre-defined static postures. A finite state machine controls the transitions from any posture to a goal posture by finding the shortest path of required motion sequences between the two. If the motion between any two postures is not collision free, a collision avoidance strategy is invoked and the posture is changed to one that satisfies the required goal while respecting object and agent integrity.>

We describe a real-time model of terrain traversal by simulated human agents. Agent navigation includes a variety of simulated sensors, terrain reasoning with behavioral constraints, and detailed simulation of a variety of locomotion techniques. The path through the terrain is incrementally computed by a behavioral reasoning system configuring a behavioral feedback network. A number of sensors acquire information on object range, passageways, obstacles, terrain type, exposure to hostile agents, and so on. The behavioral reasoner weighs this information along with collision avoidance, cost, danger minimization, locomotion types, and other behaviors available to the agent and incrementally attempts to reach a goal location. Since the system is reactive, it can respond to moving obstacles, changing terrain, or unexpected events due to hostile agents or the effects of limited perception. A motor-level component provides general locomotion, including curved path walking and running that work in real-time with human figures.>

We describe an implemented system which automatically generates and animates conversations between multiple human-like agents with appropriate and synchronized speech, intonation, facial expressions, and hand gestures. Conversation is created by a dialogue planner that produces the text as well as the intonation of the utterances. The speaker/listener relationship, the text, and the intonation in turn drive facial expressions, lip motions, eye gaze, head motion, and arm gestures generators. Coordinated arm, wrist, and hand motions are invoked to create semantically meaningful gestures. Throughout we will use examples from an actual synthesized, fully animated conversation.

Abstract In this paper, we present a rule‐based heuristic method of motion planning for an animated human agent with massively redundant degrees of freedom. It constructs motion plans to achieve 3D‐space goals of control points on the body, e.g. a hand, while avoiding collisions. Like the artificial potential field approach, the method performs motion decisions in 3D world space rather than in joint space. To handle the massively redundant degrees of freedom, we use a qualitative kinematic model, which specifies motions of body parts and dependencies among them, without specifying the exact distance parameters. This model helps the body select appropriate primitive motions for given goals of control points more globally than does the gradient vector of an artificial potential field of the body. The method simulates (in imagination) the suggested plan to find whether some body parts hit objects, and how much they penetrate the objects. Based on this simulated collision information, the method suggests intermediate goals of the collision body parts. A subplan to achieve these intermediate goals is again postulated by using the qualitative kinematic model. This explicit reasoning helps alleviate the basic cause of local minima in the potential field approach, namely, conflicts between attractive potential fields due to goal positions of control points and repulsive potential fields due to obstacles.

An articulated figure is often modeled as a set of rigid segments connected with joints. Its configuration can be altered by varying the joint angles. Although it is straight forward to compute figure configurations given joint angles (forward kinematics), it is more difficult to find the joint angles for a desired configuration (inverse kinematics). Since the inverse kinematics problem is of special importance to an animator wishing to set a figure to a posture satisfying a set of positioning constraints, researchers have proposed several different approaches. However, when we try to follow these approaches in an interactive animation system where the object on which to operate is as highly articulated as a realistic human figure, they fail in either generality or performance. So, we approach this problem through nonlinear programming techniques. It has been successfully used since 1988 in the spatial constraint system within Jack , a human figure simulation system developed at the University of Pennsylvania, and proves to be satisfactorily efficient, controllable, and robust. A spatial constraint in our system involves two parts: one constraint on the figure, the end-effector , and one on the spatial environment, the goal . These two parts are dealt with separately, so that we can achieve a neat modular implementation. Constraints can be added one at a time with appropriate weights designating the importance of this constraint relative to the others and are always solved as a group. If physical limits prevent satisfaction of all the constraints, the system stops with the (possibly local) optimal solution for the given weights. Also, the rigidity of each joint angle can be controlled, which is useful for redundant degrees of freedom.

The authors discuss a class of robots which resemble the human structure. They are characterized by having a torso, two manipulator arms, a head mounted with vision system, and a lower body which is either a transport unit or legs. The total number of degrees-of-freedom involved is usually fairly large for these robots. An efficient motion planning algorithm for this type of robot is an aid to the achievement of task level programming. An efficient algorithm is presented that has successfully computed collision-free motions for anthropometric figures with many degrees of freedom within a cluttered environment.>

A fast algorithm is presented that can handle the motion planning problem for articulated figures with branches and many degrees of freedom. The algorithm breaks down the degrees of freedom of the figure into Cspace groups and compute the free motion for each of these groups in a sequential fashion. It traverses the tree in a depth first order to compute the motion for all the branches. A special playback routine is then used to traverse the tree again in a reverse order to playback the final motion. The planner runs in linear time with respect to the total number of Cspace groups without backtracking. We believe that the planner would find a path in most cases and is fast enough for practical use in a wide range of applications.

This paper describes a technique for augmenting the process of 3D direct manipulation by automaticallyfinding an effective placement for the virtual camera.Many of the best techniques for direct manipulation of 3D geometric objects are sensitive to the angle of view, and can thus require that the user coordinate the placement of the viewpoint during the manipulation process.In some cases, this process can be automated.This means that the system can automatically avoid degenerate situations in which translations and rotations are difficult to perform.The system can also select viewpoints and viewing angles which make the object being manipulated visible, ensuring that it is not obstructed by other objects.

Abstract To facilitate the effective use of strength data in a human‐task simulation environment, we have developed graphical methods to display the multidimensional characteristics of strength data. We use human figures together with two and three dimensional graphical symbols, colours and animations to visualize the effects of different parameters on strength. These displays give a dynamic view of the effects of various parameters on strength, and illustrate safe and forbidden body postures (or regions) in terms of strength capabilities.

We describe techniques for interactively controlling bipedal articulated figures through kinematic constraints. These constraints model certain behavioral tendencies which capture some of the characteristics of human-like movement, and give us control over such elements as the figures' balanced and stability. They operate in near real-time, so provide behavioral control for interactive manipulation. These constraints form the basis of an interactive motion-generation system that allows the active movement elements to be layered on top of the passive behavioral constraints.

In this paper, we describe an interactive system for positioning articulated figures which uses a 3D direct manipulation technique to provide input to an inverse kinematics algorithm running in real time. The system allows the user to manipulate highly articulated figures, such as human figure models, by interactively dragging 3D "reach goals." The user may also define multiple "reach constraints" which are enforced during the manipulation. The 3D direct manipulation interface provides a good mechanism for control of the inverse kinematics algorithm and helps it to overcome problems with redundancies and singularities which occur with figures of many degrees of freedom. We use an adaptive technique for evaluating the constraints which allows us to ensure that only a certain user-controllable amount of time will be consumed by the inverse kinematics algorithm at each iteration of the manipulation process. This technique is also sensitive to the time it takes to redraw the screen, so it prevents the frame display rate of the direct manipulation from become too slow for interactive control.

A methodology and algorithm are presented that generate motions imitating the way humans complete a lifting task under various loading conditions. The path taken depends on "natural" parameters: the figure geometry, the given load, the final destination, and, especially, the strength model of the agent. Additional user controllable parameters of the motion are the comfort of the action and the perceived exertion of the agent. The algorithm uses this information to incrementally compute a motion path of the end-effector moving the load. It is therefore instantaneously adaptable to changing force, loading, and strength conditions. Various strategies are used to model human behavior (such as reducing moment, pull back, add additional joints, and jerk) that compute the driving torques as the situation changes. The strength model dictates acceptable kinematic postures. The resulting algorithm offers torque control without the tedious user expression of driving forces under a dynamics model. The algorithm runs in near-realtime and offers an agent-dependent toolkit for fast path prediction. Examples are presented for various lifting tasks, including one-and two-handed lifts, and raising the body from a seated posture.

Abstract Creating objects with surface imperfections is accomplished through texture specification and generation techniques. Based on fractal subdivision techniques and relatively simple distribution models, a wide class of surface imperfections may be generated, combined and rendered. The surface effects include scratches, splotches, smudges, corrosion, mould, stains and rust. A rule‐based system is used to position the various surface imperfections on the texture map, and a simple natural language interface is used to specify the kinds of imperfections and their generative parameters through adverbs and prepositional phrases. Results along some of the imperfection dimensions are illustrated.

The problem of positioning and manipulating three dimensional articulated figures is often handled by ad hoc techniques which are cumbersome to use. In this paper, we describe a system which provides a consistent and flexible user interface to a complex representation for articulated figures in a 3D environment. Jack is a toolkit of routines for displaying and manipulating complex geometric figures, and it provides a method of interactively manipulating arbitrary homogeneous transformations with a mouse. These transformations may specify the position and orientation of figures within a scene or the joint transformations within the figures themselves. Jack combines this method of 3D input with a flexible and informative screen management facility to provide a user-friendly interface for manipulating three dimensional objects.

Parametric keyframing is a popular animation technique where values for parameters which control the position, orientation, size, and shape of modeled objects are determined at key times, then interpolated for smooth animation. Typically the parameter values defined by the keyframes are interpolated by spline techniques with the result that the parameter change kinetics are implicitly defined by the given keyframe times and data points. Existing interpolation systems for animation are examined and found to lack certain desirable features such as continuity of acceleration or convenient kinetic control. The requirements of interpolation for animation are analyzed in order to determine the characteristics of a satisfactory system. A new interpolation system is developed and implemented which incorporates second-derivative continuity (continuity of acceleration), local control, convenient kinetic control, and joining and phrasing of successive motions. Phrasing control includes the ability to parametrically control the degree and extent of smooth motion flow between separately defined motions.

We briefly survey the major thrusts of computer graphics activities, examining trends and topics rather than offering a comprehensive survey of all that is happening. The directions of professional activities, hardware, software, and algorithms are outlined. Within hardware we examine workstations, personal graphics systems, high performance systems, and low level VLSI chips; within software, standards and interactive system design; within algorithms, visible surface rendering and shading, three-dimensional modeling techniques, and animation.

To modify the location and size of a window for some world.

The desirability of incorporating temporal anti-aliasing, or motion blur, into computer generated animation is discussed and two algorithms for achieving this effect are described. The first approximates continuous object movement and determines intervals during which each object covers each pixel. Hidden surface removal is then performed, allowing the calculation of visible object intensity functions and subsequent filtering. The second form of algorithm detailed involves supersampling the moving image and then filtering the resulting intensity function to “multiply-expose” each output picture. The effects of filter types and the relationship of the algorithms to forms of spatial anti-aliasing are discussed.

Recognition and simulation of actions performable on rigidly-jointed actors such as human bodies have been the subject of our research for some time. One part of an ongoing effort towards a total human movement simulator is to develop a system to perform the actions of American Sign Language (ASL). However, one of the “channels” of ASL communication, the face, presents problems which are not well handled by a rigid model.

Three dimensional animation of human movement may be obtained by specifying movements in a goal-directed manner and constructing a sophisticated simulator to execute those movements. We briefly describe an architecture for such a simulator and then concentrate on five special problems which arise: scheduling movements which occur concurrently, computing motion of three-link chains, processing contacts, moving the center of gravity and maintaining balance, and adjusting limb twist for a standard orientation.

A process for creating and displaying stellations of a given polyhedral solid is described. A stellation is one of many star-like polyhedra which can be derived from a single solid by extending its existing faces. A program has been implemented which performs the stellation process on an input object and generates a 3-dimensional image of the stellated object on a computer graphics display screen. Pictures of icosahedron and rhombictriacontahedron stellations generated by the program are included in the paper.

A theory of music visualization proposed by Nancy Herman postulates an association between colors and pitches of musical scales. A color raster graphics display is used to generate images of notes, chords, and chord progressions based on this theory. Temporal adjacency of notes or chords is mapped to spatial adjacency of colors, usually in a concentric pattern of squares or circles. By varying certain image parameters, different “brush stroke” effects may be obtained. Illustrations of several computer generated “musical paintings” are included, along with some original paintings for comparison.

Algorithms are presented for converting between different three-dimensional object representations: from a collection of cross section outlines to surface points, and from surface points to a collection of overlapping spheres. The algorithms effect a conversion from surface representations (outlines or surface points) to a volume representation (spheres). The spherical representation can be useful for graphical display, and perhaps as an intermediate representation for conversions to representations with other primitives. The spherical decomposition also permits the computation of points on the symmetric surface of an object, the three-dimensional analog of Blum's symmetric axis. The algorithms work in real coordinates rather than in a discrete space, and so avoid error introduced by the quantization of the space.

Representing complex three-dimensional objects in a computer involves more than just evaluating its display capabilities. Other factors are the uses and costs of the representation, what operations can be performed on it and, ultimately, how useful it is for computer recognition or description or three-dimensional objects. Many of the questions which are posed arise from the joint consideration of computer graphics and computer vision, and a specific representation hierarchy is proposed for complex objects which makes them amenable to display, manipulation, measurement, and analysis.

Our goal was to design and implement an interactive system using a color video display to assist an archaeologist in the task of analyzing the spatial distribution of various objects from an excavated site. The existing system configuration required controlling the display over a slow communications line from the mainframe CPU, and there was little local intelligence in the minicomputer display driver. The user of the system was expected to be computer-naive. Certain design principals evolved which should be applicable to graphical database query systems in other domains.