Detecting vehicles in aerial images is difficult due to complex backgrounds, small object sizes, shadows, and occlusions. Although recent deep learning advancements have improved object detection, these models remain susceptible to adversarial attacks (AAs), challenging their reliability. Traditional AA strategies often ignore practical implementation constraints. Our work proposes realistic and practical constraints on texture (lowering resolution, limiting modified areas, and color ranges) and analyzes the impact of shape modifications on attack performance. We conducted extensive experiments with three object detector architectures, demonstrating the performance-practicality trade-off: more practical modifications tend to be less effective, and vice versa. We release both code and data to support reproducibility at https://github.com/humansensinglab/texture-shape-adversarial-attacks.

We present Diffuse-CLoC, a guided diffusion framework for physics-based look-ahead control that enables intuitive, steerable, and physically realistic motion generation. While existing kinematics motion generation with diffusion models offer intuitive steering capabilities with inference-time conditioning, they often fail to produce physically viable motions. In contrast, recent diffusion-based control policies have shown promise in generating physically realizable motion sequences, but the lack of kinematics prediction limits their steerability. Diffuse-CLoC addresses these challenges through a key insight: modeling the joint distribution of states and actions within a single diffusion model makes action generation steerable by conditioning it on the predicted states. This approach allows us to leverage established conditioning techniques from kinematic motion generation while producing physically realistic motions. As a result, we achieve planning capabilities without the need for a high-level planner. Our method handles a diverse set of unseen long-horizon downstream tasks through a single pre-trained model, including static and dynamic obstacle avoidance, motion in-betweening, and task-space control. Experimental results show that our method significantly outperforms the traditional hierarchical framework of high-level motion diffusion and low-level tracking.

Hand motion capture data are now relatively easy to obtain, even for complicated grasps; however, these data are of limited use without the ability to retarget it onto the hands of a specific character or robot. The target hand may differ dramatically in geometry, number of degree of freedom (DOF), or number of fingers. We present a simple but effective framework capable of kinematically retargeting human hand-object manipulations from a publicly available dataset to diverse target hands through the exploitation of contact areas. We do so by formulating the retargeting operation as a nonisometric shape matching problem and use a combination of both surface contact and marker data to progressively estimate, refine, and fit the final target hand trajectory using inverse kinematics. Foundational to our framework is the introduction of a novel shape matching process, which we show enables predictable and robust transfer of contact data over full manipulations (pregrasp, pickup, in-hand re-orientation, and release) while providing an intuitive means for artists to specify correspondences with relatively few inputs. We validate our framework through demonstrations across five different hands and six motions of different objects. We additionally demonstrate a bimanual task, perform stress tests, and compare our method against existing hand retargeting approaches. Finally, we demonstrate our method enabling novel capabilities such as object substitution and the ability to visualize the impact of hand design choices over full trajectories.

Hair plays a significant role in personal identity and appearance, making it an essential component of high-quality, photorealistic avatars. Existing approaches either focus on modeling the facial region only or rely on personalized models, limiting their generalizability and scalability. In this paper, we present a novel method for creating high-fidelity avatars with diverse hairstyles. Our method leverages the local similarity across different hairstyles and learns a universal hair appearance prior from multi-view captures of hundreds of people. This prior model takes 3D-aligned features as input and generates dense radiance fields conditioned on a sparse point cloud with color. As our model splits different hairstyles into local primitives and builds prior at that level, it is capable of handling various hair topologies. Through experiments, we demonstrate that our model captures a diverse range of hairstyles and generalizes well to challenging new hairstyles. Empirical results show that our method improves the state-of-the-art approaches in capturing and generating photorealistic, personalized avatars with complete hair.

Tactile sensing plays a pivotal role in human perception and manipulation tasks, allowing us to intuitively understand task dynamics and adapt our actions in real time. Transferring such tactile intelligence to robotic systems would help intelligent agents understand task constraints and accurately interpret the dynamics of both the objects they are interacting with and their own operations. While significant progress has been made in imbuing robots with this tactile intelligence, challenges persist in effectively utilizing tactile information due to the diversity of tactile sensor form factors, manipulation tasks, and learning objectives involved. To address this challenge, we present a unified tactile embedding space capable of predicting a variety of task-centric qualities over multiple manipulation tasks. We collect tactile data from human demonstrations across various tasks and leverage this data to construct a shared latent space for task stage classification, object dynamics estimation, and tactile dynamics prediction. Through experiments and ablation studies, we demonstrate the effectiveness of our shared tactile latent space for more accurate and adaptable tactile networks, showing an improvement of up to 84% over the single-task training.

Cross-pose estimation between rigid objects is a fundamental building block for robotic applications. In this paper, we propose a new cross-pose estimation method that predicts correspondences on a set level as opposed to a point level. This contrasts methods that predict cross-pose from per-point correspondences, which can encounter optimization problems for objects with symmetries, since each point may have multiple valid correspondences. Our method, SCAlign, consists of a Set Correspondence Network (SCN) which predicts these sets and their correspondences, and an alignment module to compute their relative cross-pose. Taking point clouds of two objects as input, SCN predicts a set label for each point such that such that points that share a set label form a cross object correspondence. The alignment module then computes the cross-pose as the SE(3) transformation that aligns these set correspondences. We compare SCAlign against other cross-pose estimation baselines on a synthetically generated dataset, SynWidth, which contains randomly generated width-mate objects with symmetric or near-symmetric intercepts. SCAlign significantly outperforms the baselines on this challenging dataset. Additionally, we show that set correspondences can be leveraged to distinguish positive and negative matches between pegs and holes. Robot experiments further validate the practical application of this approach.

This paper explores the use of synthetic data in a human action recognition (HAR) task to avoid the challenges of obtaining and labeling real-world datasets. We introduce a new dataset suite comprising five datasets, eleven common human activities, three synchronized camera views (aerial and ground) in three outdoor environments, and three visual domains (real and two synthetic). For the synthetic data, two rendering methods (standard computer graphics and neural rendering) and two sources of human motions (motion capture and video-based motion reconstruction) were employed. We evaluated each dataset type by training popular activity recognition models and comparing the performance on the real test data. Our results show that synthetic data achieve slightly lower accuracy (4–8 %) than real data. On the other hand, a model pre-trained on synthetic data and fine-tuned on limited real data surpasses the performance of either domain alone. Standard computer graphics (CG)-rendered data delivers better performance than the data generated from the neural-based rendering method. The results suggest that the quality of the human motions in the training data also affects the test results: motion capture delivers higher test accuracy. Additionally, a model trained on CG aerial view synthetic data exhibits greater robustness against camera viewpoint changes than one trained on real data. See the project page: http://humansensinglab.github.io/REMAG/

In this work, we present a method for personalizing human-robot interaction by using emotive facial expressions to generate affective robot movements. Movement is an important medium for robots to communicate affective states, but the expertise and time required to craft new robot movements promotes a reliance on fixed preprogrammed behaviors. Enabling robots to respond to multimodal user input with newly generated movements could stave off staleness of interaction and convey a deeper degree of affective understanding than current retrieval-based methods. We use autoencoder neural networks to compress robot movement data and facial expression images into a shared latent embedding space. Then, we use a reconstruction loss to generate movements from these embeddings and triplet loss to align the embeddings by emotion classes rather than data modality. To subjectively evaluate our method, we conducted a user survey and found that generated happy and sad movements could be matched to their source face images. However, angry movements were most often mismatched to sad images. This multimodal data-driven generative method can expand an interactive agent’s behavior library and could be adopted for other multimodal affective applications.

The capture and animation of human hair are two of the major challenges in the creation of realistic avatars for the virtual reality. Both problems are highly challenging, because hair has complex geometry and appearance and exhibits challenging motion. In this paper, we present a two-stage approach that models hair independently of the head to address these challenges in a data-driven manner. The first stage, state compression, learns a low-dimensional latent space of 3D hair states including motion and appearance via a novel autoencoder-as-a-tracker strategy. To better disentangle the hair and head in appearance learning, we employ multi-view hair segmentation masks in combination with a differentiable volumetric renderer. The second stage optimizes a novel hair dynamics model that performs temporal hair transfer based on the discovered latent codes. To enforce higher stability while driving our dynamics model, we employ the 3D point-cloud autoencoder from the compression stage for denoising of the hair state. Our model outperforms the state of the art in novel view synthesis and is capable of creating novel hair animations without relying on hair observations as a driving signal.††Project page at https://ziyanwl.github.io/neuwigs/.

Clothing is an important part of human appearance but challenging to model in photorealistic avatars. In this work we present avatars with dynamically moving loose clothing that can be faithfully driven by sparse RGB-D inputs as well as body and face motion. We propose a Neural Iterative Closest Point (N-ICP) algorithm that can efficiently track the coarse garment shape given sparse depth input. Given the coarse tracking results, the input RGB-D images are then remapped to texel-aligned features, which are fed into the drivable avatar models to faithfully reconstruct appearance details. We evaluate our method against recent image-driven synthesis baselines, and conduct a comprehensive analysis of the N-ICP algorithm. We demonstrate that our method can generalize to a novel testing environment, while preserving the ability to produce high-fidelity and faithful clothing dynamics and appearance.

We introduce CT2Hair, a fully automatic framework for creating high-fidelity 3D hair models that are suitable for use in downstream graphics applications. Our approach utilizes real-world hair wigs as input, and is able to reconstruct hair strands for a wide range of hair styles. Our method leverages computed tomography (CT) to create density volumes of the hair regions, allowing us to see through the hair unlike image-based approaches which are limited to reconstructing the visible surface. To address the noise and limited resolution of the input density volumes, we employ a coarse-to-fine approach. This process first recovers guide strands with estimated 3D orientation fields, and then populates dense strands through a novel neural interpolation of the guide strands. The generated strands are then refined to conform to the input density volumes. We demonstrate the robustness of our approach by presenting results on a wide variety of hair styles and conducting thorough evaluations on both real-world and synthetic datasets. Code and data for this paper are at github.com/facebookresearch/CT2Hair.

Children’s drawings have a wonderful inventiveness, creativity, and variety to them. We present a system that automatically animates children’s drawings of the human figure, is robust to the variance inherent in these depictions, and is simple and straightforward enough for anyone to use. We demonstrate the value and broad appeal of our approach by building and releasing the Animated Drawings Demo, a freely available public website that has been used by millions of people around the world. We present a set of experiments exploring the amount of training data needed for fine-tuning, as well as a perceptual study demonstrating the appeal of a novel twisted perspective retargeting technique. Finally, we introduce the Amateur Drawings Dataset, a first-of-its-kind annotated dataset, collected via the public demo, containing over 178,000 amateur drawings and corresponding user-accepted character bounding boxes, segmentation masks, and joint location annotations.

Capturing and rendering life-like hair is particularly challenging due to its fine geometric structure, the complex physical interaction and its non-trivial visual appearance. Yet, hair is a critical component for believable avatars. In this paper, we address the aforementioned problems: 1) we use a novel, volumetric hair representation that is composed of thousands of primitives. Each primitive can be rendered efficiently, yet realistically, by building on the latest advances in neural rendering. 2) To have a reliable control signal, we present a novel way of tracking hair on the strand level. To keep the computational effort manageable, we use guide hairs and classic techniques to expand those into a dense hood of hair. 3) To better enforce temporal consistency and generalization ability of our model, we further optimize the 3D scene flow of our representation with multiview optical flow, using volumetric raymarching. Our method can not only create realistic renders of recorded multi-view sequences, but also create renderings for new hair configurations by providing new control signals. We compare our method with existing work on viewpoint synthesis and drivable animation and achieve state-of-the-art results. https://ziyanw1.github.io/hvh.

High-quality motion capture datasets are now publicly available, and researchers have used them to create kinematics-based controllers that can generate plausible and diverse human motions without conditioning on specific goals (i.e., a task-agnostic generative model). In this paper, we present an algorithm to build such controllers for physically simulated characters having many degrees of freedom. Our physics-based controllers are learned by using conditional VAEs, which can perform a variety of behaviors that are similar to motions in the training dataset. The controllers are robust enough to generate more than a few minutes of motion without conditioning on specific goals and to allow many complex downstream tasks to be solved efficiently. To show the effectiveness of our method, we demonstrate controllers learned from several different motion capture databases and use them to solve a number of downstream tasks that are challenging to learn controllers that generate natural-looking motions from scratch. We also perform ablation studies to demonstrate the importance of the elements of the algorithm. Code and data for this paper are available at: https://github.com/facebookresearch/PhysicsVAE

Despite recent progress in developing animatable full-body avatars, realistic modeling of clothing - one of the core aspects of human self-expression - remains an open challenge. State-of-the-art physical simulation methods can generate realistically behaving clothing geometry at interactive rates. Modeling photorealistic appearance, however, usually requires physically-based rendering which is too expensive for interactive applications. On the other hand, data-driven deep appearance models are capable of efficiently producing realistic appearance, but struggle at synthesizing geometry of highly dynamic clothing and handling challenging body-clothing configurations. To this end, we introduce pose-driven avatars with explicit modeling of clothing that exhibit both photorealistic appearance learned from real-world data and realistic clothing dynamics. The key idea is to introduce a neural clothing appearance model that operates on top of explicit geometry: at training time we use high-fidelity tracking, whereas at animation time we rely on physically simulated geometry. Our core contribution is a physically-inspired appearance network, capable of generating photorealistic appearance with view-dependent and dynamic shadowing effects even for unseen body-clothing configurations. We conduct a thorough evaluation of our model and demonstrate diverse animation results on several subjects and different types of clothing. Unlike previous work on photorealistic full-body avatars, our approach can produce much richer dynamics and more realistic deformations even for many examples of loose clothing. We also demonstrate that our formulation naturally allows clothing to be used with avatars of different people while staying fully animatable, thus enabling, for the first time, photorealistic avatars with novel clothing.

Knitting is a popular craft that can be used to create customized fabric objects such as household items, clothing and toys. Additionally, many knitters find knitting to be a relaxing and calming exercise. Little is known about how disabled knitters use and benefit from knitting, and what accessibility solutions and challenges they create and encounter. We conducted interviews with 16 experienced, disabled knitters and analyzed 20 threads from six forums that discussed accessible knitting to identify how and why disabled knitters knit, and what accessibility concerns remain. We additionally conducted an iterative design case study developing knitting tools for a knitter who found existing solutions insufficient. Our innovations improved the range of stitches she could produce. We conclude by arguing for the importance of improving tools for both pattern generation and modification as well as adaptations or modifications to existing tools such as looms to make it easier to track progress

Photorealistic rendering of dynamic humans is an important capability for telepresence systems, virtual shopping, special effects in movies, and interactive experiences such as games. Recently, neural rendering methods have been developed to create high-fidelity models of humans and objects. Some of these methods do not produce results with high-enough fidelity for driveable human models (Neural Volumes) whereas others have extremely long rendering times (NeRF). We propose a novel compositional 3D representation that combines the best of previous methods to produce both higher-resolution and faster results. Our representation bridges the gap between discrete and continuous volumetric representations by combining a coarse 3D-structure-aware grid of animation codes with a continuous learned scene function that maps every position and its corresponding local animation code to a view-dependent emitted radiance and local volume density. Differentiable volume rendering is employed to compute photo-realistic novel views of the human head and upper body as well as to train our novel representation end-to-end using only 2D supervision. In addition, we show that the learned dynamic radiance field can be used to synthesize novel unseen expressions based on a global animation code. Our approach achieves state-of-the-art results for synthesizing novel views of dynamic human heads and the upper body. See our project page1for more results.

Aerial vehicles are revolutionizing the way filmmakers can capture shots of actors by composing novel aerial and dynamic viewpoints. However, despite great advancements in autonomous flight technology, generating expressive camera behaviors is still a challenge and requires non-technical users to edit a large number of unintuitive control parameters. In this work, we develop a data-driven framework that enables editing of these complex camera positioning parameters in a semantic space (e.g. calm, enjoyable, establishing). First, we generate a database of video clips with a diverse range of shots in a photo-realistic simulator, and use hundreds of participants in a crowd-sourcing framework to obtain scores for a set of semantic descriptors for each clip. Next, we analyze correlations between descriptors and build a semantic control space based on cinematography guidelines and human perception studies. Finally, we learn a generative model that can map a set of desired semantic video descriptors into low-level camera trajectory parameters. We evaluate our system by demonstrating that our model successfully generates shots that are rated by participants as having the expected degrees of expression for each descriptor. We also show that our models generalize to different scenes in both simulation and real-world experiments. Data and video found at: https://sites.google.com/view/robotcam.

Current models of gesture synthesis focus primarily on a speech signal to synthesize gestures. In this paper, we take a critical look at this approach from the point of view of gesture’s tendency to disambiguate the verbal component of the expression. We identify and contribute an analysis of three challenge factors for these models: 1) synthesizing gesture in the presence of ambiguous utterances seems to be a overwhelmingly useful case for gesture production yet is not at present supported by present day models of gesture generation, 2) finding the best f-formation to convey spatial gestural information like gesturing directions makes a significant difference for everyday users and must be taken into account, and 3) assuming that captured human motion is a plentiful and easy source for retargeting gestural motion may not yet take into account the readability of gestures under kinematically constrained feasibility spaces.Recent approaches to generate gesture for agents[1] and robots [2] treat gesture as co-speech that is strictly dependent on verbal utterances. Evidence suggests that gesture selection may leverage task context so it is not dependent on verbal utterance only. This effect is particularly evident when attempting to generate gestures from ambiguous verbal utterances (e.g. "You do this when you get to the fork in the road"). Decoupling this strict dependency may allow gesture to be synthesized for the purpose of clarification of the ambiguous verbal utterance.

In two-player competitive sports, such as boxing and fencing , athletes often demonstrate efficient and tactical movements during a competition. In this paper, we develop a learning framework that generates control policies for physically simulated athletes who have many degrees-of-freedom. Our framework uses a two step-approach, learning basic skills and learning bout-level strategies, with deep reinforcement learning, which is inspired by the way that people how to learn competitive sports. We develop a policy model based on an encoder-decoder structure that incorporates an autoregressive latent variable, and a mixture-of-experts decoder. To show the effectiveness of our framework, we implemented two competitive sports, boxing and fencing , and demonstrate control policies learned by our framework that can generate both tactical and natural-looking behaviors. We also evaluate the control policies with comparisons to other learning configurations and with ablation studies.

We have recently seen great progress in building photorealistic animatable full-body codec avatars, but generating high-fidelity animation of clothing is still difficult. To address these difficulties, we propose a method to build an animatable clothed body avatar with an explicit representation of the clothing on the upper body from multi-view captured videos. We use a two-layer mesh representation to register each 3D scan separately with the body and clothing templates. In order to improve the photometric correspondence across different frames, texture alignment is then performed through inverse rendering of the clothing geometry and texture predicted by a variational autoencoder. We then train a new two-layer codec avatar with separate modeling of the upper clothing and the inner body layer. To learn the interaction between the body dynamics and clothing states, we use a temporal convolution network to predict the clothing latent code based on a sequence of input skeletal poses. We show photorealistic animation output for three different actors, and demonstrate the advantage of our clothed-body avatars over the single-layer avatars used in previous work. We also show the benefit of an explicit clothing model that allows the clothing texture to be edited in the animation output.

We present a method to capture temporally coherent dynamic clothing deformation from a monocular RGB video input. In contrast to the existing literature, our method does not require a pre-scanned personalized mesh template, and thus can be applied to in-the-wild videos. To constrain the output to a valid deformation space, we build statistical deformation models for three types of clothing: T- shirt, short pants and long pants. A differentiable renderer is utilized to align our captured shapes to the input frames by minimizing the difference in both silhouette, segmentation, and texture. We develop a UV texture growing method which expands the visible texture region of the clothing sequentially in order to minimize drift in deformation tracking. We also extract fine-grained wrinkle detail from the input videos by fitting the clothed surface to the normal maps estimated by a convolutional neural network. Our method produces temporally coherent reconstruction of body and clothing from monocular video. We demonstrate successful clothing capture results from a variety of challenging videos. Extensive quantitative experiments demonstrate the effectiveness of our method on metrics including body pose error and surface reconstruction error of the clothing.

Abstract Plausible conversations among characters are required to generate the ambiance of social settings such as a restaurant, hotel lobby, or cocktail party. In this paper, we propose a motion synthesis technique that can rapidly generate animated motion for characters engaged in two‐party conversations. Our system synthesizes gestures and other body motions for dyadic conversations that synchronize with novel input audio clips. Human conversations feature many different forms of coordination and synchronization. For example, speakers use hand gestures to emphasize important points, and listeners often nod in agreement or acknowledgment. To achieve the desired degree of realism, our method first constructs a motion graph that preserves the statistics of a database of recorded conversations performed by a pair of actors. This graph is then used to search for a motion sequence that respects three forms of audio‐motion coordination in human conversations: coordination to phonemic clause, listener response, and partner's hesitation pause. We assess the quality of the generated animations through a user study that compares them to the originally recorded motion and evaluate the effects of each type of audio‐motion coordination via ablation studies.

Many of the actions that we take with our hands involve self-contact and occlusion: shaking hands, making a fist, or interlacing our fingers while thinking. This use of of our hands illustrates the importance of tracking hands through self-contact and occlusion for many applications in computer vision and graphics, but existing methods for tracking hands and faces are not designed to treat the extreme amounts of self-contact and self-occlusion exhibited by common hand gestures. By extending recent advances in vision-based tracking and physically based animation, we present the first algorithm capable of tracking high-fidelity hand deformations through highly self-contacting and self-occluding hand gestures, for both single hands and two hands. By constraining a vision-based tracking algorithm with a physically based deformable model, we obtain an algorithm that is robust to the ubiquitous self-interactions and massive self-occlusions exhibited by common hand gestures, allowing us to track two hand interactions and some of the most difficult possible configurations of a human hand.

Human characters with a broad range of natural looking and physically realistic behaviors will enable the construction of compelling interactive experiences. In this paper, we develop a technique for learning controllers for a large set of heterogeneous behaviors. By dividing a reference library of motion into clusters of like motions, we are able to construct experts , learned controllers that can reproduce a simulated version of the motions in that cluster. These experts are then combined via a second learning phase, into a general controller with the capability to reproduce any motion in the reference library. We demonstrate the power of this approach by learning the motions produced by a motion graph constructed from eight hours of motion capture data and containing a diverse set of behaviors such as dancing (ballroom and breakdancing), Karate moves, gesturing, walking, and running.

Knitting creates complex, soft fabrics with unique texture properties that can be used to create interactive objects.However, little work addresses the challenges of designing and using knitted textures computationally. We present KnitPick: a pipeline for interpreting hand-knitting texture patterns into KnitGraphs which can be output to machine and hand-knitting instructions. Using KnitPick, we contribute a measured and photographed data set of 472 knitted textures. Based on findings from this data set, we contribute two algorithms for manipulating KnitGraphs. KnitCarving shapes a graph while respecting a texture, and KnitPatching combines graphs with disparate textures while maintaining a consistent shape. KnitPick is the first system to bridge the gap between hand- and machine-knitting when creating complex knitted textures.

Engagement with social robots is influenced by their appearance and shape. While robots are designed with various features, almost all designs have some form of eyes. In this paper, we evaluate eye design variations for tabletop robots in a lab study, with the goal of learning how they influence participants' perception of the robots' personality and functionality. This evaluation is conducted with non-working “paper prototypes”, a common design methodology which enables quick evaluation of a variety of designs. By comparing sixteen eye designs we found: (1) The more lifelike the design of the eyes was, the higher the robot was rated on personable qualities, and the more suitable it was perceived to be for the home; (2) Eye design did not affect how professional and how suitable for the office the robot was perceived to be. We suggest that designers can use paper prototypes as a design methodology to quickly evaluate variations of a particular feature for social robots.

Abstract Motion capture sequences may contain erroneous data, especially when the motion is complex or performers are interacting closely and occlusions are frequent. Common practice is to have specialists visually detect the abnormalities and fix them manually. In this paper, we present a method to automatically analyze and fix motion capture sequences by using self‐similarity analysis. The premise of this work is that human motion data has a high‐degree of self‐similarity. Therefore, given enough motion data, erroneous motions are distinct when compared to other motions. We utilizemotion‐wordsthat consist of short sequences of transformations of groups of joints around a given motion frame. We search for the K‐nearest neighbors (KNN) set of each word using dynamic time warping and use it to detect and fix erroneous motions automatically. We demonstrate the effectiveness of our method in various examples, and evaluate by comparing to alternative methods and to manual cleaning.

Many analysis tasks for human motion rely on high-level similarity between sequences of motions, that are not an exact matches in joint angles, timing, or ordering of actions. Even the same movements performed by the same person can vary in duration and speed. Similar motions are characterized by similar sets of actions that appear frequently. In this paper we introduce motion motifs and motion signatures that are a succinct but descriptive representation of motion sequences. We first break the motion sequences to short-term movements called motion words, and then cluster the words in a high-dimensional feature space to find motifs. Hence, motifs are words that are both common and descriptive, and their distribution represents the motion sequence. To cluster words and find motifs, the challenge is to define an effective feature space, where the distances among motion words are semantically meaningful, and where variations in speed and duration are handled. To this end, we use a deep neural network to embed the motion words into feature space using a triplet loss function. To define a signature, we choose a finite set of motion-motifs, creating a bag-of-motifs representation for the sequence. Motion signatures are agnostic to movement order, speed or duration variations, and can distinguish fine-grained differences between motions of the same class. We illustrate examples of characterizing motion sequences by motifs, and for the use of motion signatures in a number of applications.

Basketball is one of the world's most popular sports because of the agility and speed demonstrated by the players. This agility and speed makes designing controllers to realize robust control of basketball skills a challenge for physics-based character animation. The highly dynamic behaviors and precise manipulation of the ball that occur in the game are difficult to reproduce for simulated players. In this paper, we present an approach for learning robust basketball dribbling controllers from motion capture data. Our system decouples a basketball controller into locomotion control and arm control components and learns each component separately. To achieve robust control of the ball, we develop an efficient pipeline based on trajectory optimization and deep reinforcement learning and learn non-linear arm control policies. We also present a technique for learning skills and the transition between skills simultaneously. Our system is capable of learning robust controllers for various basketball dribbling skills, such as dribbling between the legs and crossover moves. The resulting control graphs enable a simulated player to perform transitions between these skills and respond to user interaction.

We present the first computational approach that can transform three-dimensional (3D) meshes, created by traditional modeling programs, directly into instructions for a computer-controlled knitting machine. Knitting machines are able to robustly and repeatably form knitted 3D surfaces from yarn but have many constraints on what they can fabricate. Given user-defined starting and ending points on an input mesh, our system incrementally builds a helix-free, quad-dominant mesh with uniform edge lengths, runs a tracing procedure over this mesh to generate a knitting path, and schedules the knitting instructions for this path in a way that is compatible with machine constraints. We demonstrate our approach on a wide range of 3D meshes.

As children begin to watch more television programming on systems that allow for interaction, such as tablets and videogame systems, there are different opportunities to engage them. For example, the traditional pseudo-interactive features that cue young children's participation in television viewing (e.g., asking a question and pausing for two seconds to allow for an answer) can be restructured to include correct response timing by the program or eventually even feedback. We performed three studies to examine the effects of accurate program response times, repeating unanswered questions, and providing feedback on the children's likelihood of response. We find that three- to five-year-old children are more likely to verbally engage with programs that wait for their response and repeat unanswered questions. However, providing feedback did not affect response rates for children in this age range.

Deformation of skin and muscle is essential for bringing an animated character to life. This deformation is difficult to animate in a realistic fashion using traditional techniques because of the subtlety of the skin deformations that must move appropriately for the character design. In this paper, we present an algorithm that generates natural, dynamic, and detailed skin deformation (movement and jiggle) from joint angle data sequences. The algorithm has two steps: identification of parameters for a quasi-static muscle deformation model, and simulation of skin deformation. In the identification step, we identify the model parameters using a musculoskeletal model and a short sequence of skin deformation data captured via a dense marker set. The simulation step first uses the quasi-static muscle deformation model to obtain the quasi-static muscle shape at each frame of the given motion sequence (slow jump). Dynamic skin deformation is then computed by simulating the passive muscle and soft tissue dynamics modeled as a mass–spring–damper system. Having obtained the model parameters, we can simulate dynamic skin deformations for subjects with similar body types from new motion data. We demonstrate our method by creating skin deformations for muscle co-contraction and external impacts from four different behaviors captured as skeletal motion capture data. Experimental results show that the simulated skin deformations are quantitatively and qualitatively similar to measured actual skin deformations.

Designing a unified framework for simulating a broad variety of human behaviors has proven to be challenging. In this article, we present an approach for control system design that can generate animations of a diverse set of behaviors including walking, running, and a variety of gymnastic behaviors. We achieve this generalization with a balancing strategy that relies on a new form of inverted pendulum model (IPM), which we call the momentum-mapped IPM (MMIPM). We analyze reference motion capture data in a pre-processing step to extract the motion of the MMIPM. To compute a new motion, the controller plans a desired motion, frame by frame, based on the current pendulum state and a predicted pendulum trajectory. By tracking this time-varying trajectory, the controller creates a character that dynamically balances, changes speed, makes turns, jumps, and performs gymnastic maneuvers.

Given a robust control system, physical simulation offers the potential for interactive human characters that move in realistic and responsive ways. In this article, we describe how to learn a scheduling scheme that reorders short control fragments as necessary at runtime to create a control system that can respond to disturbances and allows steering and other user interactions. These schedulers provide robust control of a wide range of highly dynamic behaviors, including walking on a ball, balancing on a bongo board, skateboarding, running, push-recovery, and breakdancing. We show that moderate-sized Q-networks can model the schedulers for these control tasks effectively and that those schedulers can be efficiently learned by the deep Q-learning algorithm.

We introduce a simple and effective deep learning approach to automatically generate natural looking speech animation that synchronizes to input speech. Our approach uses a sliding window predictor that learns arbitrary nonlinear mappings from phoneme label input sequences to mouth movements in a way that accurately captures natural motion and visual coarticulation effects. Our deep learning approach enjoys several attractive properties: it runs in real-time, requires minimal parameter tuning, generalizes well to novel input speech sequences, is easily edited to create stylized and emotional speech, and is compatible with existing animation retargeting approaches. One important focus of our work is to develop an effective approach for speech animation that can be easily integrated into existing production pipelines. We provide a detailed description of our end-to-end approach, including machine learning design decisions. Generalized speech animation results are demonstrated over a wide range of animation clips on a variety of characters and voices, including singing and foreign language input. Our approach can also generate on-demand speech animation in real-time from user speech input.

Animated characters are commonly used in children's television, movies, and applications. Artists seek to create characters that maximally engage their audiences and tailor these characters carefully. In order to examine the relationship between stylistic elements of animated characters and the target ages of their audiences, we performed a series of qualitative and quantitative studies. By using existing media, we determined that characters created for younger children have larger head height, larger eye height, and rounder eyes than those created for older children. However, we found no systematic differences by age when we had children express preferences for existing characters or create their own characters. These results suggest that current artistic trends do not accurately reflect the character design preferences of children.

Autism spectrum disorder (ASD) is characterized by unusual social communication and interaction. These traits are often targets for intervention, particularly computer-based interventions (CBIs). We examined whether interactive behaviors in children with autism could be influenced by modifying the facial characteristics of computer avatars and how behavior toward avatars compared to that toward video. Participants spoke with a therapist over a modified videoconferencing system that permitted manipulation of her appearance (i.e., using cartoon or more realistic avatars versus video) and motion (i.e., exaggerating or damping facial movements). We measured the participants' speech, gaze, and gestures. In the first study, we found that the appearance complexity of the avatar did not significantly affect any social interaction behaviors. However, the results of the second study suggest that exaggerated facial motion can improve nonverbal social behaviors, such as gaze and gesture. These findings have implications for character design in CBIs for ASD.

We present a new type of hydrostatic transmission that uses a hybrid air-water configuration, analogous to N+1 cable-tendon transmissions, using N hydraulic lines and 1 pneumatic line for a system with N degrees of freedom (DOFs). The common air-filled line preloads all DOFs in the system, allowing bidirectional operation of every joint. This configuration achieves the high stiffness of a water-filled transmission with half the number of bulky hydraulic lines. We implemented this transmission using pairs of rolling-diaphragm cylinders to form rotary hydraulic actuators, with a new design achieving a 600-percent increase in specific work density per cycle. These actuators were used to build a humanoid robot with two 4-DOF arms, connected via the hydrostatic transmission to an identical master. Stereo cameras mounted on a 2-DOF servo-controlled neck stream live video to the operator's head-mounted display, which in turn sends the real-time attitude of the operator's head to the neck servos in the robot. The operator is visually immersed in the robot's physical workspace, and through the bilateral coupling of the low-impedance hydrostatic transmission, directly feels interaction forces between the robot and external environment. We qualitatively assessed the performance of this system for remote object manipulation and use as a platform to safely study physical human-robot interaction.

Abstract We present a method to automatically convert videos and CG animations to stylized animated line drawings. Using a data‐driven approach, the animated drawings can follow the sketching style of a specific artist. Given an input video, we first extract edges from the video frames and vectorize them to curves. The curves are matched to strokes from an artist's library, while following the artist's stroke distribution and characteristics. The key challenge in this process is to match the large number of curves in the frames over time, despite topological and geometric changes, allowing to maintain temporal coherence in the output animation. We solve this problem using constrained optimization to build correspondences between tracked points and create smooth sheets over time. These sheets are then replaced with strokes from the artist's database to render the final animation. We evaluate our tracking algorithm on various examples and show stylized animation results based on various artists.

Animated characters are expected to fulfill a variety of social roles across different domains. To be successful and effective, these characters must display a wide range of personalities. Designers and animators create characters with appropriate personalities by using their intuition and artistic expertise. Our goal is to provide evidence-based principles for creating social characters. In this article, we describe the results of two experiments that show how exaggerated and damped facial motion magnitude influence impressions of cartoon and more realistic animated characters. In our first experiment, participants watched animated characters that varied in rendering style and facial motion magnitude. The participants then rated the different animated characters on extroversion, warmth, and competence, which are social traits that are relevant for characters used in entertainment, therapy, and education. We found that facial motion magnitude affected these social traits in cartoon and realistic characters differently. Facial motion magnitude affected ratings of cartoon characters’ extroversion and competence more than their warmth. In contrast, facial motion magnitude affected ratings of realistic characters’ extroversion but not their competence nor warmth. We ran a second experiment to extend the results of the first. In the second experiment, we added emotional valence as a variable. We also asked participants to rate the characters on more specific aspects of warmth, such as respectfulness, calmness, and attentiveness. Although the characters’ emotional valence did not affect ratings, we found that facial motion magnitude influenced ratings of the characters’ respectfulness and calmness but not attentiveness. These findings provide a basis for how animators can fine-tune facial motion to control perceptions of animated characters’ personalities.

Skinning algorithms that work across a broad range of character designs and poses are crucial to creating compelling animations. Currently, linear blend skinning (LBS) and dual quaternion skinning (DQS) are the most widely used, especially for real-time applications. Both techniques are efficient to compute and are effective for many purposes. However, they also have many well-known artifacts, such as collapsing elbows, candy wrapper twists, and bulging around the joints. Due to the popularity of LBS and DQS, it would be of great benefit to reduce these artifacts without changing the animation pipeline or increasing the computational cost significantly. In this paper, we introduce a new direct skinning method that addresses this problem. Our key idea is to pre-compute the optimized center of rotation for each vertex from the rest pose and skinning weights. At runtime, these centers of rotation are used to interpolate the rigid transformation for each vertex. Compared to other direct skinning methods, our method significantly reduces the artifacts of LBS and DQS while maintaining real-time performance and backwards compatibility with the animation pipeline.

Industrial knitting machines can produce finely detailed, seamless, 3D surfaces quickly and without human intervention. However, the tools used to program them require detailed manipulation and understanding of low-level knitting operations. We present a compiler that can automatically turn assemblies of high-level shape primitives (tubes, sheets) into low-level machine instructions. These high-level shape primitives allow knit objects to be scheduled, scaled, and otherwise shaped in ways that require thousands of edits to low-level instructions. At the core of our compiler is a heuristic transfer planning algorithm for knit cycles, which we prove is both sound and complete. This algorithm enables the translation of high-level shaping and scheduling operations into needle-level operations. We show a wide range of examples produced with our compiler and demonstrate a basic visual design interface that uses our compiler as a backend.

Research indicates that the facial expressions of animated characters and agents can influence people's perceptions and interactions with these entities. We designed an experiment to examine how an interactive animated avatar's facial expressiveness influences dyadic conversations between adults and the avatar. We animated the avatar in realtime using the tracked facial motion of a confederate. To adjust facial expressiveness, we damped and exaggerated the avatar's facial motion. We found that ratings of the avatar's extroversion were positively related to its expressiveness. However, impressions of the avatar's realism and naturalness worsened with increased expressiveness. We also found that the confederate was more influential when she appeared as the damped or exaggerated avatar. Adjusting the expressiveness of interactive animated avatars may be a simple way to influence people's social judgments and willingness to collaborate with animated avatars. These results have implications for using avatar facial expressiveness to improve the effectiveness of avatars in various contexts. Adjusting the expressiveness of interactive animated avatars may be a simple way to influence people's social judgments and willingness to collaborate with animated avatars.

Universities have traditionally drawn firm lines between classes in art and those in technology based fields such as computer science, placing them in separate departments, schools, and colleges. Human resources departments of companies have drawn similar lines between their "creative" and their "tech" employees, recruiting from different universities and creating different job titles and pay structures. In this talk, I will argue that the leaders of the next generation are going to be "hybrids" who each contribute to both sides of the art and tech divide and find it natural to interact and collaborate with co-workers with varied and mixed educational and work backgrounds. As an example of this style of education, I will report on an interdisciplinary course entitled Animation Art and Technology, which I have co-taught with Professor James Duesing in the School of Art for the past ten years at Carnegie Mellon University. The students are an interdisciplinary mix drawn from the traditional majors of art, computer science as well as a computer science and art bachelors degree program. The class produces four or five animations each semester, most of which have a substantive technical component, and the students are challenged to consider content innovation as equal to the technical aspects of their projects. Building on this style of education, Carnegie Mellon has recently created a program called IDeATE (Integrative Design, Arts and Technology Network) that offers a variety of minors and concentrations to students interested in blending art and technology in a variety of ways. 120 students enrolled in the first year indicating that the students recognize the need for these "hybrid" skill sets. As an example of the success of these efforts in industry, I will report on several research projects completed at Disney Research that would not have been possible without the highly collaborative efforts of teams of creative and techies.

Given the current profusion of devices for viewing media, video content created at one aspect ratio is often viewed on displays with different aspect ratios. Many previous solutions address this problem by retargeting or resizing the video, but a more general solution would re-edit the video for the new display. Our method employs the three primary editing operations: pan, cut, and zoom. We let viewers implicitly reveal what is important in a video by tracking their gaze as they watch the video. We present an algorithm that optimizes the path of a cropping window based on the collected eyetracking data, finds places to cut, and computes the size of the cropping window. We present results on a variety of video clips, including close-up and distant shots, and stationary and moving cameras. We conduct two experiments to evaluate our results. First, we eyetrack viewers on the result videos generated by our algorithm, and second, we perform a subjective assessment of viewer preference. These experiments show that viewer gaze patterns are similar on our result videos and on the original video clips, and that viewers prefer our results to an optimized crop-and-warp algorithm.

We present a perceptual control space for simulation of cloth that works with any physical simulator, treating it as a black box. The perceptual control space provides intuitive, art-directable control over the simulation behavior based on a learned mapping from common descriptors for cloth ( e.g. , flowiness, softness) to the parameters of the simulation. To learn the mapping, we perform a series of perceptual experiments in which the simulation parameters are varied and participants assess the values of the common terms of the cloth on a scale. A multi-dimensional sub-space regression is performed on the results to build a perceptual generative model over the simulator parameters. We evaluate the perceptual control space by demonstrating that the generative model does in fact create simulated clothing that is rated by participants as having the expected properties. We also show that this perceptual control space generalizes to garments and motions not in the original experiments.

This paper presents a novel solution for realtime generation of stylistic human motion that automatically transforms unlabeled, heterogeneous motion data into new styles. The key idea of our approach is an online learning algorithm that automatically constructs a series of local mixtures of autoregressive models (MAR) to capture the complex relationships between styles of motion. We construct local MAR models on the fly by searching for the closest examples of each input pose in the database. Once the model parameters are estimated from the training data, the model adapts the current pose with simple linear transformations. In addition, we introduce an efficient local regression model to predict the timings of synthesized poses in the output style. We demonstrate the power of our approach by transferring stylistic human motion for a wide variety of actions, including walking, running, punching, kicking, jumping and transitions between those behaviors. Our method achieves superior performance in a comparison against alternative methods. We have also performed experiments to evaluate the generalization ability of our data-driven model as well as the key components of our system.

We propose a shape editing method where the user creates geometric deformations using a set of semantic attributes, thus avoiding the need for detailed geometric manipulations. In contrast to prior work, we focus on continuous deformations instead of discrete part substitutions. Our method provides a platform for quick design explorations and allows non-experts to produce semantically guided shape variations that are otherwise difficult to attain. We crowdsource a large set of pairwise comparisons between the semantic attributes and geometry and use this data to learn a continuous mapping from the semantic attributes to geometry. The resulting map enables simple and intuitive shape manipulations based solely on the learned attributes. We demonstrate our method on large datasets using two different user interaction modes and evaluate its usability with a set of user studies.

Animated characters appear in applications for entertainment, education, and therapy. When these characters display appropriate emotions for their context, they can be particularly effective. Characters can display emotions by accurately mimicking the facial expressions and vocal cues that people display or by damping or exaggerating the emotionality of the expressions. In this work, we explored which of these strategies would be most effective for animated characters. We investigated the effects of altering the auditory and facial levels of expressiveness on emotion recognition accuracy and ratings of perceived emotional intensity and naturalness. We ran an experiment with emotion (angry, happy, sad), auditory emotion level (low, high), and facial motion magnitude (damped, unaltered, exaggerated) as within-subjects factors. Participants evaluated animations of a character whose facial motion matched that of an actress we tracked using an active appearance model. This method of tracking and animation can capture subtle facial motions in real-time, a necessity for many interactive animated characters. We manipulated auditory emotion level by asking the actress to speak sentences at varying levels, and we manipulated facial motion magnitude by exaggerating and damping the actress's spatial motion. We found that the magnitude of auditory expressiveness was positively related to emotion recognition accuracy and ratings of emotional intensity. The magnitude of facial motion was positively related to ratings of emotional intensity but negatively related to ratings of naturalness.

Interactive animated characters have the potential to engage and educate children, but there is little research on children's interactions with animated characters and real people. We conducted an experiment with 69 children between the ages of 4 and 10 years to investigate how they might engage in conversation differently if their interactive partner appeared as a cartoon character or as a person. A subset of the participants interacted with characters that displayed exaggerated and damped facial motion. The children completed two conversations with an adult confederate who appeared once as herself through video and once as a cartoon character. We measured how much the children spoke and compared their gaze and gesture patterns. We asked them to rate their conversations and indicate their preferred partner. There was no difference in children's conversation behavior with the cartoon character and the person on video, even among those who preferred the person and when the cartoon exhibited altered motion. These results suggest that children will interact with animated characters as they would another person.

When designing a robot for human-safety during direct physical interaction, one approach is to size the robot's actuators to be physically incapable of exerting damaging impulses, even during a controller failure. Merely lifting the arms against their own weight may consume the entire available torque budget, preventing the rapid and expressive movement required for anthropomorphic robots. To mitigate this problem, gravity-counterbalancing of the arms is a common tactic; however, most designs adopt a shoulder singularity configuration which, while favorable for simple counterbalance design, has a range of motion better suited for industrial robot arms. In this paper we present a shoulder design using a novel differential mechanism to counterbalance the arm while preserving an anthropomorphically favorable singularity configuration and natural range-of-motion. Furthermore, because the motors driving the shoulder are completely grounded, counterbalance masses or springs are easily placed away from the shoulder and low in the torso, improving mass distribution and balance. A robot arm using this design is constructed and evaluated for counterbalance efficacy and backdrivability under closed-loop force control.

We present a passive fluid transmission based on antagonist pairs of rolling diaphragm cylinders. The transmission fluid working volume is completely sealed, forming a closed, passive system, ensuring input-output symmetry and complete backdrivability. Rolling diaphragm-sealed cylinders provide leak-free operation without the stiction of a traditional sliding seal. Fluid pressure preloading allows for bidirectional operation and also serves to preload the gears or belts in the linear-to-rotary output coupler, eliminating system backlash end-to-end. A prototype transmission is built and tested for stiffness, bandwidth, and frictional properties using either air or water as working fluids. Torque transmission is smooth over the entire stroke and stiction is measured to be one percent of full-range torque or less. We also present a tendon-coupled design where the rolling diaphragm is inverted from its normal orientation; this design does not require shaft support bushings, tolerates misalignment, and can be made out of substantially soft materials. Actuator units and a passive transmission are demonstrated using this new soft cylinder design.

For compelling human-robot interaction, social gestures are widely believed to be important. This paper investigates the effects of adding gestures to a physical game between a human and a humanoid robot. Human participants repeatedly threw a ball to the robot, which attempted to catch it. If the catch was successful, the robot threw the ball back to the human. For half of the cases in which the catch was unsuccessful, the robot made a physical gesture, such as shrugging its shoulders, shaking its head, or throwing up its hands. In the other half of cases, no gestures were produced. We used questionnaires and smile detection to compare participants' feelings about the robot when it made gestures after failure versus when it did not. Participants smiled more and rated the robot as more engaging, responsive, and humanlike when it gestured. We conclude that social gesturing of a robot enhances physical interactions between humans and robots.

Creating facial animations that convey an animator’s intent is a difficult task because animation techniques are necessarily an approximation of the subtle motion of the face. Some animation techniques may result in linearization of the motion of vertices in space (blendshapes, for example), and other, simpler techniques may result in linearization of the motion in time. In this article, we consider the problem of animating smiles and explore how these simplifications in space and time affect the perceived genuineness of smiles. We create realistic animations of spontaneous and posed smiles from high-resolution motion capture data for two computer-generated characters. The motion capture data is processed to linearize the spatial or temporal properties of the original animation. Through perceptual experiments, we evaluate the genuineness of the resulting smiles. Both space and time impact the perceived genuineness. We also investigate the effect of head motion in the perception of smiles and show similar results for the impact of linearization on animations with and without head motion. Our results indicate that spontaneous smiles are more heavily affected by linearizing the spatial and temporal properties than posed smiles. Moreover, the spontaneous smiles were more affected by temporal linearization than spatial linearization. Our results are in accordance with previous research on linearities in facial animation and allow us to conclude that a model of smiles must include a nonlinear model of velocities.

We present an approach that takes multiple videos captured by social cameras---cameras that are carried or worn by members of the group involved in an activity---and produces a coherent "cut" video of the activity. Footage from social cameras contains an intimate, personalized view that reflects the part of an event that was of importance to the camera operator (or wearer). We leverage the insight that social cameras share the focus of attention of the people carrying them. We use this insight to determine where the important "content" in a scene is taking place, and use it in conjunction with cinematographic guidelines to select which cameras to cut to and to determine the timing of those cuts. A trellis graph representation is used to optimize an objective function that maximizes coverage of the important content in the scene, while respecting cinematographic guidelines such as the 180-degree rule and avoiding jump cuts. We demonstrate cuts of the videos in various styles and lengths for a number of scenarios, including sports games, street performances, family activities, and social get-togethers. We evaluate our results through an in-depth analysis of the cuts in the resulting videos and through comparison with videos produced by a professional editor and existing commercial solutions.

In many application areas, such as animation for pre-visualizing movie sequences and choreography for dance or other types of performance, only a high-level description of the desired scene is provided as input, either written or verbal. Such sparsity, however, lends itself well to the creative process, as the choreographer, animator or director can be given more choice and control of the final scene. Animating scenes with multi-character interactions can be a particularly complex process, as there are many different constraints to enforce and actions to synchronize. Our novel 'generate-and-rank' approach rapidly and semi-automatically generates data-driven multi-character interaction scenes from high-level graphical descriptions composed of simple clauses and phrases. From a database of captured motions, we generate a multitude of plausible candidate scenes. We then efficiently and intelligently rank these scenes in order to recommend a small but high-quality and diverse selection to the user. This set can then be refined by re-ranking or by generating alternatives to specific interactions. While our approach is applicable to any scenes that depict multi-character interactions, we demonstrate its efficacy for choreographing fighting scenes and evaluate it in terms of performance and the diversity and coverage of the results.

Animated characters are frequently used in television programs, movies, and video games, but relatively little is known about how their characteristics affect attention and viewer opinions. We used eyetracking and questionnaires to examine the role of visual complexity and animation style on viewing patterns and ratings of video-recorded and animated movie clips. We created videos of an actress performing and describing a series of actions with blocks. Of the videos, one set included regular HD recordings of the actress. The remaining video sets were animated using motion capture data from that actress for three characters: realistic, cartoon, and robot. Increased facial looking time correlated with unpleasantness ratings for individual characters and clips, determining that animation styles have an effect on both viewing patterns and audience members' subjective opinions of characters. In addition, the method described in this paper can expand future research on character animation.

Motion blur effects are commonly used in racing games [Sousa 2008; Vlachos 2008; Ritchie et al. 2010] to add a sense of realism as well as to minimize artifacts due to strobing and temporal aliasing [Glassner 1999]. Typically, motion blur computations are expensive, and for real-time applications, trade-offs are made between the quality of the effects and the computational cost. In this work, we wanted to understand: (i) the practical impact of the motion blur effect on the player experience; and (ii) whether the value gained by including the effect is worth the extra cost in computation, real-time performance, development time, etc. We studied the objective and subjective aspects of the player experience for Split Second: Velocity (Black Rock Studios, Disney), a high-speed racing game, in the presence and absence of the motion blur effect. We found that neither objective measures of participants' performance (e.g., time to complete a race) nor subjective measures of the player experience (e.g, enjoyment of a race, perceived speed) were affected, even though participants could reliably detect the presence of the motion blur effect. We conclude that motion blur effects, while useful for reducing artifacts and achieving a realistic 'look', do not significantly enhance the player experience.

Abstract The ability to accurately achieve performance capture of athlete motion during competitive play in near real‐time promises to revolutionize not only broadcast sports graphics visualization and commentary, but also potentially performance analysis, sports medicine, fantasy sports and wagering. In this paper, we present a highly portable, non‐intrusive approach for synthesizing human athlete motion in competitive game‐play with lightweight instrumentation of both the athlete and field of play. Our data‐driven puppetry technique relies on a pre‐captured database of short segments of motion capture data to construct a motion graph augmented with interpolated motions and speed variations. An athlete's performed motion is synthesized by finding a related action sequence through the motion graph using a sparse set of measurements from the performance, acquired from both worn inertial and global location sensors. We demonstrate the efficacy of our approach in a challenging application scenario, with a high‐performance tennis athlete wearing one or more lightweight body‐worn accelerometers and a single overhead camera providing the athlete's global position and orientation data. However, the approach is flexible in both the number and variety of input sensor data used. The technique can also be adopted for searching a motion graph efficiently in linear time in alternative applications.

Temporal segmentation of human motion into plausible motion primitives is central to understanding and building computational models of human motion. Several issues contribute to the challenge of discovering motion primitives: the exponential nature of all possible movement combinations, the variability in the temporal scale of human actions, and the complexity of representing articulated motion. We pose the problem of learning motion primitives as one of temporal clustering, and derive an unsupervised hierarchical bottom-up framework called hierarchical aligned cluster analysis (HACA). HACA finds a partition of a given multidimensional time series into m disjoint segments such that each segment belongs to one of k clusters. HACA combines kernel k-means with the generalized dynamic time alignment kernel to cluster time series data. Moreover, it provides a natural framework to find a low-dimensional embedding for time series. HACA is efficiently optimized with a coordinate descent strategy and dynamic programming. Experimental results on motion capture and video data demonstrate the effectiveness of HACA for segmenting complex motions and as a visualization tool. We also compare the performance of HACA to state-of-the-art algorithms for temporal clustering on data of a honey bee dance. The HACA code is available online.

We use a data-driven approach to study both style and abstraction in sketching of a human face. We gather and analyze data from a number of artists as they sketch a human face from a reference photograph. To achieve different levels of abstraction in the sketches, decreasing time limits were imposed -- from four and a half minutes to fifteen seconds. We analyzed the data at two levels: strokes and geometric shape. In each, we create a model that captures both the style of the different artists and the process of abstraction. These models are then used for a portrait sketch synthesis application. Starting from a novel face photograph, we can synthesize a sketch in the various artistic styles and in different levels of abstraction.

Recent advances in rendering and data-driven animation have enabled the creation of compelling characters with impressive levels of realism. While data-driven techniques can produce animations that are extremely faithful to the original motion, many challenging problems remain because of the high complexity of human motion. A better understanding of the factors that make human motion recognizable and appealing would be of great value in industries where creating a variety of appealing virtual characters with realistic motion is required. To investigate these issues, we captured thirty actors walking, jogging and dancing, and applied their motions to the same virtual character (one each for the males and females). We then conducted a series of perceptual experiments to explore the distinctiveness and attractiveness of these human motions, and whether characteristic motion features transfer across an individual's different gaits. Average faces are perceived to be less distinctive but more attractive, so we explored whether this was also true for body motion. We found that dancing motions were most easily recognized and that distinctiveness in one gait does not predict how recognizable the same actor is when performing a different motion. As hypothesized, average motions were always amongst the least distinctive and most attractive. Furthermore, as 50% of participants in the experiment were Caucasian European and 50% were Asian Korean, we found that the latter were as good as or better at recognizing the motions of the Caucasian actors than their European counterparts, in particular for dancing males, whom they also rated more highly for attractiveness.

Comics are a compelling, though complex, visual storytelling medium. Researchers are interested in the process of comic art creation to be able to automatically tell new stories, and also, summarize videos and catalog large collections of photographs for example. A primary organizing principle used by artists to lay out the components of comic art (panels, word bubbles, objects inside each panel) is to lead the viewer's attention along a deliberate visual route that reveals the narrative. If artists are successful in leading viewer attention, then their intended visual route would be accessible through recorded viewer attention, i.e., eyetracking data. In this paper, we conduct an experiment to verify if artists are successful in their goal of leading viewer gaze. We eyetrack viewers on images taken from comic books, as well as photographs taken by experts, amateur photographers and a robot. Our data analyses show that there is increased consistency in viewer gaze for comic pictures versus photographs taken by a robot and by amateur photographers, thus confirming that comic artists do indeed direct the flow of viewer attention.

This paper presents a method for the detection and recognition of social interactions in a day-long first-person video of u social event, like a trip to an amusement park. The location and orientation of faces are estimated and used to compute the line of sight for each face. The context provided by all the faces in a frame is used to convert the lines of sight into locations in space to which individuals attend. Further, individuals are assigned roles based on their patterns of attention. The rotes and locations of individuals are analyzed over time to detect and recognize the types of social interactions. In addition to patterns of face locations and attention, the head movements of the first-person can provide additional useful cues as to their attentional focus. We demonstrate encouraging results on detection and recognition of social interactions in first-person videos captured from multiple days of experience in amusement parks.

Endotracheal intubation (ETI) is a crucial medical procedure performed on critically ill patients. It involves insertion of a breathing tube into the trachea i.e. the windpipe connecting the larynx and the lungs. Often, this procedure is performed by the paramedics (aka providers) under challenging prehospital settings e.g. roadside, ambulances or helicopters. Successful intubations could be lifesaving, whereas, failed intubation could potentially be fatal. Under prehospital environments, ETI success rates among the paramedics are surprisingly low and this necessitates better training and performance evaluation of ETI skills. Currently, few objective metrics exist to quantify the differences in ETI techniques between providers. In this pilot study, we develop a quantitative framework for discriminating the kinematic characteristics of providers with different experience levels. The system utilizes statistical analysis on spatio-temporal multimodal features extracted from optical motion capture, accelerometers and electromyography (EMG) sensors. Our experiments involved three individuals performing intubations on a dummy, each with different levels of training. Quantitative performance analysis on multimodal features revealed distinctive differences among different skill levels. In future work, the feedback from these analysis could potentially be harnessed for enhanced ETI training.

We present a method for rapidly prototyping interactive robot skins using flexible 3D printed material and analogue air pressure sensors. We describe a set of building blocks for presenting affordances for different manipulations (twist, bend, stretch, etc.). Each building block is a hollow air chamber that can be printed as an integral part of the skin to easily add sensing capabilities over any broad area. Changes in volume caused by manipulating the chambers are captured using air pressure sensors; the sensors can be plugged in and removed, allowing rapid iteration on new designs. We demonstrate our method by prototyping three robot skins that attach to the Keepon Pro armature. With fully operational robot skins, we can study the dependencies between appearance, movement, and interactions at a deeper level than would previously be possible at the concept stage.

During conversations, speakers employ a number of verbal and nonverbal mechanisms to establish who participates in the conversation, when, and in what capacity. Gaze cues and mechanisms are particularly instrumental in establishing the participant roles of interlocutors, managing speaker turns, and signaling discourse structure. If humanlike robots are to have fluent conversations with people, they will need to use these gaze mechanisms effectively. The current work investigates people's use of key conversational gaze mechanisms, how they might be designed for and implemented in humanlike robots, and whether these signals effectively shape human-robot conversations. We focus particularly on whether humanlike gaze mechanisms might help robots signal different participant roles, manage turn-exchanges, and shape how interlocutors perceive the robot and the conversation. The evaluation of these mechanisms involved 36 trials of three-party human-robot conversations. In these trials, the robot used gaze mechanisms to signal to its conversational partners their roles either of two addressees, an addressee and a bystander, or an addressee and a nonparticipant. Results showed that participants conformed to these intended roles 97% of the time. Their conversational roles affected their rapport with the robot, feelings of groupness with their conversational partners, and attention to the task.

Drawing shapes by hand and manipulating computer-generated objects are the two dominant forms of animation. Though each medium has its own advantages, the techniques developed for one medium are not easily leveraged in the other medium because hand animation is two-dimensional, and inferring the third dimension is mathematically ambiguous. A second challenge is that the character is a consistent three-dimensional (3D) object in computer animation while hand animators introduce geometric inconsistencies in the two-dimensional (2D) shapes to better convey a character's emotional state and personality. In this work, we identify 3D proxies to connect hand-drawn animation and 3D computer animation. We present an integrated approach to generate three levels of 3D proxies: single-points, polygonal shapes, and a full joint hierarchy. We demonstrate how this approach enables one medium to take advantage of techniques developed for the other; for example, 3D physical simulation is used to create clothes for a hand-animated character, and a traditionally trained animator is able to influence the performance of a 3D character while drawing with paper and pencil.

Capturing the body movements of actors to create animations for movies, games, and VR applications has become standard practice, but finger motions are usually added manually as a tedious post-processing step. In this paper, we present a surprisingly simple method to automate this step for gesturing and conversing characters. In a controlled environment, we carefully captured and post-processed finger and body motions from multiple actors. To augment the body motions of virtual characters with plausible and detailed finger movements, our method selects finger motion segments from the resulting database taking into account the similarity of the arm motions and the smoothness of consecutive finger motions. We investigate which parts of the arm motion best discriminate gestures with leave-one-out cross-validation and use the result as a metric to select appropriate finger motions. Our approach provides good results for a number of examples with different gesture types and is validated in a perceptual experiment.

Marker-less motion capture is a challenging problem, particularly when only monocular video is available. We estimate human motion from monocular video by recovering three-dimensional controllers capable of implicitly simulating the observed human behavior and replaying this behavior in other environments and under physical perturbations. Our approach employs a state-space biped controller with a balance feedback mechanism that encodes control as a sequence of simple control tasks. Transitions among these tasks are triggered on time and on proprioceptive events ( e.g ., contact). Inference takes the form of optimal control where we optimize a high-dimensional vector of control parameters and the structure of the controller based on an objective function that compares the resulting simulated motion with input observations. We illustrate our approach by automatically estimating controllers for a variety of motions directly from monocular video. We show that the estimation of controller structure through incremental optimization and refinement leads to controllers that are more stable and that better approximate the reference motion. We demonstrate our approach by capturing sequences of walking, jumping, and gymnastics.

We present a method for designing and constructing rugged and soft multi-point sensors. Interactions applied to a soft material are reduced to structural units of deformation. These structures can then be embedded and instrumented anywhere inside a soft sensor. This simplification lets us design complex, durable sensors in easily manufacturable ways. In particular, we present a construction method of layering electronics between silicone pours to easily create sensors for arbitrary combinations of these deformations. We present several prototype sensors and discuss applications including toys, games, and therapy.

We present a new tongue input device, the tongue joystick, for use by an actor inside an articulated-head character costume. Using our device, the actor can maneuver through a dialogue tree, selecting clips of prerecorded audio to hold a conversation in the voice of the character. The device is constructed of silicone sewn with conductive thread, a unique method for creating rugged, soft, low-actuation force devices. This method has application for entertainment and assistive technology. We compare our device against other portable mouth input devices, showing it to be the fastest and most accurate in tasks mimicking our target application. Finally, we show early results of an actor inside an articulated-head costume using the tongue joystick to interact with a child.

Facial animation often falls short in conveying the nuances present in the facial dynamics of humans. In this article, we investigate the subtleties of the spatial and temporal aspects of eye blinks. Conventional methods for eye blink animation generally employ temporally and spatially symmetric sequences; however, naturally occurring blinks in humans show a pronounced asymmetry on both dimensions. We present an analysis of naturally occurring blinks that was performed by tracking data from high-speed video using active appearance models. Based on this analysis, we generate a set of key-frame parameters that closely match naturally occurring blinks. We compare the perceived naturalness of blinks that are animated based on real data to those created using textbook animation curves. The eye blinks are animated on two characters, a photorealistic model and a cartoon model, to determine the influence of character style. We find that the animated blinks generated from the human data model with fully closing eyelids are consistently perceived as more natural than those created using the various types of blink dynamics proposed in animation textbooks.

Motion capture technology generally requires that recordings be performed in a laboratory or closed stage setting with controlled lighting. This restriction precludes the capture of motions that require an outdoor setting or the traversal of large areas. In this paper, we present the theory and practice of using body-mounted cameras to reconstruct the motion of a subject. Outward-looking cameras are attached to the limbs of the subject, and the joint angles and root pose are estimated through non-linear optimization. The optimization objective function incorporates terms for image matching error and temporal continuity of motion. Structure-from-motion is used to estimate the skeleton structure and to provide initialization for the non-linear optimization procedure. Global motion is estimated and drift is controlled by matching the captured set of videos to reference imagery. We show results in settings where capture would be difficult or impossible with traditional motion capture systems, including walking outside and swinging on monkey bars. The quality of the motion reconstruction is evaluated by comparing our results against motion capture data produced by a commercially available optical system.

We present a novel approach for robust localization of multiple people observed using multiple cameras. We use this location information to generate sports visualizations, which include displaying a virtual offside line in soccer games, and showing players' positions and motion patterns. Our main contribution is the modeling and analysis for the problem of fusing corresponding players' positional information as finding minimum weight K-length cycles in complete K-partite graphs. To this end, we use a dynamic programming based approach that varies over a continuum of being maximally to minimally greedy in terms of the number of paths explored at each iteration. We present an end-to-end sports visualization framework that employs our proposed algorithm-class. We demonstrate the robustness of our framework by testing it on 60,000 frames of soccer footage captured over 5 different illumination conditions, play types, and team attire.

Videos of multi-player team sports provide a challenging domain for dynamic scene analysis. Player actions and interactions are complex as they are driven by many factors, such as the short-term goals of the individual player, the overall team strategy, the rules of the sport, and the current context of the game. We show that constrained multi-agent events can be analyzed and even predicted from video. Such analysis requires estimating the global movements of all players in the scene at any time, and is needed for modeling and predicting how the multi-agent play evolves over time on the field. To this end, we propose a novel approach to detect the locations of where the play evolution will proceed, e.g. where interesting events will occur, by tracking player positions and movements over time. We start by extracting the ground level sparse movement of players in each time-step, and then generate a dense motion field. Using this field we detect locations where the motion converges, implying positions towards which the play is evolving. We evaluate our approach by analyzing videos of a variety of complex soccer plays.

This paper presents a method for mapping captured human motion with stepping to a humanoid model, considering the current state and the controller behavior. The mapping algorithm modifies the joint angle, trunk and center of mass (COM) trajectories so that the motion can be tracked and desired contact states can be achieved. The mapping is performed in two steps. The first step modifies the joint angle and trunk trajectories to adapt to the robot kinematics and actual contact foot positions. The second step uses a predicted center of pressure (COP) to determine if the balance controller can successfully maintain the robot's balance, and if not, modifies the COM trajectory. Unlike most humanoid control work that handles motion synthesis and control separately, our COM trajectory modification is performed based on the behavior of the robot controller. We verify the approach in simulation using a captured Tai-chi motion that involves unstructured contact state changes.

Abstract The human shoulder complex is perhaps the most complicated joint in the human body being comprised of a set of three bones, muscles, tendons, and ligaments. Despite this anatomical complexity, computer graphics models for motion capture most often represent this joint as a simple ball and socket. In this paper, we present a method to determine a shoulder skeletal model that, when combined with standard skinning algorithms, generates a more visually pleasing animation that is a closer approximation to the actual skin deformations of the human body. We use a data‐driven approach and collect ground truth skin deformation data with an optical motion capture system with a large number of markers (200 markers on the shoulder complex alone). We cluster these markers during movement sequences and discover that adding one extra joint around the shoulder improves the resulting animation qualitatively and quantitatively yielding a marker set of approximately 70 markers for the complete skeleton. We demonstrate the effectiveness of our skeletal model by comparing it with ground truth data as well as with recorded video. We show its practicality by integrating it with the conventional rendering/animation pipeline.

We consume video content in a multitude of ways, including in movie theaters, on television, on DVDs and Blu-rays, online, on smart phones, and on portable media players. For quality control purposes, it is important to have a uniform viewing experience across these various platforms. In this work, we focus on voice synchronization, an aspect of video quality that is strongly affected by current post-production and transmission practices. We examined the synchronization of an actor's voice and lip movements in two distinct scenarios. First, we simulated the temporal mismatch between the audio and video tracks that can occur during dubbing or during broadcast. Next, we recreated the pitch changes that result from conversions between formats with different frame rates. We show, for the first time, that these audio visual mismatches affect viewer enjoyment. When temporal synchronization is noticeably absent, there is a decrease in the perceived performance quality and the perceived emotional intensity of a performance. For pitch changes, we find that higher pitch voices are not preferred, especially for male actors. Based on our findings, we advise that mismatched audio and video signals negatively affect viewer experience.

Virtual characters are much in demand for animated movies, games, and other applications. Rapid advances in performance capture and advanced rendering techniques have allowed the movie industry in particular to create characters that appear very human-like. However, with these new capabilities has come the realization that such characters are yet not quite “right.” One possible hypothesis is that these virtual humans fall into an “Uncanny Valley”, where the viewer's emotional response is repulsion or rejection, rather than the empathy or emotional engagement that their creators had hoped for. To explore these issues, we created three animated vignettes of an arguing couple with detailed motion for the face, eyes, hair, and body. In a set of perceptual experiments, we explore the relative importance of different anomalies using two different methods: a questionnaire to determine the emotional response to the full-length vignettes, with and without facial motion and audio; and a 2AFC (two alternative forced choice) task to compare the performance of a virtual “actor” in short clips (extracts from the vignettes) depicting a range of different facial and body anomalies. We found that the facial anomalies are particularly salient, even when very significant body animation anomalies are present.

We present a technique for learning clothing models that enables the simultaneous animation of thousands of detailed garments in real-time. This surprisingly simple conditional model learns and preserves the key dynamic properties of a cloth motion along with folding details. Our approach requires no a priori physical model, but rather treats training data as a "black box." We show that the models learned with our method are stable over large time-steps and can approximately resolve cloth-body collisions. We also show that within a class of methods, no simpler model covers the full range of cloth dynamics captured by ours. Our method bridges the current gap between skinning and physical simulation, combining benefits of speed from the former with dynamic effects from the latter. We demonstrate our approach on a variety of apparel worn by male and female human characters performing a varied set of motions typically used in video games ( e.g. , walking, running, jumping, etc. ).

This paper presents a control framework for humanoid robots that uses all joints simultaneously to track motion capture data and maintain balance. The controller comprises two main components: a balance controller and a tracking controller. The balance controller uses a regulator designed for a simplified humanoid model to obtain the desired input to keep balance based on the current state of the robot. The simplified model is chosen so that a regulator can be designed systematically using, for example, optimal control. An example of such controller is a linear quadratic regulator designed for an inverted pendulum model. The desired inputs are typically the center of pressure and/or torques of some representative joints. The tracking controller then computes the joint torques that minimize the difference from desired inputs as well as the error from desired joint accelerations to track the motion capture data, considering exact full-body dynamics. We demonstrate that the proposed controller effectively reproduces different styles of storytelling motion using dynamics simulation considering limitations in hardware.

In this paper, our aim is to determine factors that influence the perceived sex of virtual characters. In Experiment 1, four different model types were used: highly realistic male and female models, an androgynous character, and a point light walker. Three different types of motion were applied to all models: motion captured male and female walks, and neutral synthetic walks. We found that both form and motion influence sex perception for these characters: for neutral synthetic motions, form determines perceived sex, whereas natural motion affects the perceived sex of both androgynous and realistic forms. These results indicate that the use of neutral walks is better than creating ambiguity by assigning an incongruent motion. In Experiment 2 we investigated further the influence of body shape and motion on realistic male and female models and found that adding stereotypical indicators of sex to the body shapes influenced sex perception. Also, that exaggerated female body shapes influences sex judgements more than exaggerated male shapes. These results have implications for variety and realism when simulating large crowds of virtual characters.

Temporal segmentation of human motion into actions is a crucial step for understanding and building computational models of human motion. Several issues contribute to the challenge of this task. These include the large variability in the temporal scale and periodicity of human actions, as well as the exponential nature of all possible movement combinations. We formulate the temporal segmentation problem as an extension of standard clustering algorithms. In particular, this paper proposes aligned cluster analysis (ACA), a robust method to temporally segment streams of motion capture data into actions. ACA extends standard kernel k-means clustering in two ways: (1) the cluster means contain a variable number of features, and (2) a dynamic time warping (DTW) kernel is used to achieve temporal invariance. Experimental results, reported on synthetic data and the Carnegie Mellon Motion Capture database, demonstrate its effectiveness.

In this video presentation we introduce a 50 degrees of freedom humanoid robot, CB -ComputationalBrain[1]. CB is a humanoid robot created for exploring the underlying processing of the human brain while dealing with the real world. We place our investigations within real world contexts, as humans do. In so doing, we focus on utilising a system that is closer to humans - in sensing, kinematics configuration and performance. We present a full-body compliance controller that was developed for the motion control of our humanoid robot [2]. Our initial experimentation on our system includes: 1) full-body compliant control - physical interactions/balancing/motion control; 2) the integrated visual ocular-motor responses; 3) perception and control - reaching, foveation, and active object recognition; 4) our studies of Central Pattern Generator for walking.

In this paper, we present a data-driven technique for synthesizing skin deformation from skeletal motion. We first create a database of dynamic skin deformations by recording the motion of the surface of the skin with a very large set of motion capture markers. We then build a statistical model of the deformations by dividing them into two parts: static and dynamic. Static deformations are modeled as a function of pose. Dynamic deformations are caused by the actions of the muscles as they move the joints and the inertia of muscles and fat. We approximate these effects by fitting a set of dynamic equations to the pre-recorded data. We demonstrate the viability of this approach by generating skin deformations from the skeletal motion of an actor. We compare the generated animation both to synchronized video of the actor and to ground truth animation created directly from the large marker set.

In late 2006, Nintendo released a new game controller, the Wiimote, which included a three-axis accelerometer. Since then, a large variety of novel applications for these controllers have been developed by both independent and commercial developers. We add to this growing library with three performance interfaces that allow the user to control the motion of a dynamically simulated, animated character through the motion of his or her arms, wrists, or legs. For comparison, we also implement a traditional joystick/button interface. We assess these interfaces by having users test them on a set of tracks containing turns and pits. Two of the interfaces (legs and wrists) were judged to be more immersive and were better liked than the joystick/button interface by our subjects. All three of the Wiimote interfaces provided better control than the joystick interface based on an analysis of the failures seen during the user study.

In this paper, we present a methodology for recognizing seated postures using data from pressure sensors installed on a chair. Information about seated postures could be used to help avoid adverse effects of sitting for long periods of time, or to predict a user’s activities as input to a humancomputer interface. Our approach to posture recognition avoids the use of expensive hardware and complex prediction algorithms while providing recognition for users, for whom the classifier is not trained, using a near-optimal sensor placement strategy. We evaluated the performance of our technology in a series of empirical evaluations including (1) cross-validation experiments (classification accuracy of 87% for ten postures), and (2) a physical deployment of our system (78% classification accuracy).

We present an extension to Lagrangian finite element methods to allow for large plastic deformations of solid materials. These behaviors are seen in such everyday materials as shampoo, dough, and clay as well as in fantastic gooey and blobby creatures in special effects scenes. To account for plastic deformation, we explicitly update the linear basis functions defined over the finite elements during each simulation step. When these updates cause the basis functions to become ill-conditioned, we remesh the simulation domain to produce a new high-quality finite-element mesh, taking care to preserve the original boundary. We also introduce an enhanced plasticity model that preserves volume and includes creep and work hardening/softening. We demonstrate our approach with simulations of synthetic objects that squish, dent, and flow. To validate our methods, we compare simulation results to videos of real materials.

In this paper, we present a technique for generating animation from a variety of user-defined constraints. We pose constraint-based motion synthesis as a maximum a posterior (MAP) problem and develop an optimization framework that generates natural motion satisfying user constraints. The system automatically learns a statistical dynamic model from motion capture data and then enforces it as a motion prior. This motion prior, together with user-defined constraints, comprises a trajectory optimization problem. Solving this problem in the low-dimensional space yields optimal natural motion that achieves the goals specified by the user. We demonstrate the effectiveness of this approach by generating whole-body and facial motion from a variety of spatial-temporal constraints.

Many compelling applications would become feasible if novice users had the ability to synthesize high quality human motion based only on a simple sketch and a few easily specified constraints. We approach this problem by representing the desired motion as an interpolation of two time-scaled paths through a motion graph. The graph is constructed to support interpolation and pruned for efficient search. We use an anytime version of A* search to find a globally optimal solution in this graph that satisfies the user's specification. Our approach retains the natural transitions of motion graphs and the ability to synthesize physically realistic variations provided by interpolation. We demonstrate the power of this approach by synthesizing optimal or near optimal motions that include a variety of behaviors in a single motion.

In this paper, we explore how dynamic visual cues can be used to create accessible and meaningful social interfaces without raising expectations beyond what is achievable with current technology. Our approach is inspired by research in perceptual causality, which suggests that simple displays in motion can evoke high-level social and emotional content. For our exploration, we iteratively designed and implemented a public social interface using abstraction and motion as design elements. Our interface communicated simple social and emotional content such as displaying happiness when there is high social interaction in the environment. Our qualitative evaluations showed that people frequently and repeatedly interacted with the interface while they tried to make sense of the underlying social content. They also shared their models with others, which led to more social interaction in the environment.

Recent developments in humanoid robotics have made possible a vision of robots in everyday use in the home and workplace. However, little is known about how we should design social interactions with humanoid robots. We explored how co-operation versus competition in a game shaped people's perceptions of ASIMO. We found that in the co-operative interaction, people found the robot more sociable and more intellectual than in the competitive interaction while people felt more positive and were more involved in the task in the competitive condition than in the co-operative condition. Our poster presents these findings with the supporting theoretical background.

Recent developments in humanoid robotics have made possible technologically advanced robots and a vision for their everyday use as assistants in the home and workplace. Nonetheless, little is known about how we should design interactions with humanoid robots. In this paper, we argue that adaptation for user attributes (in particular gender) and task structure (in particular a competitive vs. a cooperative structure) are key design elements. We experimentally demonstrate how these two elements affect the user's social perceptions of ASIMO after playing an interactive video game with him

During dynamic activities, the surface of the human body moves in many subtle but visually significant ways: bending, bulging, jiggling, and stretching. We present a technique for capturing and animating those motions using a commercial motion capture system and approximately 350 markers. Although the number of markers is significantly larger than that used in conventional motion capture, it is only a sparse representation of the true shape of the body. We supplement this sparse sample with a detailed, actor-specific surface model. The motion of the skin can then be computed by segmenting the markers into the motion of a set of rigid parts and a residual deformation (approximated first as a quadratic transformation and then with radial basis functions). We demonstrate the power of this approach by capturing flexing muscles, high frequency motions, and abrupt decelerations on several actors. We compare these results both to conventional motion capture and skinning and to synchronized video of the actors.

Despite the recent achievements in stable dynamic walking for many humanoid robots, relatively little navigation autonomy has been achieved. In particular, the ability to autonomously select foot placement positions to avoid obstacles while walking is an important step towards improved navigation autonomy for humanoids. We present a footstep planner for the Honda ASIMO humanoid robot that plans a sequence of footstep positions to navigate toward a goal location while avoiding obstacles. The possible future foot placement positions are dependent on the current state of the robot. Using a finite set of state-dependent actions, we use an A* search to compute optimal sequences of footstep locations up to a time-limited planning horizon. We present experimental results demonstrating the robot navigating through both static and dynamic known environments that include obstacles moving on predictable trajectories.

This paper introduces an approach to performance animation that employs video cameras and a small set of retro-reflective markers to create a low-cost, easy-to-use system that might someday be practical for home use. The low-dimensional control signals from the user's performance are supplemented by a database of pre-recorded human motion. At run time, the system automatically learns a series of local models from a set of motion capture examples that are a close match to the marker locations captured by the cameras. These local models are then used to reconstruct the motion of the user as a full-body animation. We demonstrate the power of this approach with real-time control of six different behaviors using two video cameras and a small set of retro-reflective markers. We compare the resulting animation to animation from commercial motion capture equipment with a full set of markers.

In this paper, we investigate whether it is possible to develop a measure that quantifies the naturalness of human motion (as defined by a large database). Such a measure might prove useful in verifying that a motion editing operation had not destroyed the naturalness of a motion capture clip or that a synthetic motion transition was within the space of those seen in natural human motion. We explore the performance of mixture of Gaussians (MoG), hidden Markov models (HMM), and switching linear dynamic systems (SLDS) on this problem. We use each of these statistical models alone and as part of an ensemble of smaller statistical models. We also implement a Naive Bayes (NB) model for a baseline comparison. We test these techniques on motion capture data held out from a database, keyframed motions, edited motions, motions with noise added, and synthetic motion transitions. We present the results as receiver operating characteristic (ROC) curves and compare the results to the judgments made by subjects in a user study.

We present a vision-based performance interface for controlling animated human characters. The system interactively combines information about the user's motion contained in silhouettes from three viewpoints with domain knowledge contained in a motion capture database to produce an animation of high quality. Such an interactive system might be useful for authoring, for teleconferencing, or as a control interface for a character in a game. In our implementation, the user performs in front of three video cameras; the resulting silhouettes are used to estimate his orientation and body configuration based on a set of discriminative local features. Those features are selected by a machine-learning algorithm during a preprocessing step. Sequences of motions that approximate the user's actions are extracted from the motion database and scaled in time to match the speed of the user's motion. We use swing dancing, a complex human motion, to demonstrate the effectiveness of our approach. We compare our results to those obtained with a set of global features, Hu moments, and ground truth measurements from a motion capture system.

This paper presents a novel algorithm for synthesizing and editing video of natural phenomena that exhibit continuous flow patterns. The algorithm analyzes the motion of textured particles in the input video along user-specified flow lines, and synthesizes seamless video of arbitrary length by enforcing temporal continuity along a second set of user-specified flow lines. The algorithm is simple to implement and use. We used this technique to edit video of water-falls, rivers, flames, and smoke.

Optimization is an appealing way to compute the motion of an animated character because it allows the user to specify the desired motion in a sparse, intuitive way. The difficulty of solving this problem for complex characters such as humans is due in part to the high dimensionality of the search space. The dimensionality is an artifact of the problem representation because most dynamic human behaviors are intrinsically low dimensional with, for example, legs and arms operating in a coordinated way. We describe a method that exploits this observation to create an optimization problem that is easier to solve. Our method utilizes an existing motion capture database to find a low-dimensional space that captures the properties of the desired behavior. We show that when the optimization problem is solved within this low-dimensional subspace, a sparse sketch can be used as an initial guess and full physics constraints can be enabled. We demonstrate the power of our approach with examples of forward, vertical, and turning jumps; with running and walking; and with several acrobatic flips.

Even such simple tasks as placing a box on a shelf are difficult to animate, because the animator must carefully position the character to satisfy geometric and balance constraints while creating motion to perform the task with a natural-looking style. In this paper, we explore an approach for animating characters manipulating objects that combines the power of path planning with the domain knowledge inherent in data-driven, constraint-based inverse kinematics. A path planner is used to find a motion for the object such that the corresponding poses of the character satisfy geometric, kinematic, and posture constraints. The inverse kinematics computation of the character's pose resolves redundancy by biasing the solution toward natural-looking poses extracted from a database of captured motions. Having this database greatly helps to increase the quality of the output motion. The computed path is converted to a motion trajectory using a model of the velocity profile. We demonstrate the effectiveness of the algorithm by generating animations across a wide range of scenarios that cover variations in the geometric, kinematic, and dynamic models of the character, the manipulated object, and obstacles in the scene.

To present an accurate and compelling view of a new environment, architectural and urban planning applications both require animations of people. Ideally, these animations would be easy for a non-programmer to construct, just as buildings and streets can be modeled by an architect or artist using commercial modeling software. In this paper we explore an approach for generating reactive path following based on the user's examples of the desired behavior. The examples are used to build a model of the desired reactive behavior. The model is combined with reactive control methods to produce natural 2D pedestrian trajectories. The system then automatically generates 3D pedestrian locomotion using motion capture resequencing algorithms. We discuss the accuracy of the model of pedestrian motion and show that simple direction primitives can be recorded and used to build natural, reactive, path-following behaviors.

In this paper, we present a method for controlling a motorized, string-driven marionette using motion capture data from human actors. The motion data must be adapted for the marionette because its kinematic and dynamic properties differ from those of the human actor in degrees of freedom, limb length, workspace, mass distribution, sensors, and actuators. This adaptation is accomplished via an inverse kinematics algorithm that takes into account marker positions, joint motion ranges, string constraints, and potential energy. We also apply a feedforward controller to prevent extraneous swings of the hands. Experimental results show that our approach enables the marionette to perform motions that are qualitatively similar to the original human motion capture data.

We present a wireless virtual reality system and a prototype full body Tai Chi training application. Our primary contribution is the creation of a virtual reality system that tracks the full body in a working volume of 4 meters by 5 meters by 2.3 meters high to produce an animated representation of the user with 42 degrees of freedom. This - combined with a lightweight (<3 pounds) belt-worn video receiver and head-mounted display - provides a wide area, untethered virtual environment that allows exploration of new application areas. Our secondary contribution is our attempt to show that user interface techniques made possible by such a system can improve training for a full body motor task. We tested several immersive techniques, such as providing multiple copies of a teacher's body positioned around the student and allowing the student to superimpose his body directly over the virtual teacher None of these techniques proved significantly better than mimicking traditional Tai Chi instruction, where we provided one virtual teacher directly in front of the student. We consider the implications of these findings for future motion training tasks.

No abstract available.

Using the pre-recorded human motion and trajectory tracking, we can control the motion of a humanoid robot for free-space, upper body gestures. However, the number of degrees of freedom, range of joint motion, and achievable joint velocities of today's humanoid robots are far more limited than those of the average human subject. In this paper, we explore a set of techniques for limiting human motion of upper body gestures to that achievable by a Sarcos humanoid robot located at ATR. We assess the quality of the results by comparing the motion of the human actor to that of the robot, both visually and quantitatively.

I've now been Editor-in-Chief of ACM TOG for two full years. We've made a number of changes in TOG recently. Perhaps the most visible change is the new format which began with January 2002. The format change is to more closely approximate that of ACM SIGGRAPH.The July 2002 issue of TOG will be the Proceedings of ACM SIGGRAPH. These papers will be conditionally accepted at the SIGGRAPH Program Committee meeting in March 2002 and then will undergo a second reviewing phase in which a referee will ensure that the authors have altered the paper to address the concerns of the original reviewers. This referee's job will be much like that of the associate editor in the normal TOG review process.As you have no doubt noticed, publication of TOG has not kept pace with the calendar. However, the number of submissions has increased substantially (57 in 2001 versus 34 in 2000) and the time until the authors receive the first decision on their paper has been reduced to less than four months for almost every paper. We should be publishing on time by the next regular issue, October 2002, and will then be able to start building a backlog of accepted papers at ACM to keep the issues on schedule.This improvement has been made possible by the hard work of the Associate Editors listed in the front cover of the journal and by the reviewers who are listed at the end of this issue. I'm very grateful to both groups for their significant help throughout the year.I hope that the next year sees another 50% or larger increase in the number of submissions. I would be happy to have twice as much work to do so please keep sending those strong submissions our way.

No abstract available.

Real-time control of three-dimensional avatars is an important problem in the context of computer games and virtual environments. Avatar animation and control is difficult, however, because a large repertoire of avatar behaviors must be made available, and the user must be able to select from this set of behaviors, possibly with a low-dimensional input device. One appealing approach to obtaining a rich set of avatar behaviors is to collect an extended, unlabeled sequence of motion data appropriate to the application. In this paper, we show that such a motion database can be preprocessed for flexibility in behavior and efficient search and exploited for real-time avatar control. Flexibility is created by identifying plausible transitions between motion segments, and efficient search through the resulting graph structure is obtained through clustering. Three interface techniques are demonstrated for controlling avatar motion using this data structure: the user selects from a set of available choices, sketches a path through an environment, or acts out a desired motion in front of a video camera. We demonstrate the flexibility of the approach through four different applications and compare the avatar motion to directly recorded human motion.

In this paper, we describe a method for realistically animating ductile fracture in common solid materials such as plastics and metals. The effects that characterize ductile fracture occur due to interaction between plastic yielding and the fracture process. By modeling this interaction, our ductile fracture method can generate realistic motion for a much wider range of materials than could be realized with a purely brittle model. This method directly extends our prior work on brittle fracture [O'Brien and Hodgins, SIGGRAPH 99]. We show that adapting that method to ductile as well as brittle materials requires only a simple to implement modification that is computationally inexpensive. This paper describes this modification and presents results demonstrating some of the effects that may be realized with it.

We present a method for designing truss structures, a common and complex category of buildings, using non-linear optimization. Truss structures are ubiquitous in the industrialized world, appearing as bridges, towers, roof supports and building exoskeletons, yet are complex enough that modeling them by hand is time consuming and tedious. We represent trusses as a set of rigid bars connected by pin joints, which may change location during optimization. By including the location of the joints as well as the strength of individual beams in our design variables, we can simultaneously optimize the geometry and the mass of structures. We present the details of our technique together with examples illustrating its use, including comparisons with real structures.

This paper describes a technique for generating transitions between simulated behaviors. By parametrizing individual basis behaviors, we can design control systems such that the exit states of one leaves the simulated character in a valid initial state for the next. The parametrization allows one to generate a wide variety of motions from a single basis behavior. The nesting of the input and output states allows easy transitions between behaviors and generation of many behaviors from a small set of basis behaviors. We demonstrate this approach with four basis behaviors: leaping, tumbling, landing, and balancing. We demonstrate transitions by generating a diverse set of gymnastic behaviors, including a standing broad jump, vertical leap, forward somersault, backward somersault, back handspring, and various platform dives.

Construction toys are a superb medium for geometric models. We argue that such toys, suitably instrumented or sensed, could be the inspiration for a new generation of easy-to-use, tangible modeling systems—especially if the tangible modeling is combined with graphical-interpretation techniques for enhancing nascent models automatically. The three key technologies needed to realize this idea are embedded computation, vision-based acquisition, and graphical interpretation. We sample these technologies in the context of two novel modeling systems: physical building blocks that self-describe, interpret, and decorate the structures into which they are assembled; and a system for scanning, interpreting, and animating clay figures.

In this paper, we introduce techniques for animating explosions and their effects. The primary effect of an explosion is a disturbance that causes a shock wave to propagate through the surrounding medium. The disturbance determines the behavior of nearly all other secondary effects seen in explosion. We simulate the propagation of an explosion through the surrounding air using a computational fluid dynamics model based on the equations for compressible, viscous flow. To model the numerically stable formation of shocks along blast wave fronts, we employ an integration method that can handle steep pressure gradients without introducing inappropriate damping. The system includes two-way coupling between solid objects and surrounding fluid. Using this technique, we can generate a variety of effects including shaped explosive charges, a projectile propelled from a chamber by an explosion, and objects damaged by a blast. With appropriate rendering techniques, our explosion model can be used to create such visual effects as fireballs, dust clouds, and the refraction of light caused by a blast wave.

I'm very pleased to have the opportunity to take over as editor-in-chief after Holly Rushmeier's strong leadership of the past three years. ACM has made a big effort to get TOG back on schedule after the delays caused by changing to a new publication system. With this issue, we are almost on schedule again and the backlog of accepted papers is significantly reduced. I hope to further shorten the time the return reviews to authors and to get accepted paper to readers. As part of that effort, I have asked Steve Fortune for Bell Labs, John Hart from Washington State, Joe Marks from MERL, and Jorg Peters from the University of Florida to be new members of the editorial board to replace those who retired with the end of Holly's term. I expect that we'll see significant changes in the role of journals during my three-year term. Right now, journals utilize peer reviews and a revision cycle to provide accreditation to research, but this process is tied to a particular distribution medium and requires financial support in the form of subscription fees. With the increasing use of web-baed publishing, I expect that the accreditation and distribution functions will begin to separate. Accreditation will continue to play an important role in scientific reputations and academic careers, while the publication media will become far more diverse. This diversification is particularly important for graphics because many of its important results are not represented well on paper. TOG has moved in this direction with a web page that is expertly maintained by Eric Haines and contains occasional supplemental material (http://www.acm.org./tog/). If TOG is to represent the true breadth of graphics, however, we will have to make much more of an effort in this direction. During the next three years, we will almost certainly see significant changes in graphics as well. Clearly, the field is expanding beyond the creation of beautiful and useful images and image sequences. New emphases include increasingly realistic physical modeling, interactive worlds, and nonvisual modalities. Graphics is a fascinating field, in part because researchers are able to adopt and adapt new ideas from so many other fields: art, vision, physics, materials science, control, optimization and biomechanics. I look forward to the innovative and creative work that the community will submit to TOG during the next three years.

In this paper, we augment existing techniques for simulating flex-ible objects to include models for crack initiation and propagation in three-dimensional volumes. By analyzing the stress tensors com-puted over a finite element model, the simulation determines where cracks should initiate and in what directions they should propagate. We demonstrate our results with animations of breaking bowls, cracking walls, and objects that fracture when they collide. By varying the shape of the objects, the material properties, and the initial conditions of the simulations, we can create strikingly dif-ferent effects ranging from a wall that shatters when it is hit by a wrecking ball to a bowl that breaks in two when it is dropped on edge.

Computer animations often lack the subtle environmental changes that should occur due to the actions of the characters. Squealing car tires usually leave no skid marks, airplanes rarely leave jet trails in the sky, and most runners leave no footprints. In this paper, we describe a simulation model of ground surfaces that can be deformed by the impact of rigid body models of animated characters. To demonstrate the algorithms, we show footprints made by a runner in sand, mud, and snow as well as bicycle tire tracks, a bicycle crash, and a falling runner. The shapes of the footprints in the three surfaces are quite different, but the effects were controlled through only five essentially independent parameters. To assess the realism of the resulting motion, we compare the simulated footprints to human footprints in sand.

High contrast images are common in night scenes and other scenes that include dark shadows and bright light sources. These scenes are difficult to display because their contrasts greatly exceed the range of most display devices for images. As a result, the image constrasts are compressed or truncated, obscuring subtle textures and details. Humans view and understand high contrast scenes easily, “adapting” their visual response to avoid compression or truncation with no apparent loss of detail. By imitating some of these visual adaptation processes, we developed methods for the improved display of high-contrast images. The first builds a display image from several layers of lighting and surface properties. Only the lighting layers are compressed, drastically reducing contrast while preserving much of the image detail. This method is practical only for synthetic images where the layers can be retained from the rendering process. The second method interactively adjusts the displayed image to preserve local contrasts in a small “foveal” neighborhood. Unlike the first method, this technique is usable on any image and includes a new tone reproduction operator. Both methods use a sigmoid function for contrast compression. This function has no effect when applied to small signals but compresses large signals to fit within an asymptotic limit. We demonstrate the effectiveness of these approaches by comparing processed and unprocessed images.

Human figures have been animated using a variety of geometric models, including stick figures, polygonal models and NURBS-based models with muscles, flexible skin or clothing. This paper reports on experimental results indicating that a viewer's perception of motion characteristics is affected by the geometric model used for rendering. Subjects were shown a series of paired motion sequences and asked if the two motions in each pair were the same or different. The motion sequences in each pair were rendered using the same geometric model. For the three types of motion variation tested, sensitivity scores indicate that subjects were better able to observe changes with the polygonal model than they were with the stick-figure model.

This paper describes an algorithm for automatically adapting existing simulated behaviors to new characters. Animating a new character is difficult because a control system tuned for one character will not, in general, work on a character with different limb lengths, masses, or moments of inertia. The algorithm presented here adapts the control system to a new character in two stages. First, the control system parameters are scaled based on the sizes, masses, and moments of inertia of the new and the original characters. Then a subset of the parameters is fine-tuned using a search process based on simulated annealing. To demonstrate the effectiveness of this approach, we animate the running motion of a woman, child, and imaginary character by modifying the control system for a man. We also animate the bicycling motion of a second imaginary character by modifying the control system for a man. We evaluate the results of this approach by comparing the motion of the simulated human runners with video of an actual child and with data for men, women, and children in the literature. In addition to adapting a control system for a new model, this approach can also be used to adapt the control system in an on-line fashion to produce a physically realistic metamorphosis from the original to the new model while the morphing character is performing the behavior. We demonstrate this on-line adaptation with a morph from a man to a woman over a period of twenty seconds.

Article Design galleries: a general approach to setting parameters for computer graphics and animation Share on Authors: J. Marks MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MA MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MAView Profile , B. Andalman Harvard Univ. Harvard Univ.View Profile , P. A. Beardsley MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MA MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MAView Profile , W. Freeman MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MA MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MAView Profile , S. Gibson MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MA MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MAView Profile , J. Hodgins Georgia Tech. Georgia Tech.View Profile , T. Kang CMU CMUView Profile , B. Mirtich MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MA MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MAView Profile , H. Pfister MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MA MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MAView Profile , W. Ruml MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MA MERL - A Mitsubishi Electric Research Laboratory, 201 Broadway, Cambridge, MAView Profile , K. Ryall Harvard Univ. Harvard Univ.View Profile , J. Seims Univ. of Washington Univ. of WashingtonView Profile , S. Shieber Harvard Univ. Harvard Univ.View Profile Authors Info & Claims SIGGRAPH '97: Proceedings of the 24th annual conference on Computer graphics and interactive techniquesAugust 1997 Pages 389–400https://doi.org/10.1145/258734.258887Online:03 August 1997Publication History 346citation2,992DownloadsMetricsTotal Citations346Total Downloads2,992Last 12 Months156Last 6 weeks16 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access

An important attribute in the specification of many compute-intensive applications is "time".Simulation of interactive virtual environments is one such domain.There is a mismatch between the synchronization and consistency guarantees needed by such applications (which are temporal in nature) and the guarantees offered by current shared memory systems.Consequently, programming such applications using standard shared memory style synchronization and communication is cumbersome.Furthermore, such applications offer opportunities for relaxing both the synchronization and consistency requirements along the temporal dimension.In this work, we develop a temporal programming model that is more intuitive for the development of applications that need temporal correctness guarantees.This model embodies two mechanisms: "delta consistency" -a novel time-based correctness criterion to govern the shared memory access guarantees, and a companion "temporal synchronization" -a mechanism for thread synchronization along the time axis.These mechanisms are particularly appropriate for expressing the requirements in interactive application domains. in addition to the temporal programming model, we develop efficient explicit communication mechanisms that aggressively push the data out to "future" consumers to hide the read miss latency at the receiving end.We implement these mechanisms on a cluster of workstations in a software distributed shared memory architecture called "Beehive?Using a virtual environment application as the driver, we show the efficacy of the proposed mechanisms in meeting the real time requirements of such applications.

This paper describes the control of a simulated three-dimensional model of a human runner. The rigid body model has seventeen segments and thirty controlled degrees of freedom. The dynamic parameters of the simulation were computed using data measured from humans to produce a model that is biomechanically realistic within the constraints of a rigid body approximation. We developed locomotion control algorithms that allow the model to run at a variety of speeds and turn to face in an arbitrary direction on the plane. The realism of the simulation is evaluated by comparing the computed motion to that of humans performing similar maneuvers. We perform the comparison qualitatively with real video images and quantitatively with biomechanical data.

Abstract The motion of a human platform diver was simulated using a dynamic model and a control system. The dynamic model has 32 actuated degrees of freedom and dynamic parameters within the range of those reported in the literature for humans. The control system uses algorithms for balance, jumping, and twisting to initiate the dive, sequences of desired values for proportional‐derivative servos to perform the aerial portion of the dice, and a state machine to sequence the actions throughout the dice. The motion of the simulated diver closely resembles video footage of dices performed by human athletes. The control and simulation techniques presented in this paper are useful for providing realistic motion for synthetic actors in computer animations and virtual environments and may some day be useful for analysis of sports performance.

We describe a method for modeling the dynamic behavior of splashing fluids. The model simulates the behavior of a fluid when objects impact or float on its surface. The forces generated by the objects create waves and splashes on the surface of the fluid. To demonstrate the realism and limitations of the model, images from a computer-generated animation are presented and compared with video frames of actual splashes occurring under similar initial conditions.>

Many robot applications require traversing uneven or unmodeled terrain. This paper explores strategies for one class of difficult terrain: slippery surfaces. We evaluate several reflexive responses to a slip using a dynamically simulated, three-dimensional, bipedal robot. We explore two kinds of reaction strategies. One strategy continues the step, in which the slip occurred. The other lifts the slipping foot and repositions the legs for another attempt. The most successful strategy positions the legs in a fixed triangular configuration on the step following a slip.

Birds, fish, and many other animals travel as a flock, school, or herd. Animals in these groups must remain in close proximity while avoiding collisions with neighbors and with obstacles. We would like to reproduce this behavior for groups of artificial creatures with significant dynamics. In this paper we describe an algorithm for creatures that move as a group and evaluate the performance of the algorithm with three simulated systems: legged robots, human-like bicycle riders, and point-mass systems. Both the legged robots and the bicyclists are dynamic simulations that must control balance, facing direction, and forward speed as well as movement with the group. The point-mass systems have minimal dynamics and are included to facilitate our understanding of the effects of the dynamics on the performance of the algorithms.

This paper describes algorithms for the animation of men and women performing three dynamic athletic behaviors: running, bicycling, and vaulting.We animate these behaviors using control algorithms that cause a physically realistic model to perform the desired maneuver.For example, control algorithms allow the simulated humans to maintain balance while moving their arms, to run or bicycle at a variety of speeds, and to perform a handspring vault.Algorithms for group behaviors allow a number of simulated bicyclists to ride as a group while avoiding simple patterns of obstacles.We add secondary motion to the animations with springmass simulations of clothing driven by the rigid-body motion of the simulated human.For each simulation, we compare the computed motion to that of humans performing similar maneuvers both qualitatively through the comparison of real and simulated video images and quantitatively through the comparison of simulated and biomechanical data.

We describe algorithms for the control of locomotion of a simulated planar model of a human. The rigid body model has seventeen degrees of freedom and dynamic parameters approximating those given in the literature for humans. The locomotion control algorithms allow the model to run at a variety of speeds. The motion produced by the simulation can be used to produce realistic computer animations and interactive, synthetic actors for virtual environments. These algorithms also provide insight into how robotic mechanisms that resemble the complexity of humans might be controlled.>

Algorithms are described for generating biped run-to-walk and walk-to-run transitions. Simple algorithms were developed for those gait transitions by designing strategies for transforming the set of oscillations corresponding to one gait into the set corresponding to the other. For example, the gait transition from running to walking removed energy from the vertical oscillation by shortening the leg during the stance phase to initiate walking. This approach was tested by implementing run-to-walk and walk-to-run transitions on a planar biped robot.>

This paper is about the use of control algorithms to animate dynamic legged locomotion. Control could free the animator from specifying the details of joint and limb motion while producing both physically realistic and natural looking results. We implemented computer animations of a biped robot, a quadruped robot, and a kangaroo. Each creature was modeled as a linked set of rigid bodies with compliant actuators at its joints. Control algorithms regulated the running speed, organized use of the legs, and maintained balance. All motions were generated by numerically integrating equations of motion derived from the physical models. The resulting behavior included running at various speeds, traveling with several gaits (run, trot, bound, gallop, and hop), jumping, and traversing simple paths. Whereas the use of control permitted a variety of physically realistic animated behavior to be generated with limited human intervention, the process of designing the control algorithms was not automated: the algorithms were "tweaked" and adjusted for each new creature.

The task of controlling step length in the context of a dynamic biped robot that actively balances itself as it runs is discussed. Three methods for controlling step length, each of which adjusts a different parameter of the running cycle, are discussed. The adjusted parameters are forward running speed, running height, and duration of ground contact. All three control methods are successful in manipulating step length in laboratory experiments, but the method that adjusted forward speed provided the widest range of step lengths with accurate control of step length. The three methods for controlling step length manipulated the dynamics of the system so the feet could be placed on the available footholds without disturbing the system's balance. An alternative approach which ignores balance for a single step, placing the foot directly on the desired foothold, and recovering balance later is described.>

For a legged system to operate on rough terrain, it must place its feet on footholds that provide good support and traction. This can be done by adjusting the step length, the distance traveled between successive steps. The author explores three methods for controlling step length: adjusting the duration of flight, the duration of stance, or the forward velocity. She has tested these three methods for step length control on a planar, two-legged machine in the laboratory. The strategies which adjust flight duration and forward velocity produce similar accuracy in following a pattern of footholds, but adjusting forward velocity allows a greater range of step lengths. Changes in stance duration are not large enough to produce a large change in step length. The author has used these methods to enable the biped to climb up and down a short flight of stairs.>