This paper describes a new method for acquiring physically realistic hand manipulation data from multiple video streams. The key idea of our approach is to introduce a composite motion control to simultaneously model hand articulation, object movement, and subtle interaction between the hand and object. We formulate video-based hand manipulation capture in an optimization framework by maximizing the consistency between the simulated motion and the observed image data. We search an optimal motion control that drives the simulation to best match the observed image data. We demonstrate the effectiveness of our approach by capturing a wide range of high-fidelity dexterous manipulation data. We show the power of our recovered motion controllers by adapting the captured motion data to new objects with different properties. The system achieves superior performance against alternative methods such as marker-based motion capture and kinematic hand motion tracking.

This paper presents a robust physics-based motion control system for realtime synthesis of human grasping. Given an object to be grasped, our system automatically computes physics-based motion control that advances the simulation to achieve realistic manipulation with the object. Our solution leverages prerecorded motion data and physics-based simulation for human grasping. We first introduce a data-driven synthesis algorithm that utilizes large sets of prerecorded motion data to generate realistic motions for human grasping. Next, we present an online physics-based motion control algorithm to transform the synthesized kinematic motion into a physically realistic one. In addition, we develop a performance interface for human grasping that allows the user to act out the desired grasping motion in front of a single Kinect camera. We demonstrate the power of our approach by generating physics-based motion control for grasping objects with different properties such as shapes, weights, spatial orientations, and frictions. We show our physics-based motion control for human grasping is robust to external perturbations and changes in physical quantities.

This paper introduces a new generative statistical model that allows for human motion analysis and synthesis at both semantic and kinematic levels. Our key idea is to decouple complex variations of human movements into finite structural variations and continuous style variations and encode them with a concatenation of morphable functional models. This allows us to model not only a rich repertoire of behaviors but also an infinite number of style variations within the same action. Our models are appealing for motion analysis and synthesis because they are highly structured, contact aware , and semantic embedding . We have constructed a compact generative motion model from a huge and heterogeneous motion database (about two hours mocap data and more than 15 different actions). We have demonstrated the power and effectiveness of our models by exploring a wide variety of applications, ranging from automatic motion segmentation, recognition, and annotation, and online/offline motion synthesis at both kinematics and behavior levels to semantic motion editing. We show the superiority of our model by comparing it with alternative methods.

This article shows how statistical motion priors can be combined seamlessly with physical constraints for human motion modeling and generation. The key idea of the approach is to learn a nonlinear probabilistic force field function from prerecorded motion data with Gaussian processes and combine it with physical constraints in a probabilistic framework. In addition, we show how to effectively utilize the new model to generate a wide range of natural-looking motions that achieve the goals specified by users. Unlike previous statistical motion models, our model can generate physically realistic animations that react to external forces or changes in physical quantities of human bodies and interaction environments. We have evaluated the performance of our system by comparing against ground-truth motion data and alternative methods.

This paper presents a generative human motion model for synthesis, retargeting, and editing of personalized human motion styles. We first record a human motion database from multiple actors performing a wide variety of motion styles for particular actions. We then apply multilinear analysis techniques to construct a generative motion model of the form x = g(a, e) for particular human actions, where the parameters a and e control "identity" and "style" variations of the motion x respectively. The new modular representation naturally supports motion generalization to new actors and/or styles. We demonstrate the power and flexibility of the multilinear motion models by synthesizing personalized stylistic human motion and transferring the stylistic motions from one actor to another. We also show the effectiveness of our model by editing stylistic motion in style and/or identity space.

This paper presents an efficient model-based approach for automatic human motion registration, which builds temporal correspondences between structurally similar but distinctive motion examples. The key idea of the model-based registration process is to construct a parameterized motion model from a set of preregistered motion examples. With such a model, we can register an input motion with the parameterized motion model by continuously deforming the model to best match the input motion. We formulate the registration process in a gradient-based nonlinear optimization framework by minimizing an objective function that measures differences between the input motion and deforming motion. We also develop a multi-resolution optimization process to efficiently estimate the model parameters as well as the temporal correspondences between the input motion and deforming motion. We demonstrate the performance of our approach by testing the algorithm on difficult motion sequences and comparing with alternative approaches.

This article presents a new motion model deformable motion models for human motion modeling and synthesis. Our key idea is to apply statistical analysis techniques to a set of precaptured human motion data and construct a low-dimensional deformable motion model of the form x = M (α, γ), where the deformable parameters α and γ control the motion's geometric and timing variations, respectively. To generate a desired animation, we continuously adjust the deformable parameters' values to match various forms of user-specified constraints. Mathematically, we formulate the constraint-based motion synthesis problem in a Maximum A Posteriori (MAP) framework by estimating the most likely deformable parameters from the user's input. We demonstrate the power and flexibility of our approach by exploring two interactive and easy-to-use interfaces for human motion generation: direct manipulation interfaces and sketching interfaces.