Depth sensor based 3D human motion estimation hardware such as Kinect has made interactive applications more popular recently. However, it is still challenging to accurately recognize postures from a single depth camera due to the inherently noisy data derived from depth images and self-occluding action performed by the user. In this paper, we propose a new real-time probabilistic framework to enhance the accuracy of live captured postures that belong to one of the action classes in the database. We adopt the Gaussian Process model as a prior to leverage the position data obtained from Kinect and marker-based motion capture system. We also incorporate a temporal consistency term into the optimization framework to constrain the velocity variations between successive frames. To ensure that the reconstructed posture resembles the accurate parts of the observed posture, we embed a set of joint reliability measurements into the optimization framework. A major drawback of Gaussian Process is its cubic learning complexity when dealing with a large database due to the inverse of a covariance matrix. To solve the problem, we propose a new method based on a local mixture of Gaussian Processes, in which Gaussian Processes are defined in local regions of the state space. Due to the significantly decreased sample size in each local Gaussian Process, the learning time is greatly reduced. At the same time, the prediction speed is enhanced as the weighted mean prediction for a given sample is determined by the nearby local models only. Our system also allows incrementally updating a specific local Gaussian Process in real time, which enhances the likelihood of adapting to run-time postures that are different from those in the database. Experimental results demonstrate that our system can generate high quality postures even under severe self-occlusion situations, which is beneficial for real-time applications such as motion-based gaming and sport training.

We propose a new method to compute compatible triangulations of two polygons in order to create a smooth geometric transformation between them. Compared with existing methods, our approach creates triangulations of better quality, that is, triangulations with fewer long thin triangles and Steiner points. This results in visually appealing morphing when transforming the shape from one to another. Our method consists of three stages. First, we compatibly decompose the target and source polygons into a set of sub-polygons, in which each source sub-polygon is triangulated. Second, we map the triangulation of a source sub-polygon onto the corresponding sub-polygon of the target polygon using linear transformation, thereby generating the compatible meshes between the source and the target. Third, we refine the compatible meshes, which can create better quality planar shape morphing with detailed textures. Experimental results show that our method can create compatible meshes of higher quality compared with existing methods, which facilitates smoother morphing process. The proposed algorithm is robust and computationally efficient. It can be applied to produce convincing transformations such as interactive 2D animation creation and special effects in movies.

Recent work has shown that depth image based 3D posture estimation hardware such as Kinect has made interactive applications more popular. However, it is still challenging to accurately recognize postures from a single depth camera due to the inherently noisy data derived from depth images and self-occluding action performed by the user. While previous research has shown that data-driven methods can be used to reconstruct the correct postures, they usually require a large posture database, which greatly limit the usability for systems with constrained hardware such as game console. To solve this problem, we present a new probabilistic framework to enhance the accuracy of the postures live captured by Kinect. We adopt the Gaussian Process model as a prior to leverage position data obtained from Kinect and marker-based motion capture system. We also incorporate a temporal consistency term into the optimization framework to constrain the velocity variations between successive frames. To ensure that the reconstructed posture resembles the observed input data from Kinect when its tracking result is good, we embed joint reliability into the optimization framework. Experimental results demonstrate that our system can generate high quality postures even under severe self-occlusion situations, which is beneficial for real-time posture based applications such as motion-based gaming and sport training.

ABSTRACT Human motion variation synthesis is important for crowd simulation and interactive applications to enhance synthesis quality. In this paper, we propose a novel generative probabilistic model to synthesize variations of human motion. Our key idea is to model the conditional distribution of each joint via a multivariate Gaussian process model, namely semiparametric latent factor model (SLFM). SLFM can effectively model the correlations between degrees of freedom (DOFs) of joints rather than dealing with each DOF separately as implemented in existing methods. A detailed evaluation is performed to show that the proposed approach can effectively synthesize variations of different types of motions. Motions generated by our method show a richer variation compared with existing ones. Finally, our user study shows that the synthesized motion has a similar level of naturalness to captured human motions. Our method is best applied in computer games and animations to introduce motion variations. Copyright © 2014 John Wiley & Sons, Ltd.

To search for a particular motion from a large database, a user-friendly and efficient retrieval mechanism is essential. In this paper, we propose a human motion retrieval system based on sparse coding and touch less interactions. Compared with existing methods that involve vector quantization, sparse coding leads to a more compact and discriminative representation. Motion comparison based on sparse representation greatly improves the effectiveness of motion retrieval in terms of accuracy and speed. With the recent advancement in human motion tracking hardware such as Kinect, our retrieval system allows the user to specify the query motion by performing it directly. Besides, the user interacts with the retrieval system interactively using gestures so no controller is required thus delivering a natural user interface.