In this paper, we deal with the estimation of body and head poses (i.e orientations) in surveillance videos, and we make three main contributions. First, we address this issue as a joint model adaptation problem in a semi-supervised framework. Second, we propose to leverage the adaptation on multiple information sources (external labeled datasets, weak labels provided by the motion direction, data structure manifold), and in particular, on the coupling at the output level of the head and body classifiers, accounting for the restriction in the configurations that the head and body pose can jointly take. Third, we propose a kernel-formulation of this principle that can be efficiently solved using a global optimization scheme. The method is applied to body and head features computed from automatically extracted body and head location tracks. Thorough experiments on several datasets demonstrate the validity of our approach, the benefit of the coupled adaptation, and that the method performs similarly or better than a state-of-the-art algorithm.

In surveillance videos, cues such as head or body pose provide important information for analyzing people's behavior and interactions. In this paper we propose an approach that jointly estimates body location and body pose in monocular surveillance video. Our approach is based on tracks derived by multi-object tracking. First, body pose classification is conducted using sparse representation technique on each frame of the tracks, generating (noisy) observation on body poses. Then, both location and body pose in 3D space are estimated jointly in a particle filtering framework by utilizing a soft coupling of body pose with the movement. The experiments show that the proposed system successfully tracks body position and pose simultaneously in many scenarios. The output of the system can be used to perform further analysis on behaviors and interactions.

Estimating 3D pose similarity is a fundamental problem on 3D motion data. Most previous work calculates L2-like distance of joint orientations or coordinates, which does not sufficiently reflect the pose similarity of human perception. In this paper, we present a new pose distance metric. First, we propose a new rich pose feature set called Geometric Pose Descriptor (GPD). GPD is more effective in encoding pose similarity by utilizing features on geometric relations among body parts, as well as temporal information such as velocities and accelerations. Based on GPD, we propose a semisupervised distance metric learning algorithm called Regularized Distance Metric Learning with Sparse Representation (RDSR), which integrates information from both unsupervised data relationship and labels. We apply the proposed pose distance metric to applications of motion transition decision and content-based pose retrieval. Quantitative evaluations demonstrate that our method achieves better results with only a small amount of human labels, showing that the proposed pose distance metric is a promising building block for various 3D-motion related applications.

Abstract Traditional pose similarity functions based on joint coordinates or rotations often do not conform to human perception. We propose a new perceptual pose distance:Relational Geometric Distancethat accumulates the differences over a set of features that reflects the geometric relations between different body parts. An extensive relational geometric feature pool that contains a large number of potential features is defined, and the features effective for pose similarity estimation are selected using a set of labeled data by Adaboost. The extensive feature pool guarantees that a wide diversity of features is considered, and the boosting ensures that the selected features are optimized when used jointly. Finally, the selected features form a pose distance function that can be used for novel poses. Experiments show that our method outperforms others in emulating human perception in pose similarity. Our method can also adapt to specific motion types and capture the features that are important for pose similarity of a certain motion type. Copyright © 2009 John Wiley & Sons, Ltd.

Abstract We propose a simple and effective framework to deal with the problem of synthesizing interactive and competitive motions while reflecting the interactions. Two nontrivial issues are addressed in synthesizing two‐character motions in competitive environment: how to reveal the embedded routines while keeping visual reality and how to build interactive models based on singly captured motions. To solve these issues, we employ a hierarchical framework: the finite state machine (FSM) controls the state transition in the higher layer, and the hybrid approach controls the action selection in the lower layer. The proposed approach contains two folds: first, a rule‐based control scheme is proposed to simulate routine steps based on statistical analysis. Second, the interactive models are designed for simulating dense interactions between two players. The Relevance Vector Machine (RVM) algorithm is adopted to select attack styles and coupled with motion transition graph to determine combination blows. Here we apply the proposed framework of hybrid paradigm to boxing sport as an example. Copyright © 2009 John Wiley & Sons, Ltd.

Many types of shape descriptors have been proposed for 2D shape analysis, but most of them consist of component features that are not adapted to specific problems. This has two drawbacks. First, computation is wasted on the irrelevant components; second, the accuracy is impaired. This paper proposes an effective method that generates compact descriptors adapted to specific problems in hand, where each component of the new descriptor is a linear combination of the components in some classic descriptors. A progressive strategy is used to construct and select the most suitable linear combinations in successive rounds, where a variant of Adaboost is employed to ensure the optimum of the selected combinations in each round. Experiments show that our method effectively generates adaptive and compact descriptors for typical applications such as shape classification and retrieval.

Abstract In this paper we propose an approach, which allows the users to synthesize cartoon clips according to the perspective of the background image. In order to construct the cartoons smoothly, the character's edge distance and motion direction distance are demonstrated to be the factors affecting the human perception in similarity evaluation, and utilized in cartoon clips synthesis. When applying the generated cartoons to the background image, in which the perspective exists, the size of the character is coordinated according to the scaling factor calculated from the vanishing line. The experiment results demonstrate that our approach can synthesize the cartoon clips more smoothly compared with other single frame reusing strategies. The generated cartoons, which are applied to the background image, can be accepted by the human perception well. Copyright © 2008 John Wiley & Sons, Ltd.

A novel approach is proposed that extends the classical background subtraction method to extract silhouettes from videos in real time with dynamic viewpoint variation caused by camera movement. First, manifold learning is used to model the background under viewpoint variations. Then, for each new frame, the background image corresponding to the same viewpoint is synthesized on the fly by examining the local neighborhood on the manifold, and the silhouette is extracted via background subtraction. An extension is also presented to generate stabilized silhouettes at any fixed viewpoint within the training range. Experiments show that our approach can efficiently extract accurate silhouettes in complex situations while maintaining a low noise level.

Video based 3D reconstruction of human motion plays an important role in many applications. We implement a system that robustly reconstructs 3D human motion from markerless videos taken by a single camera. Our system only requires a desktop PC and a mainstream camera, and doesn't involve complex camera calibration, making it easy to implement and widely accessible in daily uses such as human computer interaction or entertainment.

Abstract In this paper, we propose a novel approach, which reuses Traditional Chinese Cartoon to create new animations. In order to extract the cartoon character precisely, a segmentation method based on edge detection is implemented. Before reusing the data, a lower‐ dimensional space of the cartoon data is constructed by ISOmap. The character's gesture difference calculated by optical flow is combined with character's edge difference through a novel distance function, which is controlled by a weight parameter. The animation is created by reordering the existing data into a sequence, which is the shortest path between two designated data in the space. Our approach utilizes image processing, computer vision, and machine learning in cartoon creation and the experiment results demonstrate that the animation's quality can be effectively improved by the fusion of these techniques. Copyright © 2007 John Wiley & Sons, Ltd.