Previous studies on human-pose estimation (HPE) rely on the design of factors to represent underlying probability distributions that model human poses. However, designing those factors manually is laborious. Moreover, manually designed factors might not represent underlying probability distributions properly. In this paper, we utilize feedforward neural networks (NNs) to design factors of our previous work on HPE and build an NN-based HPE system. We first propose a mapping that converts a Bayesian network to a feedforward NN. Then, the system is built based on the proposed mapping that consists of two steps: 1) structure identification and 2) parameter learning. In the structure identification, we develop a bottom-up approach to build a feedforward NN while preserving a Bayesian-network structure. In the parameter learning, we create a part-based approach to learn synaptic weights by decomposing a feedforward NN into parts. Using the proposed mapping, our previous work of an action-mixture model (AMM) for HPE is converted to a feedforward NN called NN-AMM. Based on the concept of distributed representation, NN-AMM is further modified to a scalable feedforward NN called NND-AMM. The NN-based HPE system is then built by using viewpoint-and-shape-feature-histogram features extracted from 3-D-point-cloud input and NND-AMM to estimate 3-D human poses. The results showed that the proposed mapping could design AMM factors automatically. NND-AMM could provide more accurate human-pose estimates with fewer hidden neurons than both AMM and NN-AMM could. Both NN-AMM and NND-AMM could adapt to different types of input, showing the adaptability of using feedforward NNs to design factors.

We address the problem of partially occluded object detection by implementing a model which includes latent visibility flags that are attached to cells and parts of a Deformable Part Model (DPM) [1]. A visibility flag indicates whether an image portion is part of a target object or part of an occluder. To compute the visibility flags and the score of the detector simultaneously, we maximize a concave objective function that is composed of the following four terms: (1) the detection scores of visible cells and parts, (2) a cell-to-cell consistency term which encourages neighboring cells to have the same visibility flags, (3) a cell-to-part consistency term which encourages compatible labeling among overlapping cells and parts, and (4) a penalty term for cells and parts that are labeled as occluded. The maximization of the concave objective function is done using the Alternating Direction Method of Multipliers (ADMM). By removing scores of occluded cells and parts from the final detection score we significantly improve detection performance on partially occluded pedestrians. In experiments we show that our system outperforms the standard DPM and other state-of-art methods on a benchmark database of partially occluded pedestrians.

Previous studies on human-pose estimation rely on the design of factors to represent underlying probability distributions. However, designing factors is laborious and yet, the designed factors may not represent the underlying probability distributions. In this paper, we propose to use a neural network to automatically design factors in one of the existing models called the action-mixture model (AMM). Factors that are designed automatically by neural networks can be adapted to different situations. The semantic meaning of random variables in AMM can be transferred to a neural network, rendering the semantic meaning of hidden neurons transparent to users. The design process consists of two stages: structure identification and parameter learning. In the structure identification, we propose a bottom-up approach to build a neural network while preserving the structure of AMM. In the parameter learning, we propose a part-based approach to learn synaptic weights by decomposing a neural network into parts. Synaptic weights that have been learnt in one part can be used as initial weights for learning synaptic weights in another part. Based on the concept of distributed representation, the proposed two-stage, neural-network-based design process is used to design a scalable neural network to realize an AMM. Experimental results showed that the scalable neural network outperformed AMM and some existing works.

Estimating human poses is an important step towards developing robots that can understand human motion. Since a human is highly articulated, changing viewpoints of sensors on robots can improve the accuracy of human-pose estimation. We propose a two-phase approach that determines the best viewpoint of a depth sensor for human-pose estimation. The proposed approach measures the quality of potential viewpoints and selects one of them as the best viewpoint for each human pose. Based on the quality of viewpoints, human poses can be directly mapped to the best viewpoint without reconstructing the human body. Thus, the proposed approach provides a discriminative mapping to determine the best viewpoint for estimating different human poses. To measure the quality of a potential viewpoint, the viewpoint is first instantiated by representing the depth sensor of the viewpoint using the finite projective camera model. The quality of the viewpoint is expressed in terms of the error of humanpose estimates. A mapping is derived by minimizing the error in a human-pose estimate among different viewpoints. The proposed two-phase approach has been evaluated on a benchmark database. Experimental results showed that the best viewpoint for a human pose could be determined by evaluating the quality of potential viewpoints. The mean error and standard deviation of human-pose estimates were reduced by using the best viewpoint determined by the proposed two-phase approach.

This paper focuses on human-pose estimation using a stationary depth sensor. The main challenge concerns reducing the feature ambiguity and modeling human poses in high-dimensional human-pose space because of the curse of dimensionality. We propose a 3-D-point-cloud system that captures the geometric properties (orientation and shape) of the 3-D point cloud of a human to reduce the feature ambiguity, and use the result from action classification to discover low-dimensional manifolds in human-pose space in estimating the underlying probability distribution of human poses. In the proposed system, a 3-D-point-cloud feature called viewpoint and shape feature histogram (VISH) is proposed to extract the 3-D points from a human and arrange them into a tree structure that preserves the global and local properties of the 3-D points. A nonparametric action-mixture model (AMM) is then proposed to model human poses using low-dimensional manifolds based on the concept of distributed representation. Since human poses estimated using the proposed AMM are in discrete space, a kinematic model is added in the last stage of the proposed system to model the spatial relationship of body parts in continuous space to reduce the quantization error in the AMM. The proposed system has been trained and evaluated on a benchmark dataset. Computer-simulation results showed that the overall error and standard deviation of the proposed 3-D-point-cloud system were reduced compared with some existing approaches without action classification.

Estimating human poses is an important step towards developing robots that can understand human motions and improving their cognitive capabilities. This paper presents a geometric feature for estimating human poses from a 3D point cloud input. The proposed feature can be considered as an extension of the idea of visual features, such as color/edge, of color/grayscale images, and it contains the geometric structure of the point cloud. It is derived by arranging the 3D points into a tree structure, which preserves the global and local properties of the 3D points. Shown experimentally, the tree structure (spatial ordering) is particularly important for estimating human poses (i.e., articulated objects). The 3D orientation (pan, tilt and yaw angles) and shape features are then extracted from each node in the tree to describe the geometric distribution of the 3D points. The proposed feature has been evaluated on a benchmark dataset and compared with two existing geometric features. Experimental results show that the proposed feature has the lowest overall error in human-pose estimation.

This paper presents a 3D-point-cloud system that extracts a 3D-point-cloud feature (VISH) from the observation of a depth sensor to reduce feature/depth ambiguity and estimates human poses using the result of action classification and a kinematic model. Based on the concept of distributed representation, a non-parametric action-mixture model is proposed in the system to represent high-dimensional human-pose space using low-dimensional manifolds in searching human poses. In each manifold, the probability distribution is estimated by the similarity of features. The distributions in the manifolds are then redistributed according to the stationary distribution of a Markov chain that models the frequency of actions. After the redistribution, the manifolds are combined according to the distribution determined by the action classification. In addition, the spatial relationship between human-body parts is explicitly modeled by a kinematic chain. Computer-simulation results showed that multiple low-dimensional manifolds can represent human-pose space. The 3D-point-cloud system showed reduction of the overall error and standard deviation compared with other approaches without using action classification.