In this paper, we address the problem of human body pose estimation from depth data. Previous works Based on random forests relied either on a classification strategy to infer the different body parts or on a regression approach to predict directly the joint positions. To permit the inference of very generic poses, those approaches did not consider additional information during the learning phase, e.g. the performed activity. In the present work, we introduce a novel approach to integrate additional information at training time that actually improves the pose prediction during the testing. Our main contribution is a multi-task forest that aims at solving a joint regression-classification task: each foreground pixel from a depth image is associated to its relative displacements to the 3D joint positions as well as the activity class. Integrating activity information in the objective function during forest training permits a better partitioning of the 3D pose space that leads to a better modelling of the posterior. Thereby, our approach provides an improved pose prediction, and as a by-product, can give an estimate of the performed activity. We performed experiments on a dataset performed by 10 people associated with the ground truth body poses from a motion capture system. To demonstrate the benefits of our approach, poses are divided into 10 different activities for the training phase. Results on this dataset show that our multi-task forest provides improved human pose estimation compared to a pure regression forest approach.

Statistical human body models, like SCAPE, capture static 3D human body shapes and poses and are applied to many Computer Vision problems. Defined in a statistical context, their parameters do not explicitly capture semantics of the human body shapes such as height, weight, limb length, etc. Having a set of semantic parameters would allow users and automated algorithms to sample the space of possible body shape variations in a more intuitive way. Therefore, in this paper we propose a method for re-parameterization of statistical human body models such that shapes are controlled by a small set of intuitive semantic parameters. These parameters are learned directly from the available statistical human body model. In order to apply any arbitrary animation to our human body shape model we perform retargeting. From any set of 3D scans, a semantic parametrized model can be generated and animated with the presented methods using any animation data. We quantitatively show that our semantic parameterization is more reliable than standard semantic parameterizations, and show a number of animations retargeted to our semantic body shape model.

In this paper, we present a fusion algorithm supplemented with appropriate visualization by selecting relevant information from different modalities in mixed and augmented reality (AR). This encompasses a learning based method upon relevance of information, defined by an expert, which ultimately enables confident interventional decisions based on mixed reality (MR) images. The performance of our developed fusion and tailored visualization techniques was evaluated by employing X-ray/optical images during surgery and validated qualitatively using a 5-point Likert scale. Our observations indicated that the proposed technique provided semantic contextual information about underlying pixels and in general was preferred over the traditional pixel-wise linear alpha-blending method.

Multi-modal registration is the task of aligning images from an object acquired with different imaging systems, sensors or parameters. The current gold standard for medical images is the maximization of mutual information by computing the joint intensity distribution. However intensities are highly sensitive to various kinds of noise and denoising is a very challenging task often involving a priori knowledge and parameter tuning. We propose to perform registration on a novel robust information support: the wavelet energy map, giving a measure of local energy for each pixel. This spatial feature is derived from local spectral components computed with a redundant wavelet transform. The multi-frequential aspect of our method is particularly adapted to robust registration of images showing ambiguities such as tissues, complex textures and multiple interfaces. We show the benefits of the wavelet energy map approach in comparison to the classical framework in 2D and 3D rigid registration experiments on synthetic and real data.