This paper proposes a novel approach to registering deformations of 3D non-rigid objects for Augmented Reality applications. Our prototype is able to handle different types of objects in real-time regardless of their geometry and appearance (with and without texture) with the support of an RGB-D camera. During an automatic offline stage, the model is processed in order to extract the data that serves as input for a physics-based simulation. Using its output, the deformations of the model are estimated by considering the simulated behaviour as a constraint. Furthermore, our framework incorporates a tracking method based on templates in order to detect the object in the scene and continuously update the camera pose without any user intervention. Therefore, it is a complete solution that extends from tracking to deformation formulation for either textured or untextured objects regardless of their geometrical shape. Our proposal focuses on providing a correct visual with a low computational cost. Experiments with real and synthetic data demonstrate the visual accuracy and the performance of our approach.

ABSTRACT Marker systems are a widely used optical tracking method that does not support occlusions. Thus, this paper proposes a new marker design to overcome the problem of marker occlusions. It is highly adaptable, because it can be used by any marker tracking system that uses its central area to codify the digital identification. Our proposal takes advantage of an untapped frame to place some textures that will be tracked during marker occlusion. In addition, these textures are customizable, which lets users make their own designs. Two tracking methods are combined to offer a robust tracking, updating the six degrees of freedom of the camera in real time. The first one is a fast technique based on temporal coherence, whereas the second one is a robust technique based on appearance, which is used as a recovery mode. Copyright © 2012 John Wiley & Sons, Ltd.