Finding objects and tracking their poses are essential functions for service robots, in order to manipulate objects and interact with humans. We present an approach for object detection and 3D pose estimation for autonomous mobile robots, that is suitable for general uses in a modularized robot control system. Our approach extracts local features from the input images, searches for the reference pattern, and then produces the 3D pose in camera coordinate system, using only a single reference image and the 6-DOF pose in it. We have created an RT (Robot Technology) component that can be used in any RT-based system, and developed an algorithm that can extend the range of detection and produce robust pose estimation. For evaluation, we have integrated our vision component in an autonomous robot system with a search-and-grasp task, and tested it with several objects that are found in ordinary domestic environment. We present the details of our approach, the design of our modular component design, and the results of the experiments in this paper.

Computing correspondence between time frames of a time-dependent 3D surface is essential for the understanding of its motion and deformation. In particular, it can be a useful tool in compression, editing, texturing, or analysis of the physical or structural properties of deforming objects. However, correspondence information is not trivial to obtain for experimentally acquired 3D animations, such as time-dependent visual hulls (typically represented as either a binary occupancy grid or as a sequence of meshes of varying connectivity). In this article we present a new nonrigid fitting method that can compute such correspondence information for objects that do not undergo large volume or topological changes, such as living creatures. Experimental results show that it is robust enough to handle visual hull data, allowing to convert it into a constant connectivity mesh with vertices moving in time. Our procedure first creates a rest-state mesh from one of the input frames. This rest-state mesh is then fitted to the consecutive frames. We do this by iteratively displacing its vertices so that a combination of surface distance and elastic potential energy is minimized. A novel rotation compensation method enables us to obtain high-quality results with linear elasticity, even in presence of significant bending.

We present a new method of fitting an element-free volumetric model to a sequence of deforming surfaces of a moving object. Given a sequence of visual hulls, we iteratively fit an element-free elastic model to the visual hull in order to extract the optimal pose of the captured volume. The fitting of the volumetric model is acheived by minimizing a combination of elastic potential energy, a surface distance measure, and a self-intersection penalty for each frame. A unique aspect of our work is that the model is mesh free - since the model is represented as a point cloud, it is easy to construct, manipulate and update the model as needed. Additionally, linear elasicity with rotation compensation makes it possible to handle local deformations and large rotations of body parts much more efficiently than other volume fitting approaches. Our experimental results for volume fitting and capture in a multi-view camera setting demonstrate the robustness of element-free elastic models against noise and self-occlusions.