In this paper, we address efficiently and robustly collecting objects stored in different trays using a mobile manipulator. A resolution complete method, based on precomputed reachability database, is proposed to explore collision-free inverse kinematics (IK) solutions and then a resolution complete set of feasible base positions can be determined. This method approximates a set of representative IK solutions that are especially helpful when solving IK and checking collision are treated separately. For real world applications, we take into account the base positioning uncertainty and plan a sequence of base positions that reduce the number of necessary base movements for collecting the target objects, the base sequence is robust in that the mobile manipulator is able to complete the part-supply task even there is certain deviation from the planned base positions. Our experiments demonstrate both the efficiency compared to regular base sequence and the feasibility in real world applications.

In this paper, we propose a method of visual recognition in cooperation with actions for automatic handling of clothing by a robot. Difficulty in visual recognition of clothing largely depends on the observed shape of the clothing. Therefore, strategy of actively making clothing into the shape easier to recognize should be effective. First, after clothing is observed by a trinocular stereo vision system, it is checked whether the observation gives enough information to recognize the clothing shape robustly or not. If not, proper “recognition-aid” actions, such as rotating and/or spreading the clothing, are automatically planned based on the visual analysis of the current shape. After executing the planned action, the clothing is observed again to recognize. The effect of the action of spreading clothes was demonstrated through experimental results using an actual humanoid.

In this paper, we propose a deformable-model-driven method to recognize the state of hanging clothes using three-dimensional (3D) observed data. For the task to pick up a specific part of the clothes, it is indispensable to obtain the 3D position and posture of the part. In order to robustly obtain such information from 3D observed data of the clothes, we take a deformable-model-driven approach[4], that recognizes the clothes state by comparing the observed data with candidate shapes which are predicted in advance. To carry out this approach despite large shape variation of the clothes, we propose a two-staged method. First, small number of representative 3D shapes are calculated through physical simulations of hanging the clothes. Then, after observing clothes, each representative shape is deformed so as to fit the observed 3D data better. The consistency between the adjusted shapes and the observed data is checked to select the correct state. Experimental results using actual observations have shown the good prospect of the proposed method.

In this paper, we propose a strategy for a dual-arm robot to pick up a specific part of clothing with one hand while holding the item of clothing with its other hand. Due to the large deformability of clothing, the handling requirements differ from those required for the handling of rigid objects. In the case of holding a specific part of clothing, large deformation leads to a large variety of positions and orientations of the target part, requiring flexibility of both visual recognition and motion control. On the other hand, since the clothing can flexibly curve over the hand, a relatively large range of suitable actions is allowed for grasping the clothing. By considering these characteristics, the following three-stage strategy is proposed. First, the state of the clothing is recognized from visual observation of the clothing using a deformable-model [1]. Then, the theoretically optimal position and orientation of the hand for handling a specific part of the clothing is calculated based on the recognition results. Finally, the position and orientation of the hand is modified by considering the executable motion range of the dual arms. Preliminary experimental results using actual observations of a humanoid robot were used to validate the effectiveness of the proposed strategy.

Considering a stereoscopic visual system, this paper deals with the error involved by a 3D reconstruction process. If image pixels are seen as surfaces instead of points, interval analysis provides bounding boxes in which the reconstructed 3D point lies with certainty. This paper presents a method which refine this bounding box and gives a tighter approximation of the error. Using bisection, and a reprojection test in the image planes, the space in which the 3D reconstructed point may be located is given as an octree. This is achieved through the resolution of a set inversion problem using the SIVIA algorithm. For a lighter manipulation of the result, an englobing ellipsoid is deduced from this approximation. Finally the three models are tested on a recognition process for a humanoid robot.

This papers presents an application of interval analysis to a 3D reconstruction problem. The aim is to build a partial boundaries representation of an object including guaranteed information according to the camera model. Features points coordinates are described using intervals. This representation is used together with a method to search stereo correspondence based on the connectivity of segments.

A new calibration algorithm for multicamera systems using a planar reference pattern is proposed. The algorithm is an extension of Sturm-Maybank-Zhang style plane-based calibration technique for use with multiple cameras. Rigid displacements between the cameras are recovered as well as the intrinsic parameters only by capturing with the cameras a model plane with known reference points placed at three or more locations. Thus the algorithm yields a simple calibration means for stereo vision systems with an arbitrary number of cameras while maintaining the handiness and flexibility of the original method. The algorithm is based on factorization of homography matrices between the model and image planes into the camera and plane parameters. To compensate for the indetermination of scaling factors, each homography matrix is rescaled by a double eigenvalue of a planar homology defined by two views and two model planes. The obtained parameters are finally refined by a nonlinear maximum likelihood estimation (MLE) process. The validity of the proposed technique was verified through simulation and experiments with real data.

Presents a method to search for stereo correspondence based on the connectivity of segments. The data structure is defined by boundary representation. The free curve can be treated by this data structure not as a straight line approximation, but as a curved line. Some candidates for correspondence pairs consist of two segments in left and right images under epipolar condition. The connectivity of two pairs is decided by their distance, intensity, and angle, based on each boundary in the left image. The similarity of paths, which are sequences of pairs, is evaluated. The main feature of our method is that it can evaluate in general situations because the similarity is calculated based on the connectivity of pairs. Multiple correspondence between pairs is removed using this similarity, and the 3D structure is reconstructed. Applications of our work include recognition for free form objects and tracking in 3D space.

We are developing an advanced 3D vision system VVV (Versatile Volumetric Vision) which can be used for many purposes in many fields; robotics, medical and welfare, CAD, VR, etc. The basic processes of 3D vision are range sensing of a 3D scene, describing 3D shape of objects in the scene, and recognizing the 3D position and orientation of objects by matching with object models. Also, moving objects can be tracked by repeating these processes. For the system to be versatile, it consistently executes these processes and makes no assumption on objects; objects may be any size, any shape (figures, polyhedra to free forms), covered with textured or glossy surfaces, at any position, in any direction, together with other objects, and partially occluded by them. The basic technical approach is structural analysis and partial pattern matching. And, the basic policy of development is to separate "functions" which are common to all tasks and "knowledges" which are specific to each task.

A new approach to integrate range data from multiple viewpoints is described. Complicated objects, for example flowers, are observed by a range finder from multiple viewpoints whose relative position is unknown. After simple segmentation, correspondences of regions between two consecutive data are examined. This matching process relies on regions which are not supposed to be influenced by occlusion. Transformation parameters between two data are calculated by referring to the matching result. An iteration process to minimize difference estimates of corresponding regions is devised so that a more accurate result is obtained. Data observed from multiple viewpoints are transferred to standard coordinates and integrated. Results on some objects, for example flowers, show that so far that this method is promising.>