The present paper describes a method for automatic 3D contour model creation for stereo-vision systems. The object model is a triangular surface mesh and a set of aspect models, which consists of model features and model points. Model features and model points are generated using 3D contours, which are estimated by the projected images of the triangular surface mesh from multiple discrete viewing directions. Using a non-photorealistic rendering approach, we extract not only the outer contours but also the inner contours of the projected images. Using both the inner and outer contours of the projected images, we create the object model which has 3D inner contour features and 3D contour generator features. Experimental results obtained using the 3D localization algorithm demonstrate the effectiveness of the proposed model.

This paper describes a method to localize 3D objects, which is the extension of the segment-based object recognition method to use on a STL CAD model. Models for localization are automatically generated using contour generators, which are estimated by occluding contours of projected images of the CAD model from multiple viewing directions and depth images computed with a graphics accelerator. In addition, the model is dynamically updated in a recognition process according to the viewing direction. Localization process is based onmultiple hypotheses verification using the model and 3D boundaries reconstructed from stereo images. Experimental results show the effectiveness of the proposed method.

This paper describes a method to localize objects which are partially buried in an unstructured environment. The objects are limited to surfaces of revolution in the present study. Candidate regions are extracted as convex surface regions from a range data that is obtained by area-based stereo matching. The surface type at each point is determined based on the signs of the maximum and minimum curvatures, which are calculated by fitting a quadric in the local window. For each candidate region, multiple hypotheses for the position and direction of the rotational axis are generated for each object model. Each hypothesis is verified and improved by an iterative method, and the most reliable hypothesis is adopted. The experimental results demonstrate the effectiveness of the proposed method for actual range data.

This paper proposes a method for a humanoid robot to generate 3D model of the environment using a stereo vision, find a movable space using it and plan feasible locomotion online. The model is generated by an accumulation of 3D grid maps which are made from the range data of the field of view obtained by a correlation based stereo vision. The locomotion is planned by an online whole body pattern generator which can modify robot’s waist height, an upper body posture and so on according to the size of the movable space.

Considering a stereoscopic visual system, this paper deals with the error involved by a 3D reconstruction process. If image pixels are seen as surfaces instead of points, interval analysis provides bounding boxes in which the reconstructed 3D point lies with certainty. This paper presents a method which refine this bounding box and gives a tighter approximation of the error. Using bisection, and a reprojection test in the image planes, the space in which the 3D reconstructed point may be located is given as an octree. This is achieved through the resolution of a set inversion problem using the SIVIA algorithm. For a lighter manipulation of the result, an englobing ellipsoid is deduced from this approximation. Finally the three models are tested on a recognition process for a humanoid robot.

As a part of research on autonomous loading system by wheel loader, a method for appropriate arrangement of bucket trajectory for scooping motion will be proposed. In this paper, relation between resistance force and advancing direction of the bucket is analyzed theoretically. Advancing direction is dominant factor for resistance force on the bucket during scooping. Based on this analysis, simple rules for bucket trajectory arrangement are proposed. In scooping procedure, scooped volume is estimated using 3D model obtained with stereo-vision system. The shape of the pile is measured and converted into pile model prior to scooping. The developed method and system are installed on experimental scale model of wheel loader. The results show good performance for different condition of pile.

This papers presents an application of interval analysis to a 3D reconstruction problem. The aim is to build a partial boundaries representation of an object including guaranteed information according to the camera model. Features points coordinates are described using intervals. This representation is used together with a method to search stereo correspondence based on the connectivity of segments.

A new calibration algorithm for multicamera systems using a planar reference pattern is proposed. The algorithm is an extension of Sturm-Maybank-Zhang style plane-based calibration technique for use with multiple cameras. Rigid displacements between the cameras are recovered as well as the intrinsic parameters only by capturing with the cameras a model plane with known reference points placed at three or more locations. Thus the algorithm yields a simple calibration means for stereo vision systems with an arbitrary number of cameras while maintaining the handiness and flexibility of the original method. The algorithm is based on factorization of homography matrices between the model and image planes into the camera and plane parameters. To compensate for the indetermination of scaling factors, each homography matrix is rescaled by a double eigenvalue of a planar homology defined by two views and two model planes. The obtained parameters are finally refined by a nonlinear maximum likelihood estimation (MLE) process. The validity of the proposed technique was verified through simulation and experiments with real data.

We have developed a humanoid robot HRP-2P with a biped locomotion controller, stereo vision software and aural human interface to realize cooperative works by a human and a humanoid robot. The robot can find a target object by the vision, and carry it cooperatively with a human by biped locomotion according to the voice commands by the human. A cooperative control is applied to the arms of the robot while it carries the object, and the walking direction of the robot is controlled by the interactive force and torque through the force/torque sensor on the wrists. The experimental results are presented in the paper.

We propose a framework for face recognition performed in 3D space. A 3D facial model consisting of a sparse depth map is constructed from stereo images using isoluminance lines for the stereo matching. By searching for arcs whose radiuses are of certain ranges, we can locate the candidate irises very efficiently. After the pose of the face is detected, the 3D model is transformed into a canonical pose. Recognition is performed by calculating the mean differences in depth between the corresponding data points in the 3D test model and all the models in the database. The corresponding data points refer to a pair of closest data points in two 3D models. We show that even without using any 2D features, we can do face recognition depending on the depth information only. The 3D pose detecting algorithm makes it possible for our face recognition algorithm to be pose invariant.

Calibration of coordinate systems between cameras and robots, i.e. hand-eye-calibration, is of key importance in sensor based robotics. We present a method which is based on using a very simple, conic calibration object, and results in fast and simple calculations. It originates from the pose estimation algorithms and contributes to earlier methods robustness for the location of the calibration object and simplicity from the point of view of image processing. Our method is also based on the Bayesian model, thus it also provides estimates for the spatial uncertainties, in the form of covariances of the pose parameters. The algorithm has been tested by simulations and practical experiments.

We propose a new method to recognize 3D free-form objects from their apparent contours. It is the extension of our established method to recognize objects with fixed edges. Object models are compared with 3D boundaries which are extracted by segment-based stereo vision. Based on the local shapes of the boundaries, candidate transformations are generated. The candidates are verified and adjusted based on the whole shapes of the boundaries. The models are built from all-around range data of the objects. Experimental results show the effectiveness of the method.

Presents a method to search for stereo correspondence based on the connectivity of segments. The data structure is defined by boundary representation. The free curve can be treated by this data structure not as a straight line approximation, but as a curved line. Some candidates for correspondence pairs consist of two segments in left and right images under epipolar condition. The connectivity of two pairs is decided by their distance, intensity, and angle, based on each boundary in the left image. The similarity of paths, which are sequences of pairs, is evaluated. The main feature of our method is that it can evaluate in general situations because the similarity is calculated based on the connectivity of pairs. Multiple correspondence between pairs is removed using this similarity, and the 3D structure is reconstructed. Applications of our work include recognition for free form objects and tracking in 3D space.

This paper presents a manipulation system which is based on visual tracking. A tracking function is useful not only for tracking moving objects but also to verify the result of a task and to detect events in various phases of a task. In our system the tracking function is provided by a stereo vision system. Recognition and tracking procedures are based on a geometric model representation that has generality for a wide range of solid objects. The implemented tracking method is robust against partial occlusion of the tracked object and enables short cycle monitoring of the target coordinates. These features make it possible to build a tracking-based manipulation system. A pick-and-insertion task was executed to show various uses of the tracking function.

We are developing an advanced 3D vision system VVV (Versatile Volumetric Vision) which can be used for many purposes in many fields; robotics, medical and welfare, CAD, VR, etc. The basic processes of 3D vision are range sensing of a 3D scene, describing 3D shape of objects in the scene, and recognizing the 3D position and orientation of objects by matching with object models. Also, moving objects can be tracked by repeating these processes. For the system to be versatile, it consistently executes these processes and makes no assumption on objects; objects may be any size, any shape (figures, polyhedra to free forms), covered with textured or glossy surfaces, at any position, in any direction, together with other objects, and partially occluded by them. The basic technical approach is structural analysis and partial pattern matching. And, the basic policy of development is to separate "functions" which are common to all tasks and "knowledges" which are specific to each task.

We have developed a teaching system based on the assumption that the assembly task is a series of operations for achieving a target contact state among objects through changing their contact states. The features of our system are as follows: 1) teaching data for fine motion is extracted from a demonstration by an operator in a virtual world, 2) the operator can edit intermediate expressions on a display, 3) intermediate expressions can be used by any robot system with a skill library which has the ability to achieve the corresponding contact states. This paper describes the configuration of the system, program generation using intermediate expressions and a skill library to achieve each contact state.

There is a new approach to define objects qualitatively and hierarchically by generic shapes (we call them primitives) arranged by generic relations so that a class of objects has the same description and to recognize them in a parallel and bottom-up way in the image. This paper presents a method to find a skewed symmetrical pattern in the image as a higher-order primitive which is defined by lower-order primitives such as line segments, curved segments, or the pairs of segments arranged by the relation skewed symmetry since a bilaterally symmetrical figure on an arbitrarily oriented plane viewed under orthographic projection yields a skewed symmetrical figure whose axes of symmetry and skew constrain the orientation of the underlying plane. The proposed method can detect the accurate skewed symmetrical axes of not only polygonal patterns but also curved patterns in a complex image even if some parts of the patterns are occluded.>

Describes a newly developed labeling board based on an algorithm in a combination of raster scanning and boundary tracking. This board employs the boundary tracking method and does not require a working memory for temporary labels, and thus it can process labels in size 512*512 images without restricting the maximum numbers of connected regions. It can also count holes within regions, and control labeling by using the perimeter of regions, at the same time of labelling. The processing time is 50 msec for size 512*512 real images.>

It is difficult to obtain sufficient accuracy in view of the outdoor environment to perform 3D stereo measurement. The authors propose a method which involves moving a single camera and prolonging the base line, then measuring the motion parameters of the camera by image correspondence. Primitives are used as units of correspondence to make the matching stable. This paper presents the method and some experimental results.>

The authors extend the algorithm for stereo vision on a mobile robot in a unstructured environment to generate the 3-D b-rep of the large environment by incrementally integrating local and incomplete stereo data. The stereo vision is based on the 2-D b-reps of stereo images. The 2-D b-rep is a 2-D version of the 3-D b-rep to describe an image; the image is represented in a hierarchy of regions, boundaries, and line-segments. The b-rep is obtained for each of stereo images and the two images are matched on this data structure. The direct output of stereo is 3-D wireframes, a set of 3-D loops (boundaries) composed of 3-D line-segments, of the scene. A unique surface is then defined for each loop to make the 3-D b-rep, the consistent geometrical model, of the scene. It is necessary for the mobile robot to change views of the scene to make uninterpretable loops interpretable by finding missing segments, and thus make the complete b-rep of the scene.>

All stereo systems assume that the correct camera parameters are obtained. In fact, those parameters can be calibrated in advance using known test patterns. However, not only there is calibration errors but also some parameters are easy to change after the calibration. Even if the errors may be small, the effects on identifying corresponding points and computing distances are very large. It is infeasible to compute camera parameters every time befort? observation especially for stereo cameras which change its focus of attention by convergence or those on mobile robots which observe their environment and acquire depth information. The self-calibration of stereo cameras, which autonomously computes its current camera parameters from unknown observed data, is really necessary for real stereo systlxns. In this paper, we propose a self-calibration method based on boundary representations of a pair of stereo images.

The problem of detecting occluding and ridge edges is solved for objects with planar surfaces, using stereo vision that is based on directional boundary segments of regions in a pair of stereo images. Methods are proposed to detect occlusions in which occluding edges are explicitly identified: to classify the other unidentified edges as either occluding or ridge edges, using the explicitly occluding edges to classify ridge edges further as convex, concave, or flat (paint or shadow) edges, based on the surface normals of both sides of the edges; and to classify the space of the scene into free, occupied, or uninterpretable real surfaces.>

Stereo problems regarding range resolution, the output available for higher-level postprocessing, and the so-called correspondence problem are discussed. To solve the problems, it is proposed to use boundary segments of regions as units for correspondence between a pair of stereo images. Not only the features of regions and segments themselves, but also the high connectivity of segments can be used as strong constraints for correspondence. A planarity constraint is introduced for segments on planar surfaces. Since the output form is a boundary representation, the most general solid model, it can be used directly as the input to an object recognition system without any preprocessing.>

A structural analysis system for describing natural textures is introduced. The analyzer automatically extracts the texture elements in an input image, measures their properties, classifies them into some distinctive classes (one ``ground'' class and some ``figure'' classes), and computes the distributions of the gray level, the shape, and the placement of the texture elements in each class. These descriptions are used for classification of texture images. An analysis-by-synthesis method for evaluating texture analyzers is also presented. We propose a synthesizer which generates a texture image based on the descriptions. By comparing the reconstructed image with the original one, we can see what information is preserved and what is lost in the descriptions.