We propose an object tracking system which recognizes everyday objects and estimates their positions by using distributed sensors in a room and mobile robots. The placement of objects is frequently changed according to human activities. Although a passive RFID tag is attached to each object for the object's recognition, the placement is often not uniquely determined due to the deficiency of measured data. We have already proposed a method for estimating the placement of objects by using the moving trajectories of objects. This estimation result is expressed as the probability distribution of the object placement. However intersections of trajectories cause the decrease of the estimation accuracy. So we propose a new method based on Bayesian inference to improve the estimation accuracy by using the size and the shape of an object measured by laser range finder. Then a mobile robot settles the placement with small workload by using the mounted sensor. The system successfully recognized and localized 10 objects in the experiment.

This paper presents a grasp planner which allows a robot to grasp the constricted parts of objects in our daily life. Even though constricted parts can be grasped more firmly than convex parts, previous planners have not sufficiently focused on grasping this part. We develop techniques for quadric surface approximation, grasp posture generation, and stability evaluation for grasping constricted parts. By modeling an object into multiple quadric surfaces, the planner generates a grasping posture by selecting one-sheet hyperbolic surfaces or two adjacent ellipsoids as constricted parts. When a grasping posture being generated, the grasp stability is evaluated based on the distribution of the stress applied to an object by the fingers. We perform several simulations and experiments to verify the effectiveness of our proposed method.

For creating a three-dimensional (3D) model of a real object using a laser scanner and a camera, texture mapping is an effective technique to enhance the reality. However, in case that the positions of the camera and the laser scanner differ from each other, some textureless regions (holes) may exist on the object surface where the appearance information is missing due to the occlusion or out-of-sight of the camera. In this paper, we propose a new texture completion technique utilizing laser reflectivity for hole-free texture mapping. The laser reflectivity, which denotes the power of a reflected laser light/pulse, is obtained as by-product of the range information at laser scanning. Since the laser reflectivity captures the appearance property of the target as a camera image, it is reasonable that the regions with similar reflectance properties have similar color textures. Based on this idea, texture information in these holes is copied and pasted from the other texture regions according to the similarity and the order determined by the texture and laser reflectivity. To verify the performance of the proposed technique, we carried out texture completion experiments in real scenes.

In this paper, we propose a new technique for adding color to a surface of a 3D geometrical model utilizing laser reflectivity. A time-of-flight laser scanner obtains a range image from the sensor toward the target by measuring the round-trip time of a laser pulse. At the same time, for most laser scanners, the reflectance image (which is the strength of the reflected light) is available as a by-product of the range value. The proposed technique first assigns appearance information to a 3D model by colorizing a reflectance image based on the similarity of color and reflectance images. Then the color information is transferred to the corresponding range image, and the colorized 3D model is obtained. We carried out experiments using a laser scanner and showed the performance of the proposed technique in several conditions.

This paper describes an object tracking system for a robot working in an everyday environment, which tracks and recognizes everyday objects. Passive RFID (Radio Frequency IDentification) tags are attached to the objects for object recognition. The system consists of static sensors embedded in the environment and mobile sensors mounted on mobile robots. By utilizing the different characteristics and advantages of these sensors, the system achieves good performance in an everyday environment. Although the tag ID and the position of an object carried by a person is not measurable by static sensors or by mobile sensors, the system can estimate them by using an SIR (Sequential importance resampling) particle filter that integrates the data obtained by the static sensors and the mobile sensors. In the experiment, the system successfully tracked 20 objects, some of which were held by a person.

This paper proposes a new visual servoing method for object manipulation robust to considerable time-delays of visual information. There still remain several problems in visual servoing methods although they are quite useful and effective for dexterous object manipulation. For instance, time-delays to obtain necessary information for object manipulation from visual images induce unstable behavior. The time-delays are mainly caused by low sampling rate of visual sensing system, computational cost for image processing, and latency of data transmission from visual sensor to processor. The method makes it possible to avoid such unstable behavior of the systems due to considerable time-delays using virtual object frame defined by only each joint angle. Firstly, a new control scheme for object manipulation using the virtual object frame is designed. Next, numerical simulations are conducted to verify the effectiveness of the control scheme. Finally, experimental results are shown to demonstrate the practical usefulness of proposed method.

This paper proposes a novel vision-based grasping and manipulation scheme of a multi-fingered hand-arm system robust for a temporary lack of sensory information. Visual information is one of the fundamental components for reliable grasping and manipulation by a multi-fingered hand-arm system. However, in case that visual information such as position and attitude of an object comes to be unavailable due to the occlusion or if the object goes out-of-sight temporarily, unstable and unfavorable behavior is often induced. The proposed method, which utilizes the stable grasping control and the concept of virtual frame, enables to grasp and manipulate an object stably even if the visual information becomes suddenly and temporarily unavailable during manipulation. Firstly, a dynamical model of object grasping using a multi-fingered hand-arm system is formulated. Next, a new control scheme for robust object grasping and manipulation using the virtual frame is proposed. Finally, numerical simulations are performed to verify the usefulness of the proposed method.

Two denoising techniques using reflectivity for noisy range images are proposed: range image smoothing by trilateral filter and range image inpainting by belief propagation. The trilateral filter makes use of reflectivity as well as spatial and intensity information so that geometric features, such as jump and roof edges, are preserved while smoothing. The range image inpainting technique based on belief propagation recovers a deteriorated range image using not only the adjacent range values but also the continuity of the reflectance image. We conduct simulations and experiments using synthesized images and actual range images taken by a laser scanner and verify that the proposed techniques suppress noise while preserving jump and roof edges and repair deteriorated range images.

For a coexisting and collaborative society that incorporates humans and robots, the detection, tracking, and recognition of human motion are indispensable techniques for a robot to safely and securely interact with humans. The present paper proposes a motion tracking system using distributed network cameras that are placed in a sizeable environment, such as a street or a town. Model-based motion tracking is adopted in this system, and an asynchronous process is invoked for updating motion estimation in each camera individually. A 2D distance map created by the Fast Marching Method is used to estimate human motion in real-time. Experiments demonstrate that human motion while walking among eight distributed cameras is tracked correctly by automatically selecting appropriate cameras.

Traditional methods require large computation to model a contact state graph of polyhedral objects for robotic application, and moreover they are heuristic. In this paper, we propose a framework to generate the contact state graph automatically. All faces of the polyhedral objects are triangulated. A sub-contact is defined as single contact between two polyhedral objects and a contact state is presented with a set of the sub-contacts. There are two sub-contacts, a vertex-triangle contact and an edge-edge contact. According to convexity or concavity of the edges composing the triangle, the vertex-triangle contact and the edge-edge contact are classified into 10 types and 7 types, respectively. A contact state graph is made by evolutionary transitions of the sub-contacts. This procedure is accomplished only using the topology of the sub-contacts, which is possible in real-time. The proposed framework is evaluated by an example of square peg-in-hole assembly.

This paper describes a tactile sensing to estimate the position and orientation of a joint-axis of a linked object. This tactile sensing is useful when a multi-jointed multi-fingered robotic hand manipulates a tool which has a joint. This estimation requires sensing of the location of a contact point and the direction of an edge of the tool as contact information measured by a robotic fingertip. A conventional hard fingertip with a force sensor can measure only the location of a contact point. In contrast, we have already developed a robotic fingertip with a force sensor and a soft skin, and it can measure not only the location of a contact point but also the direction of an edge of an object. The estimation of a joint-axis of a linked object is demonstrated by using the soft fingertip.

We propose an object tracking system for a service robot working in an everyday environment. The system is composed of an intelligent cabinet, a floor sensing system and a data management system. The position of an object can be classified into three areas: 1) in/on furniture, 2) on the floor, 3) held by a human or a robot. Being equipped with a RFID reader and loadcells, the intelligent cabinet measures the position of an object in/on itself. The floor sensing system which uses a laser range finder, measures the position of an object on the floor and the position of a human walking in a room. The data management system integrates the position data of the intelligent cabinets and the floor sensing system, and it performs position measurement of an object carried by a human. The data management system provides robots with position information to support robot activities.

This paper proposes a new method of measuring position of daily commodities placed on a floor. Picking up an object on a floor will be a typical task for a robot working in our daily life environment. However, it is difficult for a robotic vision to find a small daily life object left on a large floor. The floor surface may have various texture and shadow, while other furniture may obstruct the vision. Various objects may also exist on the floor. Moreover, the surface of the object has various optical characteristics: color, metallic reflection, transparent, black etc. Our method uses a laser range finder (LRF) together with a mirror installed on the wall very close to floor. The LRF scans the laser beam horizontally just above the floor and measure the distance to the object. Some beams are reflected by the mirror and measure the distance of the object from virtually different origin. Even if the LRF fails two measurements, the method calculates the position of the object by utilizing information that the two measurements are unavailable. Thus, the method achieves two major advantages: 1) robust against occlusion and 2) applicable to variety of daily life commodities. In the experiment, success rate of observation of our method achieves 100% for any daily commodity, while that of the existing method for a cell-phone is 69.4%.

Repeated patterns are useful clues to learn previously unknown events in an unsupervised way. This paper presents a novel method that detects relatively long variable-length unknown repeated patterns in a motion sequence efficiently. The major contribution of the paper is two-fold: (1) Partly Locality Sensitive Hashing (PLSH) [1] is employed to find repeated patterns efficiently and (2) the problem of finding consecutive time frames that have a large number of repeated patterns is formulated as a combinatorial optimization problem which is solved via Dynamic Programming (DP) in polynomial time O(N1+1/α) thanks to PLSH where N is the total amount of data. The proposed method was evaluated by detecting repeated interactions between objects in everyday manipulation tasks and outperformed previous methods in terms of accuracy or computational time.

In order to construct three-dimensional shape models of large-scale architectural structures using a laser range finder, a number of range images are taken from various viewpoints. These images are aligned using post-processing procedures such as the ICP algorithm. However, in general, before applying the ICP algorithm, these range images must be aligned roughly by a human operator in order to converge to precise positions. The present paper proposes a new modeling system using a group of multiple robots and an on-board laser range finder. Each measurement position is identified by a highly precise positioning technique called Cooperative Positioning System (CPS), which utilizes the characteristics of the multiple-robot system. Thus, the proposed system can construct 3D shapes of large-scale architectural structures without any post-processing procedure or manual registration. ICP is applied optionally for a subsequent refinement of the model. Measurement experiments in unknown and large indoor/outdoor environments are carried out successfully using the newly developed measurement system consisting of three mobile robots named CPS-V. Generating a model of Dazaifu Tenmangu, a famous cultural heritage, for its digital archive completes the paper.

In this paper, we propose a method that detects repeated motion patterns in a long motion sequence efficiently. Repeated motion patterns are the structured information that can be obtained without knowledge of the context of motions. They can be used as a seed to find causal relationships between motions or to obtain contextual information of human activity, which is useful for intelligent systems that support human activity in everyday environment. The major contribution of the proposed method is two-fold: (1) motion density is proposed as a repeatability measure and (2) the problem of finding consecutive time frames with large motion density is formulated as a combinatorial optimization problem which is solved via Dynamic Programming (DP) in polynomial time O(N log N) where N is the total amount of data. The proposed method was evaluated by detecting repeated interactions between objects in everyday manipulation tasks and outperformed the previous method in terms of both detectability and computational time.

This paper proposes a novel control method for stable grasping using a multi-fingered hand-arm system with soft hemispherical finger tips. The proposed method is simple but easily achieves stable grasping of an arbitrary polyhedral object using an arbitrary number of fingers. Firstly, we formulate nonholonomic constraints between a multi-fingered hand-arm system and an object constrained by rolling contact with finger tips, and derive a condition for stable grasping by stability analysis. A new index for evaluating the possibility of stable grasping is proposed and efficient initial relative positions between finger tips and the object for realizing stable grasping are analyzed. The stability of the proposed system and the validity of the index are verified through numerical simulations.

This paper proposes a segmentation method of human manipulation task based on measurement of contact force imposed by a human hand on a grasped object. We define an index measure for segmenting a human manipulation task into primitives. The indices are calculated from the set of the contact forces measured at all the contact points during a manipulation task. Then, we apply the EM algorithm to the set of the indices in order to segment the manipulation task into primitives. These primitives are mapped onto the robotic hand to impose appropriate contact forces on a grasped object. In the experiments, manipulation tasks performed in daily human life have been successfully segmented.

This paper presents a parallel algorithm of the Level Set Method named the Parallel Fast Level Set Method, and its application for real-time 3D reconstruction of human shape and motion. The Fast Level Set Method is an efficient implementation algorithm of the Level Set Method and has been applied to several applications such as object tracking in video images and 3D shape reconstruction using multiple stereo cameras. In this paper, we implement the Fast Level Set Method on a PC cluster and develop a real-time motion capture system for arbitrary viewpoint image synthesis. To obtain high performance on a PC cluster, efficient load-balancing and resource allocation algorithms are crucial problems. We develop a novel optimization technique of load distribution based on the estimation of moving direction of object boundaries. In this technique, the boundary motion is estimated in the framework of the Fast Level Set Method, and the optimum load distribution is predicted and performed according to the estimated boundary motion and the current load balance. Experiments of human shape reconstruction and arbitrary viewpoint image synthesis using the proposed system are successfully carried out.

The present paper describes a decision method for the placement of tactile elements for manipulation task recognition. Based on the mutual information of the manipulation tasks and tactile information, an effective placement of tactile elements on a sensing glove is determined. Although the effective placement consists of a small number of tactile elements, it has a recognition performance that is as high as that of a placement consisting of many tactile elements. The effective placement of tactile elements decided by the proposed method has been evaluated through experiments involving the recognition of grasp type from grasp taxonomy defined by Kamakura [1].

This paper proposes a calibration method of a distributed vision network in a unified world coordinate system. The vision network system is conceived to support a robot working in our daily human life environment: the system provides with visual observation of the dynamically changing situation surrounding the robot. Vision cameras are rather sparsely distributed to cover a wide area such as a block of a town. Position, view direction and range of view are the camera parameters of principal importance to be estimated by the proposed method. A set of calibration data for each distributed camera is provided by a group of mobile robots having a cooperative positioning function and visually distinguishable markers mounted on the body of the robot.

This paper presents a sensor network system consisting of distributed cameras and laser range finders for multiple objects tracking. Sensory information from cameras is processed by the level set method in real time and integrated with range data obtained by laser range finders in a probabilistic manner using novel SIR/MCMC combined particle filters. Though the conventional SIR particle filter is a popular technique for object tracking, it has been pointed out that the conventional particle filter has some disadvantages in practical applications such as its low tracking performance for multiple targets due to the degeneracy problem. In this paper, the new combined particle filters consisting of a low-resolution MCMC particle filter and a high-resolution SIR particle filter is proposed. Simultaneous tracking experiments for multiple moving targets are successfully carried out and it is verified that the combined particle filters has higher performance than the conventional particle filters in terms of the number of particles, the processing speed, and the tracking performance for multiple targets.

In this paper, we propose a method for estimating meaningful actions from long-term observation of everyday manipulation tasks without prior knowledge as part of an action understanding framework for life support robotic systems. The target task is defined as a sequence of interactions between objects. An interaction that appears many times is assumed to be meaningful and repetitious relative motion patterns are detected from trajectories of multiple objects. The main contribution is that the problem is formulated as a combinatorial optimization problem with two parameters, target object labels and correspondences on similar motion patterns, and is solved using local and global Dynamic Programming (DP) in polynomial time O(N logN), where N is a total amount of data. The proposed method is evaluated against manipulation tasks using everyday objects such as a cup and a tea-pot.

In medical diagnostic imaging, an X-ray CT scanner or a MRI system have been widely used to examine 3D shapes or internal structures of living organisms or bones. However, these apparatuses are generally very expensive and of large size. A prior arrangement is also required before an examination, and thus, it is not suitable for an urgent fracture diagnosis in emergency treatment. This paper proposes a method to estimate a patient-specific 3D shape of a femur from only two fluoroscopic images using a parametric femoral model. Firstly, we develop a parametric femoral model by statistical analysis of a number of 3D femoral shapes created from CT images of 51 patients. Then, the pose and shape parameters of the parametric model are estimated from two 2D fluoroscopic images using a distance map constructed by the level set method. Experiments using synthesized images and fluoroscopic images of a phantom femur are successfully carried out and the usefulness of the proposed method is verified.

This paper introduces a new motion capture system for recovering 3D models of multiple persons separately and robustly against occlusion in real-time. Various markerless motion capture systems using video cameras have been proposed so far. However, in case that there are multiple persons in the scene at the same time, it is quite difficult to reconstruct a precise 3D model of each person separately due to the occlusion between them. To deal with this problem, the fast level set method is utilized in the proposed system for integrating stereo range data which is captured by multiple stereo cameras located around the target people. To reconstruct precise 3D models in real-time, the proposed system is implemented on a PC cluster with seven PCs and four stereo cameras. Tracking experiment of multiple persons and real-time reconstruction of 3D human models using the proposed system are successfully carried out

We propose a new method of tactile sensing for a fingertip of robotic hand based on the analysis of the mechanical constraints imposed to the contact between the fingertip and the contacting object. Using a simple fingertip with soft surface cover and a six-axis force/torque sensor, a new function is developed to measure direction of local geometry of a class of objects when the fingertip contacts with them. The developed function is useful not only to plan strategies for stable grasp and dexterous manipulation but also to recognize a geometrical shape of an object. Two necessary components, a six-axis force/torque sensor and a soft skin, are commercially available. They neither restrict design of a robotic fingertip nor badly affect essential functions of stable grasping and manipulation

The inertia matching ellipsoid (IME) is proposed as a new index of dynamic performance for serial-link robotic manipulators. The IME integrates the existing dynamic manipulability and manipulating-force ellipsoids to achieve an accurate measure of the dynamic torque-force transmission efficiency between the joint torque and the force applied to a load held by an end-effector. The dynamic manipulability and manipulating-force ellipsoids can both be derived from the IME as limiting forms, with respect to the weight of the load. The effectiveness of the IME is demonstrated numerically through the selection of an optimal leg posture for jumping robots and optimal active stiffness control, and experimentally through application to a pick-up task using a commercial manipulator. The index is also extended theoretically to the case of a manipulator mounted on a free-flying satellite

We propose a new method of tactile sensing for a fingertip of robotic hand. Using a simple fingertip with soft surface cover and a six-axis force/torque sensor, developed is a new function of measuring direction of edge when the fingertip contacts with an edge of the object. The developed function is useful not only to plan strategies for stable grasp and dexterous manipulation but also to recognize a geometrical shape of an object. Two necessary components, a six-axis force/torque sensor and the soft skin, are commercially available. They will neither restrict design of a robotic fingertip nor badly affect essential functions of stable grasping and manipulation.

This paper presents a new methodology for generating a straight legged walking pattern for a biped robot utilizing up-and-down motion of an upper body. Firstly, we define two new indexes, the knee stretch index (KSI) and the knee torque index (KTI), which indicate how efficiently the knee joints are utilized. Next, up-and-down motion of the upper body is automatically planned so that these indexes are optimized and straight legged walking is realized. The basic idea of the proposed method is: i) when a large number of DOFs of motion are required for controlling the ZMP, a robot makes its body height lower; ii) when there is a extra number of DOFs of motion, the body is lifted and the knee joint is stretched. By stretching the knee joints, human-like natural walking motion is obtained. Moreover, energy efficiency is improved since required torque and energy consumption to support the body weight become small at knee joints. The effectiveness of the proposed method is demonstrated by computer simulation and experiments using a humanoid robot, HOAP-1.

This work presents a new method of combining dynamic control of LOD and conservative occlusion culling based on a new hierarchical data structure of polygons. Our method is effective for rendering a large amount of data in complex environments.

A new approach to telerobotic manipulation is proposed. Using swept volume of a slave manipulator tele-operated initially by a human operator, on-line transition to autonomous tele-operation from master-slave manipulation is achieved without any geometric model of the object in an environment. This feature enables wider application in an unstructured environment.

We propose a new index for dynamic performance analysis of serial link manipulators named impedance matching ellipsoid, or IME. Several indexes have been proposed for indicating static and dynamic performance of robot manipulators. For example, dynamic manipulability ellipsoid (DME) characterizes distributions of hand acceleration produced by normalized joint torque. manipulating-force ellipsoid (MFE) denotes static torque-force transmission efficiency from actuators at joints to a hand. On the other hand, the proposed IME characterizes dynamic torque-force transmission efficiency from actuators at joints to a load held at the hand of the manipulator. The IME includes a wide range of concepts proposed so far as measures of manipulator's performance. The DME and the MFE are both derived from the IME as limiting forms about the load mass. In this paper, we demonstrate the IME with some numerical examples including the selection of an optimal leg posture for jump robots, optimum active stiffness control, and an extension for manipulators mounted on satellites in outer space.

From 1970's, legged robots have attracted much attention of many researchers. In spite of this, it has been regarded that dynamically stable walking is very difficult to be tackled for any types of legged robots. For a trot gait for quadruped walking robots, we have proposed "the sway compensation trajectory". This method utilizes a lateral, longitudinal, and vertical motion of a robot body to keep a zero moment point (ZMP) on a diagonal line between support legs. In this paper, we develop the sway compensation trajectory for a biped robot, and show that dynamically stable walking is realized. This method makes it quite easy to design stable ZMP and COG (center of gravity) trajectories, which have been regarded as a very complicated and delicate problem. The effectiveness of the proposed method is verified through computer simulations and walking experiments by a humanoid robot, HOAP-1.

This paper describes a novel fingertip equipped with a soft elastic cover and a hard nail. The soft elastic cover of the fingertip enables changing friction coefficient at the contact with an object. It also enables detecting change of contact type with an object while manipulating it: the fingertip contact with a flat surface and with a convex sharp edge. The solid nail enables detecting a very small step of sub-millimeter on a smooth surface.

One of the fundamental problems of the mobile robots is self-localization, i.e., to estimate the self-position by comparing sensor data and a map. In non-stationary environments, a robot should avoid to use changed objects as landmarks in the localization. However, in most previous localization methods, it is assumed that there is no change, or changes are easily identified by sensing. In this paper, we propose a self-localization method that is robust against changes in environments. The method identifies changes from noisy and ambiguous sensor data. Since an object with a random shape may be added at a random position, it generates and utilizes multiple hypotheses about the changes. A number of simulation experiments have been performed in various environments, to demonstrate the effectiveness of the method.

Though a legged robot has high terrain adaptability as compared with a wheeled vehicle, its moving speed is considerably low in general. For attaining a high moving speed with a legged robot, a dynamically stable gait, such as running for a biped robot and a trot gait or a bound gait for a quadruped robot, is a promising solution. However, the energy efficiency of the dynamically stable gait is generally lower than the efficiency of the stable gait such as a crawl gait. In this paper, we present an experimental study on the energy efficiency of a quadruped walking vehicle. Energy consumption of two walking patterns for a trot gait is investigated through experiments using a quadruped walking vehicle named TITAN-VIII. The obtained results show that the 3D sway compensation trajectory proposed in our previous paper [R. Kurazume et al., 2002] has advantages in view of energy efficiency as compared with the original sway compensation trajectory.

We propose a new real-time self-localization method for a mobile robot equipped with an omni-directional camera in a dynamically changing environment. This method uses direction of two landmarks and dead reckoning. Multiple localization process in parallel results robust and accurate localization. The proposed method applies to the soccer robot in the RoboCup middle-size league and an experimental result indicates that the approach is reliable.

This paper describes a sensing function of a multi-jointed multi-fingered hand with soft fingertips. We propose a new method to detect change of contact types with an object while manipulating it: the fingertip contact with a flat surface and with a convex sharp edge. Proposed method can apply to the manipulation with rolling contact. Experimental results are demonstrated.

This paper describes the practical application of robot vision techniques to electricity distribution work. To cope with variable outdoor lighting conditions and diverse backgrounds due to unstructured outdoor environments, we have developed two methods of detecting an object's 3D position using both 2D gray-scale image and 3D range image. The first method locates a target object as a 2D image taken at a distance. Although the resolution of a 3D range image taken from the same position is too low to extract the object's surface geometry, it is effective to eliminate the background section from the 2D image based on the distance threshold. The second method generates an online template image of the object to be worked on. This enables reliable real-time visual tracking by stereovision despite the variations of the object's shape and appearance. Here we show how these methods are implemented and effectively used in distribution work.

Many mobile robots use maps for path planning. However, due to changes of obstacle configurations in the environment, the utility of the map is often deteriorated. In this paper, we address a problem of map updating task, where a robot automatically explores the environment to update a map. This task is different from well known map making task in that the robot can estimate global structure of the environment by using the map of a past environment. This enables a map updating robot effective viewpoint planning. In this paper, we propose an efficient method for motion planning in the task, in which utility of the map for its users can be taken into account.

This paper focuses on the automation of insertion work, such as inserting a metal sleeve into a wire terminal, necessary in distribution work. The major technical issues for this task automation are considered as: the detection of a target object in an unstructured, complex outdoor environment; and the insertion executed at the work site with restricted clearance. In the former, the target object is detected by applying 3D image processing and template matching techniques, based on the position and posture relationship between CCD cameras, the target object, and the background. In addition, the manipulator's targeting position is updated by commands from the visual sensor system, applying the video-rate stereo camera method. This allows for realtime visual feedback control; thus, misalignment due to deviation arising from manipulator's mechanism is avoided. In the latter issue, a new algorithm was developed for insertion. This algorithm can estimate contact force, based on the deviation between the present position and command position of the manipulator. An experiment using a testing robotic system revealed that the proposed robotic technology was effective for performing insertion work without assistance by humans.

A bilateral exchange and distribution negotiation problem is a problem that two persons who have interdependent goals try to get a solution which satisfies any constraints by exchanging alternative resources in order to distribute the resources to get more profit. It is difficult to solve this problem because it is not desirable to inform the negotiator's constraints to the opponent. The paper proposes extension of the negotiation method using the improving knowledge for considering the negotiator's satisfying level for the agreement plan by introducing the hierarchized conditions expressing the negotiator's ideal about an agreement plan.

We propose an approach to surface reconstruction using active tactile sensing and surface normal information. Unlike previous works which consider primarily the problem of reconstructing a single tensor B-spline patch based on deformed planar or spherical domain, the surface reconstructed here is a network of triangular B-spline patches based on arbitrary topological domain and can maintain tangent plane continuity. The input is unorganized tactile sensing 3D data points with normal information of arbitrary topological type surfaces, which are obtained by a dextrous multifingered hand equipped with force/moment tactile sensors. The surface reconstructed is a parameterization of the data over these patches. An adaptive refinement is explored on the patch network in order to satisfy a given error tolerance. Demonstrations are carried out on simulated and experimental data.

A bilateral exchange and distribution negotiation problem is a problem that two persons who have interdependent goals try to get a solution which satisfy any constraints by exchanging alternative resources in order to distribute the resources for more profits. It is difficult to solve this problem because it is not desirable to inform the negotiator's constraints to the opponent. This paper proposes the negotiation method that each negotiator improves the opponent's plan with knowledge, and sends it back to the opponent in order to find an agreement plan automatically.

We present a new method for controlling levels of detail (LODs) of triangulated mesh objects by a gaze-driven approach. The method relies upon a hierarchical mesh representation, called a merge-and-split tree of vertices, in which objects are described at nearly continuous LODs. The basic idea here is to adjust the mesh shape quality adaptively under a geometrical definition of gaze points and fovea regions. Not only does it generate a circular fovea region with smooth transition of LODs, but it also makes it possible to update the whole mesh quickly for a moving gaze.

The paper proposes a novel method of modeling multiobject scenes or objects of arbitrary topology by a marching method based on mathematical morphology operations with a variable-size structuring element. We assume that 3D data on the whole object surface has been acquired. A narrow-band stopping region is built by merging the neighborhood of every data point. A single close-surface surrounding the data point set is given as an initial shape. The surface shrinks and splits smoothly by repeated erosions with a ball structuring element, whose size increases continuously as the curvature of the surface increases. Every surface point stops moving if it lies in the stopping region. After this surface fitting to the stopping region, the region is narrowed, and the fitting-and-narrowing cycle is repeated until the model is consistent with the object surface on the basis of a topological check. After the above rough estimation, for model refinement, we extract a quadrangular mesh from the resultant surface, and then perform precise fitting to the 3D data by using an energy minimisation.

The paper presents a novel method of registering two range images that are taken at different 3D viewpoints. Distinct from classic methods, it allows an accurate estimation of a scaling parameter of the images in addition to the general motion parameters. It is made possible owing to the following efforts: 1) extended signature images (ESIs) are utilized to establish precise correspondence between image objects even if they are different in scale; 2) fine image alignment is achieved by incorporating scale estimates into a modified iterative closest point (ICP) algorithm. Results of experiments demonstrate the method is of practical use even for images acquired by real digitizers.

We propose a method of generating multiresolution triangulated meshes from range data clouds for modeling 3D objects. Such meshes can be obtained by applying wavelet analysis techniques directly to a fine mesh if it is equivalent in topology to a tetrahedron subdivided by a recursive 4-to-1 splitting algorithm. However, a mesh of this special characteristic is difficult to construct by a traditional meshing method, and hence another time-consuming remeshing process has to be used. To solve the problem, we try to get a topologically proper coarse mesh automatically from any given dataset. Then, finer meshes are produced by applying the 4-to-1 subdivision algorithm recursively, resulting in multiresolution meshes in an efficient manner.

Level set methods were proposed mainly by mathematicians for extracting bounding contours of multiple regions or shapes with holes. In the paper, we propose a new method of constructing a surface/solid model of a 3-D object of arbitrary topology on the basis of the approach. Given sensor data covering the whole object surface, the method begins with an initial approximation of the object by evolving a closed-surface into a model typologically equivalent to the real object. The refined approximation is then performed by deforming it at an optimally tuned speed. By numerically solving the level set equations for tracking the evolving surfaces, it becomes possible to handle 3-D objects of arbitrary topology while maintaining high accuracy in the shape fitting.

Describes a dexterous and robust manipulation system with a multi-fingered hand. The system is composed of a real-time pose estimation of an object manipulated by the hand, a precise manipulation using fingertip with rolling contact, motion primitives for mechanical assembly, and error-recovery strategies based on the task observation function. The system architecture and experimental results are shown.

For robots working in a dynamic environment, there is a great need for a global environment map that should be updated timely. The paper presents an observation planning method for the map updating purpose. The task assigned to the robot is to detect changes of environment at planned observation positions and update the map into the newest version. After the observation at a current position, the robot gets quantitative estimations on the changes that may occur at possible sequences of future observations. Then, it evaluates the sequences for the purpose of map updating and selects the most favored next position as the planned one. Efforts are made in searching the optimal position by a dynamic programming algorithm with constraints from previous searching results.

Distributed robot systems using autonomous manipulators (arms) are widely employed in complex tasks such as object pushing, handling, tumbling, or combinations of them. In such cases, the problem of arm reconfiguration is encountered if the object has to be moved into another orientation. In the paper, we describe an object-pushing robot system in which emphasis is placed on efficient planning on the changing of pushing positions, which we refer to as arm repositioning. A new algorithm is proposed for the repositioning mainly based on evaluation of the object stability and load distribution. The applicability of the algorithm is illustrated by results of experiments using simulated or real arm systems.

Several methods have been proposed for building 3D object models from multiple range images mainly on the basis of the deformable models and pseudo-dynamics. However, the patches are usually placed crossing over the edge regions, and the surface orientation discontinuities cannot be properly preserved as a result. This paper proposes a new 3D object modeling method that generates refined triangular meshes while preserving the surface discontinuities. We formulate the process into a problem of minimizing an energy functional by using the idea of line processes and Markov random fields. Moreover, we show that near-optimal solutions can be obtained by utilizing the continuation method.

The paper presents a method of creating a complete model of a curved object from a sequence of range images acquired by a fixed range finder. To accomplish the modeling fast and accurately in an optimal manner, we propose a new online viewpoint planning algorithm to choose the next best viewpoint (NBV) based on the already obtained partial model. The NBV is determined by evaluating factors such as possibility of merging new data, local shape changes, registration accuracy and control point distribution.

The paper proposes a new approach to 3D object modeling by integrating superquadric-fitting and segmentation into a top-down, recursive algorithm. Given sensor data, which are a set of multiview range data covering the whole object surface, the method begins with an initial approximation of the object by fitting a single superquadric. The fitting residuals are then examined to pick up data points either in deep concave regions or too far away from the approximating surface. A dividing plane is extracted from the points to partition the original data set into two disjoint subsets, which are further treated respectively with the same fitting-and-splitting scheme. This process is repeated until the whole data are decomposed into primitive superquadrics all within some preset error tolerance.

Automatic 3D object modeling is a problem to be solved for many vision applications. In this paper, we propose a method of constructing a B-rep model of a 3D object from multi-view range images by utilizing a dynamic balloon scheme. The model is represented with triangular meshes, and by dynamic subdivision of the triangles, the meshes can approximate the local shapes adaptively. Moreover, it is also possible to derive a hierarchical mesh representation, which is controlled by a sequence of prescribed error tolerances. Results of experiments using real range images are presented to show the practical feasibility of the method.

Provides a grasp planning algorithm for a multifingered hand-arm robot in the presence of obstacles. It is difficult to generate a grasp configuration for a multifingered hand-arm robot even though geometric information about an object and obstacles are all known, since a multifingered hand-arm robot has many degrees of freedom of motion. We don't give any information about a grasp position and a grasp shape of the hand in advance. Our grasp planning algorithm is composed of four steps. At the first step, we determine the grasp plane (GP) which includes contact points of fingertips. The feature of our grasp planning is to determine the GP according to the local structure of empty space around an object so that we generate the grasp configuration rapidly. In order to evaluate the structure of empty space around an object, we use a geodesic dome and distance transformation. After generation of the GP, we determine the contact point (CP) where the fingertip contacts an object surface. In the third step, we determine the wrist position (WP) where the hand grasps an object and the arm doesn't collide with obstacles. The actual grasp configuration of the multifingered hand-arm robot is generated based on the obtained CP and WP The effectiveness of our grasp planning algorithm is clarified by computer simulations with 3D graphics.

We describe a real-time model-based sensing system to estimate the pose of an object manipulated by a multifingered hand in a realistic environment. The system uses the least-squares method to estimate the pose, on the assumption that the approximate initial pose and geometric shapes of the object and fingertips are known. 3D measurement of features on the surface of the object is achieved by visual tracking with stereo camera using a template matching method. The primary features used for visual tracking are distinguishable texture patterns on the object surface. In complex environments, visual tracking can be failed due to the various occlusion. The system solves the occlusion problem by detecting the occlusion and using contact information of fingertips with the 3D visual information. Actual implementation of the system is explained together with results of experiments about the stability of tracking, the accuracy of estimation, and the processing time of the system.

For mobile robots working in a dynamic environment, there is a great need for a global environment map that should be updated timely. In the paper, we propose methods for the map updating purpose by using a specialized mobile robot. The map updating system is composed of two basic processing components: local change detection and global observation planning. Principal strategies for the observation position planning are developed making good use of knowledge both from an outmoded initial map and from up-to-date sensory information. The paper gives a detailed description of the new method: moving around object clusters. Simulation results show this method is efficient even for detecting large changes in a complex environment.

The paper addresses the problem of building close-surface (CS) models of 3-D objects by integrating multi-view range images of the object. For the purpose, we have to use overlapping regions between the images to register them into a common view-independent coordinate system. In this paper, we propose a method of removing redundancy in the registered multi-view data due to the image overlap. It is assumed that each image has been triangulated into a connected mesh after the image acquisition. We firstly remove triangle edges in the overlapping regions, and then reconstruct the regions by forcing both border lines to advance toward each other in an optimal retriangulation procedure.

Distributed robot systems using autonomous manipulators are widely employed in complex tasks such as object pushing, handling, tumbling, or combinations of them. In such cases, the problem of arm reconfiguration is encountered if the object has to be moved into another orientation. In the paper, we describe an object-pushing robot system in which emphasis is placed on efficient planning on the changing of pushing positions, which we call arm repositioning. A new algorithm is proposed for the repositioning mainly based on an evaluation of the object stability and load distribution. The applicability of the algorithm is illustrated by results of experiments using simulated or real arm systems.

This paper focuses on motion planning of a manipulator equipped with a parallel two-fingered gripper to execute a primitive task, i.e., moving an object to a goal pose. In a complex situation, the goal cannot be achieved in a single grasp, We propose a new, realistic approach to achieve the complex goal. In this approach, the goal can be reached by intelligent manipulations, which are planned by intelligently selecting and combining simple manipulations. This paper describes a method of motion planning of intelligent manipulations consisting of picking, placing, sliding, and rotating manipulations by a parallel two-fingered gripper. The rotating manipulation by gravity has been realized by a new small and simple mechanism we have developed and attached to a parallel two-fingered gripper. This system plans motions by adaptively selecting and combining these four simple manipulations to achieve the goal. We describe the implementation including motion planning for rotating manipulations, a searching algorithm, results of our application and the validity of this system.

For mobile robots working in a dynamic environment, there is a great need for a global environment map that is updated frequently. In the paper, we propose a new method by which a mobile robot with an original environment map automatically detects changes in the environment and modifies the map into a new version. The changes are identified by performing robot localization and change detection alternatively at some planned observation positions. To deal with robot position errors caused by dead reckoning, available information on the original map and a local map recording already detected changes is utilized as much as possible. Simulation results show this method is efficient even for detecting large changes in a complex environment.

This paper addresses the problem of mutual collision avoidance of mobile robots in a narrow and uncertain environment. The minimal application of inter-robot communication is required to develop an autonomous and flexible mobile robot in such an environment. In order to reduce communication application function of action observation is introduced into a robot. A mobile robot gains more information about the local working environment by observing actions of other robots. It always tries to plan motions for avoiding a collision with its own sensory data. The effectiveness of this approach on collision avoidance is verified with simulation experiments.

Humans have numerous manipulation skills which they can intelligently combine to execute tasks. The authors have developed motion planning systems for intelligent manipulations using parallel two-fingered grippers. They propose a new method to integrate these motion planning systems. The basic idea of this method is the state-space approach which is described in states and operators and solves problems by searching state-space from initial state to goal state. The operators correspond to the primitive manipulation skills and transform a state into another state. This method was applied to a manipulator equipped with a parallel two-fingered gripper. The key idea behind this application is to augment the handling ability of a parallel two-fingered gripper, instead of developing a difficult and complicated control method that uses a sophisticated but weak multifingered gripper. To actually implement this idea, we have developed a new, small and simple mechanism attached to a parallel two-fingered gripper.>

The authors describe a motion planning system for sliding operations with a parallel two-fingered gripper used to escape from a situation where an object cannot be grasped. First they describe the problems of escaping from a situation where an object cannot be grasped and the actions to solve them. Then they propose a new method for planning manipulator motions for these actions using sliding operations based on geometrical information about the environments. The results of the method and its validity are shown.

The authors introduce space structuring based on the applicability of different path planning methods as a promising way to improve efficiency in mobile robot path planning. They present a preprocessing of the environment populated with simple polygonal obstacles aimed at the early discovery of a collision-free path for a mobile robot among obstacles between a start configuration and a goal configuration in cluttered environments, prior to the application of a path planning method. The preprocessing is done only once for one robot and one environment. Space structuring is based on obstacle growing with circles relative to the robot. Experimental results show the practicality of the method.>

A model-based robot system is described. This system was developed to replace and improve upon the conventional robot control scheme of teaching-by-showing and replay. The key concept of this system consists of the combination of manipulation skills with an environment model. The model provides the geometric structure and the physical properties of the objects in the environment. Manipulation skills enable reliable task execution in the presence of unavoidable errors and uncertainties. Skills are implemented within the hybrid position/velocity/force control scheme and incorporate sensing procedures to detect the state to be achieved. An experiment involving the disassembly of a valve using this system is presented.>

A model-based robot system is described. This system was developed to replace and improve upon the conventional robot control scheme of teaching-by-showing and replay. The key concept of this system consists of the combination of manipulation skills with an environment model. The model provides the geometric structure and the physical properties of the objects in the environment. Manipulation skills enable reliable task execution, in the presence of unavoidable errors, and uncertainties. Skills are implemented within the hybrid position/velocity/force control scheme and incorporate sensing procedures to detect the state to be achieved. An experiment involving the disassembly of a valve using this system is presented.>

Deals with methods using approximate cells decomposition of the configuration space in path planning for mobile robots in the plane. It is shown that optimal decomposition can be attained in the case of a polygonal robot amidst convex polygonal obstacles by choosing as reference point the center of the ring of minimal surface enclosing the robot, and gives as main result the position of this reference point.>

This paper describes a method of planning grasps for parallel two-fingered grippers in order to pick up user-specified objects and place them at user-specified configurations. By characterizing local space around gripping positions, the method can plan grasps where the hand can move as freely as possible and then easily plan a path. It can plan regrasping where the grasps themselves and the regrasping placements are computed flexibly according to the environment. It can plan minimal sequences of regraspings for objects that have rotational symmetries.>

An efficient method that finds collision-free paths for a manipulator with six revolute joints is described. A global structure of the free space in the three-dimensional (3-D) joint space of the arm of a large open space in the workspace, defined as a space where any orientation of the hand is possible, is represented in this description.>

The development of high-level manipulation languages will cause an explosion in the number of declarations about the real world environment. Procedures may be written in a few higher level statements, while world descriptions will still require hundreds. Without these descriptions, manipulation procedures can not be executed. Since the world model is a complex data base describing three-dimensional objects, it is not easy to construct. A system is proposed which enables the user to construct models of objects interactively. The system provides a laser pointer as a nieasuring tool and a display monitor for visualizing the internal data base and the real world.