Analyzing the auditory scene of real environments is challenging partly because an unknown number and type of sound sources are observed at the same time and partly because these sounds are observed on a significantly different sound pressure level at the microphone. These are difficult problems even with state-of-the-art sound source localization and separation methods. In this paper, we exploit two such methods using a microphone array: (1) Bayesian nonparametric microphone array processing (BNP-MAP), which is capable of separating and localizing sound sources when the number of sound sources is unspecified, and (2) robot audition software “HARK” is capable of separating and localizing in real time. Through experimentation, we found that BNP-MAP is more robust against differences in the sound pressure levels of the source signals and in the spatial closeness of source positions. Experiments analyzing real scenes of human conversations recorded in a big exhibition hall and bird calling recorded at a natural park demonstrate the efficacy and applicability of BNP-MAP.

We propose a method of extracting watermark from 3D prints created from 3D mesh data. The watermark is embedded to the 3D mesh in the spectral domain using a robust, imperceptible, and informed algorithm based on the spread spectrum technique, and then extracted from 3D prints by reconstructing the 3D mesh homologous to the original. A suspect 3D mesh is reconstructed accurately and robustly from noisy and incomplete 3D scans of the 3D prints, by optimizing a sparse subset of the spectrum using a variant of the iterative closest point technique. The mesh is registered to the original mesh by construction, in terms of geometry and topology, which allows us to extract a suspect watermark by simple algebraic operations. Comprehensive experiments have been conducted to demonstrate the performance of our method using standard 3D datasets and multiple 3D printers. The proposed method significantly outperforms prior methods under different conditions in the practical scenario. The probability of false positive detection is kept less than 106 for the most cases of simulated and real data.

Legged robots have unique capabilities to traverse complex environments by stepping over and onto objects. Many footstep planners have been developed to take advantage of these capabilities. However, legged robots also have inherent constraints such as a maximum step height and distance. These constraints typically limit their reachable space, independent of footstep planning. Thus, we propose that robots such as humanoid robots that have manipulation capabilities should use them. A robot should autonomously modify its environment if necessary. We present a system that enabled a real robot to use a box to create itself a stair step or place a board on the ground to cross a gap, allowing it to reach its otherwise unreachable goal configuration.

This paper demonstrates some experimental proofs of the model for the classification and identification of robot sensing data. Autonomous robots are equipped with varied sensors to assist them in understanding and interacting with their environments. In contrast to traditional model approaches that are based on the Gaussian assumption, we propose the application of the infinite Gaussian mixture model (iGMM) to detect known and unknown data. Two key components are denoted: 1) simultaneous training of the number of classes and dimensions of each model, and 2) infinite modeling to adjust for observations that do not match with previous knowledge.

For an autonomous mobile robot that works in real-world environment, recognition of its surrounding environment is necessary. This paper presents an object identification system, which can identify known and unknown objects and estimate their locations using 3D point cloud data acquired from a 3D LIDAR sensor mounted on the mobile robot. The proposed system is divided into two main steps; the first step is segmentation based on the 3D point cloud and the second step is identification of the extracted objects. The 3D LIDAR sensor gives sparse 3D shape information accurately, and covers a wide range. One of the main problems of such data is that the object shape information varies according to object's orientation and distance from the sensor. To solve this problem, we use nested infinite Gaussian mixture models for object identification. The experimental results show that the proposed system can extract various types of objects and identify both known and unknown objects.

We present a method of modeling the three-dimensional shape of a deforming object from a time-series of range images. A non-rigid variant of ICP algorithm is proposed to solve the correspondence between the range images. The deformation is modeled by the combination of piecewise similarity transformation using differential coordinates, and solved stably and efficiently by alternating minimization of a quadratic objective function. The performance of the proposed method is demonstrated by conducting two modeling tasks. First, the complete shapes of human hand and full-body are reconstructed by integrating the range images acquired by a single depth camera. Second, the shape completion of four-dimensional scan data is carried out by fitting a template model to the point clouds by extending our registration technique in the spatiotemporal domain.

The paper proposes two modules for a mobile robot audition system: 1) recognizing surrounding acoustic event, 2) tracking moving sound sources. We propose nested infinite Gaussian mixture model (iGMM) for recognizing frame based feature vectors. The main advantage is that the number of classes is allowed to increase without bound, if necessary, to represent unknown audio input. The multiple hypothesis tracking module provides time-series of separated audio stream using localized directions and recognition results at each frame. Not only for continuous sounds, the proposed tracker automatically detects appearing and disappearing point of stream from multiple hypothesis. These two modules are connected to microphone array based sound localization and separation, and the combined robot audition system achieved tracking of multiple moving sounds including intermittent sound source.

Robotic mapping such as two-or three dimensional occupancy grid map has been a field of study. However such occupancy maps are not enough for advanced tasks including object search, object grasping and change detection. We focus on tables and tabletop objects because a table is one of the most important tool in human activity. This paper describes an observation system of tabletop objects based on 3D-sensors and environmental map. We developed a mobile robot equipped with a Velodyne HDL-32E Lidar sensor and a Microsoft Kinect RGB-D sensor.

This paper proposes a method to segment out a hand grasped object from human region obtained from Kinect sensor by using body dimension database. Having dataset of human body dimensions, Multiple Regression Analysis is applied to find out the best explanatory variables for forearm and upper arm length. As a result, "body height" is selected. In order to measure "body height" accurately, Kinect depth image is utilized to search with kinematical result obtained from Kinect software. After estimating wrist position, we can segment out hand grasped region. Methods and experimental results are shown.

Real time localisation of mobile robots within dense 3D polygon maps is computationally expensive. Computation can be reduced by localising against an appropriate sub-set of polygons, and methods have been suggested as to how to select particular sub-sets. In the absence of detailed experimental results it can be difficult to determine if a sub-set is useful for localization. In this study we propose a method for evaluating sets of polygons for their localization utility within an environment map, or a sub region thereof, based on the rate of the change in expected sensor measurements against change in position. The measure is described, a localization "map" of an example environment presented, and the measure is used to compare various polygon sub-sets. Finally predicted localisation utility is evaluated against actual localisation results.

The present paper proposes a walking control of a humanoid that uses heel and toe edge landing for better adaptability to rough terrain. We focus on the control of forward straight walking and the roughness, such as changes in level and inclination, along the walking direction is explicitly considered herein. The requirements for designing the walking parameters, such as the step length and the step cycle for continuous walking, are discussed based on assumptions on the roughness of the terrain. An online adaptation strategy that includes change in stepping length, step cycle, and landing edge is discussed. The adaptation strategy uses the actual center of mass motion and the contact status to the ground for changing the stepping length, the step cycle, and the landing edge. We extended a walking pattern generation method that can handle the permissible region of the ZMP for balance recovery to a version that can also simultaneously handle changes in stepping position if needed. This method is used in the abovementioned discussions. A compensation method for the effect of the multibody system is also presented, so that these discussions can be made applicable to the actual humanoid system.

The present paper introduces a microphone array design and describes the results of an evaluation of the developed microphone array. We propose an evaluation index of the directional characteristics of beamforming to optimize microphone array design. Using beamforming simulation, we obtain a microphone arrangement that minimizes sidelobes and improves the basic performance of beamforming. This microphone arrangement has 64 microphones arranged in a 350-mm-diameter sphere designed to be mounted on a mobile robot and omni-directional directivity in azimuth and elevation. The performance of the proposed microphone array is verified in different real environments. The experimental results of sound localization demonstrate the effectiveness of the array in challenging environments and the robustness of the proposed microphone array for different pressure sound sources to cover larger areas.

A walking biped robot is required to change its walking direction and speed with minimal delay and not to tip off even when subjected to an unexpected external force. A major drawback of Zero Moment Point (ZMP) based online walking pattern generation methods is that arbitrary ZMPs cannot be achieved without divergence of the Conter of Mass (CoM). In this paper, we propose a new online walking trajectory generation method that utilizes nondivergence conditions of ZMP-CoM in a successive manner. This method enables changes in the walking direction and speed of a robot and push recovery under an unknown external force in unifie form. Experiments confir walking change of up to 4.05 km/h in the speed and changes in the walking direction with minimum delay and successful push recovery under an implse of 22 Ns.

This paper describes a method to estimate a set of body dimensions of a subject using the Microsoft KINECT sensor. Principal Component Analysis(PCA) of the AIST anthropometric database shows first and second principal components are interpreted as scale and degree of obesity respectively. A few body dimensions were chosen by linear multiple regression analysis. Applying the resulting estimator to a given subject using the KINECT sensor to obtain joint positions and 3 body dimensions by analyzing depth image, we can estimate a set of body dimensions for the subject. The system is described and experimental results are shown.

This paper presents a grasp planning method that takes into account center of gravity of objects in a pre-computation phase, thereby improving an existing method for grasp planning in complex scenes[1]. In the pre-computation phase, the system additionally calculates the distance from fingertips to the center of gravity of an object to use in ranking different proposed grasps, in order to achieve stable grasping with less force. Then we implement the proposed system in the OpenRAVE[2] environment that provides the original method[1]. Evaluation experiments were conducted with the Kawada Nextage[3] robot fitted with our original hand. Results show that the proposed method always grasps target objects closer to their center of gravity.

We present an actuated handheld puppet system for controlling the posture of a virtual character. Physical puppet devices have been used in the past to intuitively control character posture. In our research, an actuator is added to each joint of such an input device to provide physical feedback to the user. This enhancement offers many benefits. First, the user can upload pre-defined postures to the device to save time. Second, the system is capable of dynamically adjusting joint stiffness to counteract gravity, while allowing control to be maintained with relatively little force. Third, the system supports natural human body behaviors, such as whole-body reaching and joint coupling. This paper describes the user interface and implementation of the proposed technique and reports the results of expert evaluation. We also conducted two user studies to evaluate the effectiveness of our method.

This paper presents an online generation method of a torso height trajectory as a part of walking pattern generation. Torso height trajectories should be designed carefully in order to meet the constraints that are given by the existence of the inverse kinematics solution and limitations of the leg joint angle velocity. Limitations of the height in the future are evaluated and taken into consideration for deciding the height of the torso at a moment. In this manner a trajectory that satisfies the limitations and given maximum acceleration and velocity is generated. The proposed method is implemented as a part of online walking control system of full-size Humanoid HRP-2. Its performance is confirmed through the pattern generation in the walking control system.

Autonomous navigation for vehicles in non-trivial environments requires fast and precise localisation. Autonomous vehicles typically operate in 3D environments and require 6DOF localisation, which is computationally costly. We have developed fast, accurate localisation by constraining the 6DOF search space by physical vehicle limitations, and by the use of Graphics Processing Units (GPU's) to enable the use of dense 3D internal representations of the environment. This paper presents a method of selecting sub sets of polygons from the 3D map for use in localisation, in order to further reduce computational load, while maintaining localisation accuracy. Polygons are indexed by spatial location and a subset is selected by ranking visible polygons by observation frequency, angle of incidence to sensor ray and likelihood to fall within the sensor field of view. Pre-computed sets of polygons are then used to perform 6DOF localisation. An autonomous navigation experiment at a power sub-station facility is presented. Localisation accuracy is maintained while computation costs are reduced by a third.

A function to find objects and to estimate their 6D poses(x, y, z, pitch, yaw, roll) is crucial for a home service robot that works in human living environment. If a robot obtains 6D poses of target object, it can bring to that and use as tools. We have proposed SIFT-Cloud-Model (SCM) [1], that is designed to represent 3D objects with textures by 1) SIFT feature descriptors, 2) their 3D positions, and 3) their observed directions (hereafter "view vectors"). This model is used to find target objects in a scene and to estimate their relative 6D poses. In this paper, we propose the SIFT-Cloud-Model generation method based on structure from motion technique combined with optimization technique. Before this research, we had only one model. So we built this system of efficiently generating models to save our time and evaluated its to confirm the effectiveness of SCM to more objects. Finally experimental results of its accuracy evaluation and computational cost will be shown.

The present paper addresses strategies of changing the reference trajectories of the future ZMP that are used for online repetitive walking pattern generation. Walking pattern generation operates with a cycle of 20 [ms], and the reference ZMP trajectory is adjusted according to the current actual motion status in order to maintain the current balance. Three different strategies are considered for adjusting the ZMP. The first strategy is to change the reference ZMP inside the sole area. The second strategy is to change the position of the next step, and the third strategy is to change the duration of the current step. The manner in which these changes affect the current balance and how to combine the three strategies are discussed. The proposed methods are implemented as part of an online walking control system with short cycle pattern generation and are evaluated using the HRP-2 full-sized humanoid robot.

This paper describes a 3D textured map generation method for autonomous vehicle in urban outdoor environment, where GPS signals can not be reached. Constructed map will be used for short cycle and accurate localization and for obstacle detection using onbody laser scanner. In order to build dense 3D polygon map, planar feature of laser scanner input is extracted. They are associated and transformation matrices in between each scan point were iteratively solved. Aligned points were converted into texture mapped 3D polygons. 400×350[m] area in Univ. of Tokyo were scanned at 59 scan points, and 3D polygon map consists of 14M polygon were obtained. Experimental results of localization and autonomous path following two-wheeled inverted mobile robot PMR are shown.

A function to find objects and to estimate their poses is crucial for a home service robot that works in human living environment.

The paper presents a multiple sound sources mapping system from a robot embedded microphone array. The robot localizes sound direction and recognizes what sound it is while the robot is in motion. Then the system estimates the positions of the sound sources using triangulation from a short time period of directional localization results. Three key components are denoted: (1) accurate directional localization and separation of multiple sound sources using a microphone array (2) separated sound recognition from a several tens of milliseconds input signal (3) sound position estimation using the RAndom SAmple Consensus (RANSAC) algorithm from a tracked sound stream. By combining these techniques, the proposed system provides surrounding sound information: “Where does the sound come from?” and “What is the sound?”. It works with short term signal input, and is helpful to initially notice surrounding events.

This paper reports real-time virtual haptization of, for example, a remote object surface based on high-speed 3D shape measurement. With this system, the operator feels sense of touch as if tracing the surface of an object. We use a high-speed projector-camera system to achieve high-speed measurement of the object surface so that high-frequency haptic feedback is maintained at a haptic device PHANToM. An actively directed structured light is projected so that the projected slit ray intersects with the haptic interaction point, which is determined by the stylus position of the haptic device. A camera image of this scene gives the positional relationship between the object surface and the haptic interaction point. We use this relationship to generate force feedback such that the haptic interaction point is pushed back to a point on the intersection of the slit ray and the object surface. Experimental results show that the proposed system displays an object shape correctly and that the high-speed measurement contributes to displaying smooth motion of an moving object.

Autonomous navigation for vehicles in non-trivial environments requires fast and precise localisation. Vehicle control based on erroneous localisation can lead to motion from which it is costly to recover. Autonomous vehicles typically operate in 3D environments and require 6DOF localisation, which is computationally costly. This paper reports the development of an accurate 6DOF particle filter based localisation system which is suitable for localisation of autonomous vehicles operating in 3D environments. Fast, accurate localisation is achieved by constraining the 6DOF search space by physical vehicle limitations, and by the use of Graphics Processing Units (GPU's) to enable the use of dense 3D internal representations of the environments in the localisation process. Experimental results of the localisation system controlling autonomous navigation are presented, with an average localisation error of approximately 10cm.

This paper describes a particle filter based sound source mapping system that builds 2D sound source maps from directional sound readings taken from a mobile robot. The method uses a sound source localization model that is represented by Gaussian distribution for both direction and distance. To do this, accurate directional localization of sound sources is required, and two key components have been developed to achieve this: 1) a 32ch low side-lobe microphone array that is designed by beam forming simulation to have a) an omni-directional response, b) a narrower main-lobe, and c) lower side-lobes, in 700-2500[Hz] acoustic signals; 2) directional localization of different pressure sound sources by combining the Delay and Sum Beam Forming (DSBF) and the Frequency Band Selection (FBS) methods. Finally, experimental results show the proposed method can map sound sources in two dimesionals with high accuracy (less than 50[cm] error).

We present a method for interactively guiding the navigation of a humanoid robot through complex terrain via an intuitive path-drawing interface. In contrast to full autonomy or direct teleoperation of the robot, the user suggests an overall global navigation route by ¿drawing¿ a path onto the environment while the robot is walking. The path is used by a footstep planner that searches online for a sequence of suitable footstep locations that follow the indicated path as closely as possible while respecting the robot dynamics and overall navigation safety. In this way, the planner provides the robot partial autonomy in selecting precise footstep sequences while the human operator retains high-level control of the global navigation route. We present experimental results of the complete system on the biped humanoid HRP-2 navigating on and around various platforms, chairs, and stairs. We use an augmented reality system so that interactively drawing paths on the world is intuitive and natural.

We present an approach to navigating a biped robot safely and efficiently through a complicated environment of previously unknown obstacles and terrain using only on-board sensing and odometry. Sensing of the environment is performed by a pivoting laser scanner, which continues to update the terrain representation as the robot walks. Safe stepping motions are planned from this data to follow the user's command, given in the form of an end goal, a rough path, or a joystick input. Results are demonstrated on a prototype robot in several environments.

This paper proposes a sound identification method for a mobile robot in home and office environment. We propose a simple sound database called Pitch-Cluster-Maps(PCMs) based on Vector Quantization approach. Binarized frequency spectrum is used for PCMs codebook generation. It can describe a variety of sound sources, not only voice, from short term sound input. The proposed PCMs sound identification requires several tens(msec) of sound input, and is suitable for a mobile robot application which condition is dynamically changing. We implemented the proposed method on our mobile robot audition system equipped with a 32ch microphone array. Robot noise reduction using proposed PCMs recognition is applied to each input signal of a microphone array. The performance of daily sound recognition for separated sound sources from robot in motion is evaluated.

We present a high-speed planning method with compact precomputed search trees using a new pruning method and evaluate the effectiveness and the efficiency of our precomputation planning. Its speed is faster than an A* planner in maps in which the obstacle rate is the same as indoor environments. Precomputed search trees are one way of reducing planning time; however, there is a time-memory trade off. Our precomputed search tree (PCS) is built with pruning based on a rule of constant memory, the maximum size pruning method (MSP) which is a preset ratio of pruning. Using MSP, we get a large precomputed search tree which is a reasonable size. Additionally, we apply the node selection strategy (NSS) to MSP. We extend the outer edge of the tree and enhance the path reachability. In maps less than 30% obstacle rates on a map, the runtime of precomputation planning is more than one order of magnitude faster than the planning without precomputed search trees. Our precomputed tree finds an optimal path in maps with 25% obstacle rates. Then our precomputation planning speedily produces the optimal path in indoor environments.

We present a new pruning method for compact precomputed search trees and evaluate the effectiveness and the efficiency of our precomputation planning with steering sets. Precomputed search trees are one method for reducing planning time; however, there is a time-memory trade off. Our precomputed search tree (PCS) is built with pruning based on a rule of constant memory, the maximum-size pruning method (MSP), which is the preset ratio of pruning. Using MSP, we get a large precomputed search tree of reasonable size. Additionally, we apply the node selection strategy (NSS) to MSP. We extend the outer edge of the tree and enhance the path reachability. In maps with less than a 12% obstacle rate, the runtime of precomputation planning is more than two orders of magnitude faster than the planning without precomputed search trees. Our precomputed search tree with steering sets finds an optimal path in the map of its obstacle rate at 20%. Then, our precomputation planning speedily produces the smooth optimal path in an indoor environment.

In order to understand utterance based human-robot interation, and to develop such a system, this paper initially analyzes how loud humans speak in a noisy environment. Experiments were conducted to measure

This video demonstrates a mixed reality (MR) environment which is constructed for development of autonomous behaviors of robots. Many kinds of functions are required to be integrated for realizing an autonomous behavior. For example, autonomous navigation of humanoid robots needs functions, such as, recognition of environment, localization and mapping, path planning, gait planning, dynamically stable biped walking pattern generation, and sensor feedback stabilization of walking. Technologies to realize each function are well investigated by many research works. However, another effort is required for constructing an autonomous behavior by integrating those functions. We demonstrate a MR environment in which internal status of a robot, such as, sensor status, recognition results, planning results, and motion control parameters, can be projected to the environment and its body. We can understand intuitively how each function works as a part of total system in the real environment by using the proposed system, and it helps solving the integration problems. The overview of the system, projection of each internal status, and the application to an autonomous locomotion experiment are presented in the video clip.

We present results of successful telemanipulation of large, heavy objects by a humanoid robot. Using a single joystick the operator controls walking and whole body manipulation along arbitrary paths for up to ten minutes of continuous execution. The robot grasps, walks, pushes, pulls, turns and re-grasps a 55kg range of loads on casters. Our telemanipulation framework changes reference frames online to let the operator steer the robot in free walking, its hands in grasping and the object during mobile manipulation. In the case of manipulation, our system computes a robot motion that satisfies the commanded object path as well as the kinematic and dynamic constraints of the robot. Furthermore, we achieve increased robot stability by learning dynamic friction models of manipulated objects.

This paper describes a speech recognition system that detects basic voice commands for a mobile robot operating in a home space. The system recognizes arbitrary timed speech with position information in a noisy housing environment. The microphone array is attached to the ceiling, and localizes sound source direction in azimuth and elevation, then separates multiple sound sources using delay and sum beam forming (DSBF) and frequency band separation (FBS) algorithm. We implement the sound localization and separation method on our 32 channel microphone array. The separated sound source is recognized using an open source speech recognizer. These sound localization, separation and recognition functions are implemented as online processing in real world. We define four indices to evaluate the performance of the recognition system, and the efficiency in a noisy environment or with distant sound sources is confirmed from experiments in varied conditions. Finally, an application for a mobile robot interface is reported.

This paper reports on the development of a mobile robot system for operation within a house equipped with a ubiquitous sensor network.Human robot interaction is achieved through the combination of on-robot audio and laser range sensing and additional audio sensors mounted in the ceiling of the ubiquitous environment. The ceiling mounted microphone arrays can be used to summon a mobile robot from a location outside the robot's range of hearing. After the robot autonomously navigates to the desired location, the on-board microphone array can be used to locate the sound source and to recognise a series of greetings and commands.

This paper describes a mobile robot for home service purpose together with ubiquitous ceiling ultrasonic locator and microphone array. User call can be detected by those ubiquitous devices and then mobile robot navigates toward given location. After reaching to the given location, the robot tries to find out user location by using stereo camera, laser and onbody microphone array. We implemented those system at our experimental house "Rokko Holone". System components and experimental results are shown.

For humanoid robots to fully realize their biped potential in a three-dimensional world and step over, around or onto obstacles such as stairs, appropriate and efficient approaches to execution, planning and perception are required. To this end, we have accelerated a robust model-based three-dimensional tracking system by programmable graphics hardware to operate online at frame-rate during locomotion of a humanoid robot. The tracker recovers the full 6 degree-of- freedom pose of viewable objects relative to the robot. Leveraging the computational resources of the GPU for perception has enabled us to increase our tracker's robustness to the significant camera displacement and camera shake typically encountered during humanoid navigation. We have combined our approach with a footstep planner and a controller capable of adaptively adjusting the height of swing leg trajectories. The resulting integrated perception-planning-action system has allowed an HRP-2 humanoid robot to successfully and rapidly localize, approach and climb stairs, as well as to avoid obstacles during walking.

A sensor feedback framework that realizes robust walking of humanoids are discussed in the present paper. We have proposed an online walking control system that generates a dynamically stable motion pattern in short cycles, such as 40 [ms]. The system is capable of reflecting the actual motion status modified by sensor feedback to the pattern generation of the next period so that the long-term stability of the walking is maintained using the dynamic model. We herein propose three categories of adaptation as a framework for realizing robust walking via the short-cycle generation system: 1) absorption of the error of the dynamic model, 2) reactive adaptation to the disturbance, and 3) adjustment of the parameters that are used to generate the walking pattern. Feedback methods for each category are discussed and validated on the full-size humanoid HRP-2.

One of the problems in developing a humanoid robot is caused by the fact that intermediate results, such as what the robot perceives the environment, and how it plans its moving path are hard to be observed online in the physical environment. What developers can see is only the behavior. Therefore, they usually investigate logged data afterwards, to analyze how well each component worked, or which component was wrong in the total system. In this paper, we present a novel environment for robot development, in which intermediate results of the system are overlaid on physical space using mixed reality technology. Real-time observation enables the developers to see intuitively, in what situation the specific intermediate results are generated, and to understand how results of a component affected the total system. This feature makes the development efficient and precise. This environment also gives a human-robot interface that shows the robot internal state intuitively, not only in development, but also in operation.

This paper describes a pedestrian tracking system for a mobile robot. The system detects human position by finding pairs of legs using a laser range finder. It distinguishes between pedestrians and human like static objects by using a static environment map which indicates places of static objects in the environment in which the robot operates. The map is made from the past 200 frames (about 2.6 seconds) of data from a laser range finder. An experiment is reported which confirms the ability of the system to track pedestrians from on board a mobile robot in a daily use environment, even when the robot is in motion. The results also show that the system can distinguish between pedestrians and human like static objects.

This paper describes a path smoothing method that results in paths with clothoidal curvature and which includes a safety margin from obstacles. The method modifies grid based A* results, and tries to push the given path away from obstacles. A technique to take into account the initial direction of robot is also shown.

This paper describes the three-dimensional simulation and measurement of the sound pressure distribution generated by 120 ch plane loudspeaker array. The authors research the local high sound pressure distribution generated by loudspeaker array. We develop the 120 ch plane loudspeaker array. The 120 plane loudspeakers are arranged at the 120 ch plane loudspeaker array. The authors simulate the three dimensional sound pressure distribution generated by the 120 ch plane loudspeaker array. From the result of the simulation, the 120 ch plane loudspeaker array can generate three dimensional sound pressure distribution. The 120 ch plane loudspeaker array is developed based on the simulation result. Actual measurement of sound pressure distribution is performed with 64 ch microphones measurement system. We confirm that the 120 ch plane loudspeaker array can generate three dimensional local high sound pressure area from result of measurement. In addition, result of simulation is compared with result of measurement. The simulation is available for the simulation of the 120 ch plane loudspeaker array from result of comparison.

We present the concept of an "intelligent" joystick, an architecture which provides simple and intuitive high-level directional control of a legged robot while adjusting the actual foot placements autonomously to avoid stepping in undesirable places. The general concept can be likened to riding a horse: high-level commands are provided, while the "intelligence" of the underlying system selects proper foot placements with respect to the shape and properties of the underlying terrain and overall balance considerations. We demonstrate a prototype system used for realtime control of the humanoid robot HRP-2

In this paper, we present an algorithm for manipulability based trajectory generation for any serial manipulator that has an inverse kinematic model that can obtain all solutions. Our strategy is a search based approach that analyzes candidate configurations at discrete points along the work-space trajectory. Given such a model we prove the configuration-space trajectories generated are optimal within the limit of the discretization of the work-space trajectory

This paper describes a mobile robot for home service purpose together with ceiling attached ultrasonic locator and microphone array. User call can be detected by those ceiling devices and then mobile robot navigates toward given location. After reaching to the given location, the robot tries to find out user location by using stereo camera, laser and triconcentric microphone array. We implemented those system at our experimental house "Rokko Holone". System components and experimental results are shown

As navigation autonomy becomes an increasingly important research topic for biped humanoid robots, efficient approaches to perception and mapping that are suited to the unique characteristics of humanoids and their typical operating environments are required. This paper presents a system for online environment reconstruction that utilizes both external sensors for global localization, and on-body sensors for detailed local mapping. An external optical motion capture system is used to accurately localize on-board sensors that integrate successive 2D views of a calibrated camera and range measurements from a SwissRanger SR-2 time-of-flight sensor to construct global environment maps in real-time. Environment obstacle geometry is encoded in 2D occupancy grids and 2.5D height maps for navigation planning. We present an on-body implementation for the HRP-2 humanoid robot that, combined with a footstep planner, enables the robot to autonomously traverse dynamic environments containing unpredictably moving obstacles

This paper addresses a dynamically stable humanoid walking pattern generation system that can update the pattern at a short cycle such as 40 [ms]. Frequent update of the pattern enables a robot to reflect change of moving direction and upper body posture with short delay after the change of the command input. It also enables a robot to change the walking pattern according to the change of measurable applied force. Moreover short cycle generation is useful in constructing a system that reflects motion status modified by sensor feedback to the motion generation layer in order to maintain consistency of dynamics. Preview control theory is adopted to decide horizontal torso trajectory that realizes desired ZMP trajectory. The proposed system is implemented on a full-size humanoid HRP-2. Walking pattern is updated at every 40 [ms] with reflecting the current modified torso position in the system. Walking control is successfully carried out by the implemented system. Then an experiment of generating and executing walking pattern while variable force is applied to the hand is shown. The system generates walking pattern reflecting the force sensor information at the wrist while walking

The paper describes a 2D sound source mapping system for a mobile robot. We developed a multiple sound sources localization method for a mobile robot with a 32 channel concentric microphone array. The system can separate multiple moving sound sources using direction localization. Directional localization and separation of different pressure sound sources is achieved using the delay and sum beam forming (DSBF) and the frequency band selection (FBS) algorithm. Sound sources were mapped by using a wheeled robot equipped with the microphone array. The robot localizes sounds direction on the move and estimates sound sources position using triangulation. Assuming the movement of sound sources, the system set a time limit and uses only the last few seconds data. By using the random sample consensus (RANSAC) algorithm for position estimation, we achieved 2D multiple sound source mapping from time limited data with high accuracy. Also, moving sound source separation is experimentally demonstrated with segments of the DSBF enhanced signal derived from the localization process

This paper explores autonomous locomotion, reaching, grasping and manipulation for the domain of navigation among movable obstacles (NAMO). The robot perceives and constructs a model of an environment filled with various fixed and movable obstacles, and automatically plans a navigation strategy to reach a desired goal location. The planned strategy consists of a sequence of walking and compliant manipulation operations. It is executed by the robot with online feedback. We give an overview of our NAMO system, as well as provide details of the autonomous planning, online grasping and compliant hand positioning during dynamically-stable walking. Finally, we present results of a successful implementation running on the humanoid robot HRP-2

This paper describes a model of human walking and the development of the step measurement system. We have used a probabilistic method to make the model that focuses on the relationship between the orientation of the foot placement and where the other foot will land. We make a hypothesis that the foot turns to where someone wants to go in walking naturally to make the model. An experiment using a motion capture system (Vicon Motion Systems, 200 Hz) was conducted to prove the validity of the hypothesis and shows that we can predict the next landing point using the orientation of the foot. Also it indicates we can generate likely walk paths stochastically. The step measurement system we developed does not require anything but a motion capture system. Experiments show that this system can detect the timing of heel contact and toe lift-off with high accuracy.

In this paper, we explore preprocessing techniques aimed at efficiently mapping the workspace to the configuration space for redundant manipulators. Exploiting the increasing availability of memory we precompute a database for online use that allows fast querying of joint configurations with high manipulability. Such a system is useful for performing reaching and manipulation tasks, online force control, and quickly computing goal configurations for path planning algorithms. We have implemented a prototype system on the digital human version humanoid robot platform 2 (DH-HRP2) which we use to efficiently compute arm inverse kinematic solutions. We demonstrate the benefits of such a system applied to converting workspace trajectories to configuration space trajectories. Although we have applied the technique to finding joint configurations with high manipulability, our approach can be generalized to a variety of different priority measures that may be used for evaluating inverse kinematic solutions for redundant manipulators.

This paper describes an online 3D reconstruction system from stereo image sequences to obtain a dense local world model for robot navigation. The proposed method consists of three components: 1) stereo depth map calculation, 2) correspondence calculation in time sequential images by tracking raw image features, 3) 6DOF camera motion estimation by RANSAC and integrate depth map into 3D reconstructed model. We examined and evaluated our method in a motion capture environment for comparison. Finally experimental results of a humanoid robot H7 are denoted.

This paper describes a 32 /spl times/ 32 matrix scan type high-speed pressure sensor for the feet of humanoid robots that has 1 kHz sampling rate. This sensor has matrix scan circuit. The matrix scan method has a problem of interference by bypass current. To resolve this problem, we suggest a novel method using a very thin conductive rubber. We adopted a very thin (0.6 mm) force sensing conductive rubber sheet for high speed sensing. Each sensing area is 4.2 /spl times/ 7.0 mm and can measure vertical force of approximately 0.25-20 N. Walking cycle of humanoid robot as well as human being is about 0.4-0.8 s and dual leg phase is about 0.1-0.15 s. The target of the sensor is biped walk stabilization so that high-speed input is important. Matrix scan type circuit is connected to sensor, and the system runs 1 kHz with 14 bit resolution at 4.2 /spl times/ 7.0 mm grid for 32 /spl times/ 32 points, and the sensor size is the same as humanoid robot foot 135 /spl times/ 228 mm, The system is running high-speed because of the very thin conductive rubber and simultaneous measurement. The sensor system, a novel scan method, and evaluation results are described.

This paper describes a three ring microphone array estimates the horizontal/vertical direction and distance of sound sources and separates multiple sound sources for mobile robot audition. Arrangement of microphones is simulated and an optimized pattern which has three rings is implemented with 32 microphones. Sound localization and separation are achieved by delay and sum beam forming (DSBF) and frequency band selection (FBS). From on-line experiments results of sound horizontal and vertical localization, we confirmed that one or two sounds sources could be localized with an error of about 5 degrees and 200 to 300 mm in the case of the distance of about lm. The off-line experiments of sound separation were evaluated by power spectrums in each frequency of separated sounds, and we confirmed that an appropriate frequency band could be selected by DSBF and FBS. The system can separate 3 different pressure speech sources without drowning out.

A humanoid robot is a real-time system controlled by a complex computer system that requires huge computing power for perception and planning, high energy efficiency for self-contained control, reduction of physical dimensions, and high reliability. This paper proposes a distributed architecture for the humanoid robot control substituting conventional centralized control architectures. In addition to the parallelism that provides scalable computing power at low clock namely at low energy, the distributed architecture contributes to reliable operations by replacing many fragile analog signal wires with a digital network with redundant routes. In order to accomplish a real-time control over the network, RMTP (responsive multi-threaded processor) for parallel and real-time computation has been newly designed. RMTP can synchronize more than thirty nodes distributed over a robot body in less than 5 micro second with a real time network called the responsive link (RL). Architectures of RMTP, RL and Linux-based real-time system software are presented.

The purpose of this paper is to report the devised arrangement of a microphone array suitable for a mobile robot and to develop a robotic audition system to recognize the environment. The paper first describes the sum and delay beam forming (SDBF) algorithm and its common problem: side lobes. The array we developed shows smaller side lobes when beam forming. It provides high quality localization and separation for multiple sound sources. Then we achieved a sound sources mapping system by using a wheeled robot equipped with the microphone array. The robot localizes sound direction on the run and estimates sound positions using triangulation. Accumulation of data results in high accuracy. The system can estimate 3 different pressure sounds with a 200 mm position error. Moreover, the high quality sound source separation has proved useful in improving speech recognition.

This work describes two circular microphone arrays and a square microphone array which can be used for sound localization and sound capture. Sound capture by microphone array is achieved by sum and delay beam former (SDBF). A dedicated PCI 128-channel simultaneous input analog-to-digital (AD) board is developed for a 128 ch microphone array with a maximum sampling rate of 22. 7 /spl mu/s/sample. Simulation of sound pressure distribution of 24 and 128 ch circular microphone array and 128 ch square microphone array are shown. Then a 24 ch circular microphone array and a 128 ch square microphone array have been developed. The 24 ch circular microphone array can capture sound from an arbitrary direction. The 128 ch square microphone array can capture sound from a specific point. Both systems are evaluated by using frequency components of the sound. The circular type system can be used on a mobile robot including humanoid robot and square type can be extended towards room coverage type application.

This paper presents a novel human-machine interface, named invisible messenger. It integrates real time visual tracking of face and sound beam forming by speaker array, and realizes remote whispering effect as if an invisible messenger were standing by you. Construction of a working prototype system and actual measurement using the system prove the feasibility and effectiveness of the proposed idea.

A method for probabilistic registration of stereo vision and sonar sensor data within 2.5D maps is presented. Both sensor modalities have characteristics which can limit available localisation information. Resulting localisation estimates can be inaccurate and can also vary widely within and between samples. Such errors are not so prohibitive in pure localisation problems, but prove catastrophic in SLAM applications. Fusion of vision and sonar data while considering height of objects in the environment is implemented to reduce such noise by taking advantage of localisation cues from both modalities. In addition, an approximation of the variance within localisation estimates is used to form a measure of localisation confidence. This measure of belief can inform the mapping task so sensor data sampled from uncertain robot poses can be integrated into the map with a reduced probability. This process not only results in improved maps, but subsequently in more accurate localisation estimates.

We present an integrated humanoid locomotion and online terrain modeling system using stereo vision. From a 3D depth map, a 2.5D probabilistic description of the nearby terrain is generated. The depth map is calculated from a pair of stereo camera images, correlation-based localization is performed, and candidate planar walking surfaces are extracted. The results are used to update a probabilistic map of the terrain, which is input to an online footstep planning system. Experimental results are shown using the humanoid robot H7, which was designed as a research platform for intelligent humanoid robotics.

We present an online algorithm for planning sequences of footstep locations that encode goal-directed navigation strategies for humanoid robots. Planning footsteps is more general than most existing navigation methods designed for wheeled robots, since the options of stepping over or upon obstacles in a cluttered terrain are available. Given a discrete set of plausible footstep locations, a forward dynamic programming approach is used to compute a footstep sequence to a specified goal location in the environment. Heuristics designed to minimize the number and complexity of the step motions are used to encode cost functions used for searching a footstep transition graph. If successful, the planner returns an optimal sequence of footstep locations according to the cost functions and plausible sets of footstep locations defined. We show results from an experimental implementation of the algorithm running on the H7 humanoid robot. Using a stereo vision system to sense obstacles in the immediate environment and identify a target goal location, the robot updates the current optimal footstep sequence to the goal from its present location.

We present a humanoid walking control system that generates body trajectories to follow a given desired motion online. A layered software and control architecture is used to aggregate system components and provide a framework for high-level autonomous locomotion behaviors. Walking characteristics such as desired torso movements, upper body posture and step cycles, can be specified and used to generate stable whole-body walking trajectories online. The basic architecture consists of 4 layers: footstep planning, trajectory generation, trajectory modification from sensor feedback, and joint servo control. In order to test the complete system, we implemented an experimental top-level behavior that enables the robot to track and follow a moving target based on stereo vision feedback. We explain the overall system design and components, and present results based on our current implementation using the humanoid robot 'H7'.

This paper describes an experimental stereo vision based motion planning system for humanoid robots. The goal is to automatically generate arm trajectories that avoid obstacles in unknown environments from high-level task commands. Our system consists of three components: 1) environment sensing using stereo vision with disparity map generation and on-line consistency checking, 2) probabilistic mesh modeling in order to accumulate continuous vision input, and 3) motion planning for the robot arm using RRTs (rapidly exploring random trees). We demonstrate results from experiments using an implementation designed for the humanoid robot H7.

This paper presents a method of visually steerable sound beam forming. The method is a combination of face detection and tracking by motion image processing and sound beam forming by speaker array. Direction towards a target person can be obtained by the face tracking in real-time. By continuously updating the sound beam direction with the result of the face detection and tracking, the system is possible to keep transmitting sounds towards the target person selectively, even if he or she moves around. Experimental results prove the feasibility and effectiveness of the method.

This paper describes sound beam generation and capture by speaker array and microphone array system. 16ch online speaker array system generates sound beam towards appropriate directions. 128ch square speaker array system generates sound spot towards appropriate positions. Since it transmits necessary sound information to only a specific person, it can be used in quiet places, such as a museum. 8ch online microphone array system captures sound from appropriate direction. It can be used as ears of a humanoid robot. The authors call a position to want to generate/capture sound "focus". Focusing is achieved by sum and delay beam former. (SDBF) More than two focuses can be simultaneously generated/captured by speaker array/microphone array. Speaker array system is evaluated by using gantry and sound level meter that are controlled by PC system. Microphone array system is evaluated by using frequency ingredient of the sound.

We present an approach to self-collision detection suitable for complex articulated robots such as humanoids. Preventing self-collisions is vital for the safe operation of robots that generate body trajectories online. Our approach uses a fast distance determination method for convex polyhedra in order to conservatively guarantee that a given trajectory is free of self-collision. Experimental results using an online joystick control application for the humanoid robot "H7" demonstrate the feasibility and effectiveness of the method.

Addresses the extension of a humanoid's action capability by attaching toe joints. The effectiveness of toe joints is discussed in three aspects. One is utilizing it to speed up the walking, another is using it to enable a humanoid to go up higher steps, and the other is using it to whole-body action in which knees are contacting the ground. Feet with toe joints are developed for humanoid 'H6'. An experiment of the wholebody action in which knees are contacting the ground is carried out to show the usefulness of toe joints for such actions. Then the walking pattern generation system is extended to use toe joints. Using this extended system maximum speed of knee joints can be reduced at the same walking speed, and 80% faster walking speed is achieved on humanoid 'H6'.

This paper describes a design of six-axis force sensor that measures ground reaction force of human or humanoid robot. The key concept is parallel support mechanisms, that allow large torques and forces which are caused when foot is hitting to the environment. Basic concept and design of parallel support mechanisms are denoted. Finally ground reaction force measurement system for human walking, and application to humanoid robot walking are described.

This paper describes the design of an integrated system for humanoid robotics that consists of three key components, 3D vision, motion planning and bipedal control. Layered system design is adopted to achieve concurrency as well as small latency. Implementation using our humanoid type robot H7 and experiments with this architecture are described. The H7 is expected to be a common test-bed in experiments and discussion for various aspects of intelligent humanoid robotics.

This paper presents an efficient online method to generate humanoid walking motions that satisfy desired upper body trajectories while simultaneously carrying objects. A fast motion pattern generation technique that follows the desired ZMP is adopted. In order to satisfy the control input given online, subsequent motion patterns are updated and connected in a stable manner to the old ones while executing. During the creation of motion trajectories online, the commanded motion parameters are checked and modified automatically considering the performance limitations of the hardware. As an example application, we have implemented a one step cycle control system on the Humanoid H7. Experiments controlling the upper body motion and walking direction using a joystick interface are explained to demonstrate the validity of the proposed method.

This paper presents a working prototype system of a novel computer-human interface, named invisible messenger. It integrates visual face detection and tracking, and speaker array signal processing. By speaker array it is possible to form acoustic focus at the arbitrary location that is measured by the face tracking. Thus the implemented system can whisper in a person's ear as if an invisible virtual messenger were standing by the person. In this paper, the authors describe both ideas behind the system and key technology for the implementation. In order to confirm the effectiveness of the system, the authors conduct experiments using it. This paper also reports the experiment. Experimental remits demonstrate the effectiveness of the proposed idea.

This paper describes a design of remote operation interface for humanoid robot with following three functions: (1) Body DOFs control interface through 3D robot model in a virtual environment, and it has two types of online stabilizing software for maintaining the body balance. (2) Environmental recognition display interface for sensors, sound and 3D vision. (3) Voice and face recognition interface for interacting with operator and a human in front of the robot body. Then, implementation and experiments on our humanoid robot H6 are denoted.

The H6 humanoid robot has been developed as a platform for the research on perception-action coupling in intelligent behaviour of humanoid type robots. The H6 features: 1) a body which has enough DOFs and each joint has enough torque for full body motion; 2) a PC/AT compatible high-performance on-board computer which is controlled by RT-Linux so that from low-level to high-level control is achieved simultaneously; 3) self-contained and connected to a network via radio Ethernet; and 4) dynamic walking trajectory generation, motion planning and 3D vision functions are available. The H6 can be used as a common test-bed in experiment for various aspects of intelligent humanoid robotics.

We present an approach to path planning for humanoid robots that computes dynamically-stable, collision-free trajectories from full-body posture goals. Given a geometric model of the environment and a statically-stable desired posture, we search the configuration space of the robot for a collision-free path that simultaneously satisfies dynamic balance constraints. We adapt existing randomized path planning techniques by imposing balance constraints on incremental search motions in order to maintain the overall dynamic stability of the final path. A dynamics filtering function that constrains the ZMP (zero moment point) trajectory is used as a post-processing step to transform statically-stable, collision-free paths into dynamically-stable, collision-free trajectories for the entire body. Although we have focused our experiments on biped robots with a humanoid shape, the method generally applies to any robot subject to balance constraints (legged or not). The algorithm is presented along with computed examples using the humanoid robot "H6".

This paper introduces and describes a novel method which enables the online generation of humanoid walking patterns that follow desired footprint locations. Online generation is realized by the dynamically stable mixture and connection of pre-designed motions. Characteristics of zero moment point (ZMP) are utilized in order to maintain the overall dynamic stability of "mixed" motions. Experiments using an online pattern generation software system and online walking control joystick interface for the humanoid robot H6 are introduced to show the validity of the method.

This paper describes the development of a plane segment finder, which is able to detect three-dimensional planar surfaces from input images in real-time. We propose an algorithm for detecting plane segments, that includes: 1) plane segment candidate extraction using the 3D Hough transform from depth map information; and 2) fitting the plane segment candidates to the depth map in order to detect the partial plane segment, since the extracted plane segment candidates are general planes, with no boundary. To achieve real-time plane segment finding system, we apply: 1) the recursive correlation method for depth map generation; and 2) the randomized Hough transform for plane segment extraction. Finally, experimental results using an implementation of our system along with a humanoid robot are shown.

This paper shows walking human avoidance and detection behavior of a mobile robot using the 3D depth flow. The 3D depth flow proposed is able to measure the 3D motion vector of every pixels between two time sequential images. First, a definition of the 3D depth flow and a simple 3D depth flow calculation method are presented. Then an implementation of the real-time 3D depth flow generation system using a standard PC is described, and the experimental results are given. Finally, as an application, the walking human detection and avoidance task using a mobile robot in real environments is shown.

We present an algorithm for planning safe navigation strategies for biped robots moving in obstacle-cluttered environments. From a discrete set of plausible statically-stable, single-step motions, a forward dynamic programming approach is used to compute a sequence of feasible footstep locations. In contrast to existing navigation strategies for mobile robots, our method is a global method that takes into account the unique ability of legged robots such as bipedal humanoids to traverse obstacles by stepping over them. Heuristics designed to minimize the number and complexity of the step motions are used to encode cost functions used for searching a footstep transition graph. We show preliminary results of an experimental implementation of the algorithm using a model of the H6 humanoid navigating on an office floor littered with obstacles.

This paper proposes a 3D depth flow generation method which measures 3D motion vector of every pixels between two time sequential images. First, the definition of 3D depth flow and a simple method in order to generate the 3D depth flow are described. Then the implementation of a real time 3D depth flow generation system using only a PC is presented, and experimental results are given. Finally, as an application, a walking human tracking task using mobile robot is included.

This paper describes the design and implementation of real-time depth map generation system with four key issues: 1) recursive (normalized) correlation technique; 2) cache optimization; 3) online consistency checking method; and 4) applying the MMX/SSE/sup R/ multimedia instructions set. The system is implemented on a standard PC/AT hardware with simple image capture board. Implementation details and system evaluations are also denoted. Furthermore, experiments with robots in real world are shown.

Physically-based simulation software is commonly used for developing and testing low-level robot control algorithms. In order to facilitate the development and evaluation of higher-level robot behaviors, broader-based simulations are needed. Examples include software for simulating 3D vision, motion planning for obstacle avoidance, and integrating vision and planning. In addition to modeling the general interaction between the robot and its environment, the software can be used as a graphical user interface for directly controlling or interacting with a robot operating in the real world. This paper describes our current efforts toward building a large-scale software simulation framework for the development and testing of high-level behaviors for humanoid robots. We view this as a potential useful tool for the visualization and development of robotic systems, as well as an interactive, task-level programming interface for robots.

A humanoid robot "H6" is developed as a platform for the research on perception-action coupling in intelligent behaviour of humanoid type robots. The H6 has the features as follows: 1) a body which has enough DOFs and each joint has enough torque for full body motion, 2) a PC/AT compatible on-board computer which is controlled by RT-linux so that low-level to high-level control is achieved simultaneously, 3) is self-contained and connected to a network via radio ethernet, 4) a 3D vision function can be applied. The H6 is expected to be a common test-bed for experiment and discussion for various aspects of intelligent humanoid robotics.

This paper describes a vision system which recognizes 3D objects in real-time by modeling the shapes of objects and matching the generated models. We develop the following methods for practically solving important problems of integration such as the estimation of sensor accuracy as well as real-time processing: 1) we reduce the computation of signed-distance, which is necessary to apply the marching cubes algorithm, and select the optimal resolution of models to be generated using an octree, thereby enabling us to generate hierarchical mesh models in real-time; 2) we apply spin-image matching by selecting the resolution of generated models and the coarse to-fine algorithm, consequently, we are able to efficiently match multiple objects of different sizes.

A legged robot "JROB-1" is developed for a robotics research platform as a result of inter-university research program on intelligent robotics supported by the Ministry of Education Grant-in-Aid for Scientific Research on Priority Areas in Japan. The JROB-1 features: 1) self-contained, 2) RT-Linux running on PG/AT processes vision and sensor processing, motion planning and control, 3) connected to a network via radio Ethernet as to utilize networked resources, 4) Fujitsu color tracking vision board and Hitachi general purpose vision processing board, 5) all parts are commercially available, and 6) it is extensible with respect to sensor, sensor processing hardware and software. JROB-1 is expected to be a common testbed for experiment and intelligent robotics research by integrating perception and motion.

View changing because of vibration while walking is one of the most fundamental problem for a vision based legged robot. To overcome this difficulty, three key issues are denoted: a) integration of color segmentation, optical flow and stereo, which is able to apply to vibrating view using correlation hardware, b) software servo loop implemented as a kernel module for the purpose of soft actuation, and c) smooth walking pattern generation using solid model and dynamics simulator. Furthermore, a quadruped legged robot "JROB-1" is developed as a platform for the research on perception-action coupling in intelligent behavior of robots.

A robot body is not only an active unit but also a sensing unit. This paper describes a humanoid that integrates stereo vision, full-body tactile sensor, sound sensors and wrist force sensors into a sensor image. The sensor image reflects the current state of the environment and is regarded as all the input to the robot brain in this system. The brain program processes a series of sensor images and controls its body using them. The robot is remote-brained and designed to be a testbed to do research on sensor based behaviors of a full-body humanoid. This paper describes the design concept and the details of the system.

In this paper, we describe the design implementation of this real-time layer for remote-brained robot. "Remote-brained approach" is a paradigm for software research by separating the robot brain from the robot body. The interface from the body to the brain top-level software is also denoted. Furthermore, an application of the reactive motion of humanoid-type robot is described. The real-time facility is fundamentally important for a robot that behaves in the read-world. To accomplish both large scale intelligent software and real-time functionality, we design the system by dividing the brain into a low-level real-time layer and a high-level action decision layer. The action decision layer is put on a WS, and the real-time layer is put on a transputer network.

We present design and implementation of a tactile sensor system, sensor suit, that covers the entire body of a robot. The sensor suit is designed to be soft and flexible and to have a large number of sensing regions. We have built the sensor suit using electrically conductive fabric and string. The current version of the sensor suit has 192 sensing regions. Each sensing region works as a binary switch in the current version. All of the signals from the sensor suit are gathered and superimposed on a visual image of the robot. The video multiplexer for the sensor signals is built on a field programmable gate array set. The construction of sensor suit system and its application with a full-body humanoid are presented.

We present a 35 DOF full-body humanoid is designed to be a testbed to integrate research tools for humanoid full-body behaviors. Each limb has 6 DOF. The neck has 3 DOF. Each hand has 4 DOF. The key idea of the system architecture is the remote-brained approach. This paper describes the design concept and the details of the system and introduces experiments to stand up using table for support and to perform action selection in reaching and grasping an object based on vision.

We present a vision-based real-time control system of a human-form robot in swing motion. The robot has 16 DOF joints and a camera which is designed to do research on human-form robot behaviors. The key idea to develop the vision based system for the robots are the remote-brained approach and real-time visual tracking. In this approach, the robot system is designed to have the brain and the body separate and to connect them by a radio link. The body sends the visual image to the brain and the body receives the motion, references. The basic algorithm to observe the swing motion by vision uses the optical flow generation. The control algorithm for flow-based visual feedback is described.

Constructing an environment for robot software research, which can program and experiment in various robot behaviors, design of a software development platform becomes important problem. This paper describes a design of "Software Platform" for real-time large scale robot software, and its implementation in the "Remote-Brained Robot Project". The Software Platform is designed as three layers, "MOTHER, BRAIN, SENSOR-MOTOR". MOTHER consists of tools to produce and evolve BRAIN programs. For tools and libraries in such platform, there are two major problems. One is, there is tradeoff relationship between "Extension" and "Share". A way of overcoming this problem is described. The other is, the methodology is needed between low-level real-time parallel environment monitoring program and high-level software that takes much time. A system developed for remote-brained robots and its application are described. This system aims at developing many sorts of brain architecture and high level software that consists of various flexible multiprocess network.

Presents a new type of robot which has two arms and two legs like an ape and is aimed at studying a variety of vision-based behaviors. The robot does not bring its own brain within the body. It leaves the brain in the mother environment and talks with it by radio links. The brain is raised in the mother environment inherited over generations. The key idea of the remote-brained approach is that of interfacing intelligent software systems with real robot bodies through wireless technology. In this framework the robot system can have a powerful vision system in the brain environment. The authors have applied this approach to formation of vision-based dynamic and intelligent behaviors of a multi-limbed mobile robot. In this paper the authors present an apelike robot with the remote-brained environment and describe vision-based experiments carried out with the apelike robot.

Focusing attention on flexibility and intelligent reactivity in the real world, it is more important to build, not a robot that won't fall down, but a robot that can get up if it does fall down. This paper presents research on a two-armed bipedal robot, an apelike robot, which can perform biped walking, rolling over and standing up. The robot consists of a head, two arms, and two legs. The control system of the biped robot is designed based on the remote-brained approach in which a robot does not bring its own brain within the body and talks with it by radio links. This remote-brained approach enables a robot to have both a heavy brain with powerful computation and a lightweight body with multiple joints. The robot can keep balance while standing using tracking vision, detect whether it falls down or not by a set of vertical sensors, and perform a getting up motion by coordinating two arms and two legs. The developed system and experimental results are described with illustrated real examples.

We present a variety of vision-based adaptive and interactive behaviors in mechanical animals. The mechanical animal is a multi-legged robot designed as a remote-brained robots which does not bring its own brain within the body. It leaves the brain in the mother environment and talks with it by radio links. The brain is raised in the mother environment inherited over generations. The key idea of the remote-brained approach is that of interfacing intelligent software systems with real robot bodies through wireless technology. In this framework the robot system can have a powerful vision system in the brain environment. We have applied this approach toward formation of vision-based dynamic and intelligent behaviors of mechanical animals such as doglike robots and apelike robots. In this paper we introduce the remote-brained approach and describe some remote-brained robots and visual processes for adaptive and interactive behaviors with them.>