We propose a novel walking control scheme based on the dynamics of the Linear Inverted Pendulum (LIP) model. The pattern generation incorporates a model of contact forces, enabling closed-loop control of the humanoid robot's state, including the Center of Mass (CoM) position, velocity, and Zero Moment Point (ZMP). No additional control policies are required to maintain static and dynamic balance. Our approach also includes dynamic re-planning of step locations and timings, thus preserving the LIP's boundedness condition. We validated this controller on five different humanoid robots, testing its robustness through various disturbances, including sudden pushes during walking and static phases. Additionally, our controller demonstrated effective locomotion over uneven and compliant terrain. Both simulation and experimental results confirm the effectiveness and robustness of this controller.

On the one hand, cameras of conventional field-of-view usually considered in computer vision and robotics are very often modeled as a pinhole plus possibly a distortion model. On the other hand, there is a large variety of models for panoramic cameras. Many camera models have been proposed for fisheye cameras, catadioptric cameras, and super fisheye cameras. But in both cases, few models offer the possibility of converting them into another model.This paper contributes to filling this gap in, to allow an algorithm designed with a projection model to accept data of a camera calibrated with another model. So, a pre-existing data set can be used without having to recalibrate the camera. We provide the methodology and mathematical developments for three conversions considering three different types of cameras that are evaluated with respect to calibration and within a visual Simultaneous Localization And Mapping benchmark. The source code of the camera model conversions studied in this paper is shared within the libPeR library for Perception in Robotics: https://github.com/PerceptionRobotique/libPeRbase.

This work presents a novel method to keep stable contact and balance while stepping over partial footholds for biped humanoid robots with flat feet. We exploit plantar tactile feedback to detect the geometry of the terrain and reconstruct online the new supporting polygon after landing every step. Plantar tactile feedback detects early contacts to stop the swing foot motion. Then we compute the convex hull of the cluster of contact points detected by distributed normal force sensors over the foot soles. The centroid of the supporting polygon is then used for retargeting the reference ZMP and DCM positions. Finally, the supporting polygon is used to define constraints for ZMP balance feedback control. These methods were implemented in two biped humanoid robots running different walking controllers.

To efficiently achieve complex humanoid loco-manipulation tasks in industrial contexts, we propose a combined vision-based tracker-localization interplay integrated as part of a task-space whole-body optimization control. To achieve good perception complementarity between manipulation and localization, a new fast dense 3D model-based tracking using wide-angle depth image is developed and used in conjunction with a simultaneous localization and mapping software. Our approach allows humanoid robots, targeted for industrial manufacturing, to manipulate and assemble large-scale objects while walking. It is assessed with experiments consisting in rolling and assembling in an unwinder a heavy and wide bobbin using bimanual grasping and bipedal locomotion at a time. This experimental use-case is found in some large-scale manufacturing where bobbins are enrolled with various materials (cables, papers, rubbers, etc.). The same experiments are made using two different humanoid robots of the same family.Note to Practitioners—This paper aims at deploying humanoid robots in large-scale manufacturing industries. We consider non-added value tasks related to transporting large tools or objects such as large bobbins by means of locomanipulation skills, similarly to human workers. We developed a task-space control framework that has been successfully applied in the aircraft industry. In the frame of a current collaboration with other major industrial sectors, we enhanced our control framework to interplay between SLAM and visual tracking to realize robust loco-manipulation tasks. Our approach can be applied and ported to any humanoid robot or bi-manual wheeled mobile robots with minor programming effort as the software is made open. Preliminary experiments with two different humanoids and use-cases suggest that our approach is feasible. In future research, we will address the problem of performance to reach at least human-speed in the execution of locomanipulation tasks in large-scale industry and automation contexts.

In this paper, we propose a trajectory planning framework for a robot that exploits a pre-computed database of end-effector trajectories as the guidance of optimization-based inverse kinematics. We constructed a reachable graph of a robot offline, which represents feasible end-effector paths with corresponding configurations. When performing the online trajectory planning, we applied A* search to the reachable graph to find a feasible path between input start and goal globally in the task space. Its cost function has the separated term dependent on the robot, which comes from the manipulability of configurations preserved in the reachable graph, and that is dependent on the environment. Then, we solve optimization-based inverse kinematics to generate an optimal joint trajectory while utilizing the end-effector trajectory and its corresponding configurations as the guidance to avoid local optimum. We evaluated our framework quantitatively by comparing it with existing methods to confirm that it achieved a high success rate and quality of results while suppressing its computational time. We also qualitatively proved its practicality by applying it to the material handling task in the real-world. This result shows that it improved the performance of the optimization-based inverse kinematics avoiding local optimum and applicability to the different environments of the pre-computed motion database.

Deep Convolutional Neural Networks (CNNs) have been successfully deployed on robots for 6-DoF object pose estimation through visual perception. However, obtaining labeled data on a scale required for the supervised training of CNNs is a difficult task - exacerbated if the object is novel and a 3D model is unavailable. To this end, this work presents an approach for rapidly generating real-world, pose-annotated RGB-D data for unknown objects. Our method not only circumvents the need for a prior 3D object model (textured or otherwise) but also bypasses complicated setups of fiducial markers, turntables, and sensors. With the help of a human user, we first source minimalistic labelings of an ordered set of arbitrarily chosen keypoints over a set of RGB-D videos. Then, by solving an optimization problem, we combine these labels under a world frame to recover a sparse, keypoint-based representation of the object. The sparse representation leads to the development of a dense model and the pose labels for each image frame in the set of scenes. We show that the sparse model can also be efficiently used for scaling to a large number of new scenes. We demonstrate the practicality of the generated labeled dataset by training a CNN based 6-DoF object pose estimator.

In this paper we report the safety-oriented framework for controlling the torque in the case of robots with high reduction gears and having no joint torque feedback. This kind of robots suffer from high joint friction and low backdrivability, requiring high gains and integral feedback, which can be dangerous. Our optimization-based framework includes feasibility and safety features borrowed from position control, and we introduce novel ones. We show how we limit the integral terms using a QP-based anti-windup which produces the optimal torque that maintains the best performances under safety limits. We show also a new controller for null-space compliance, providing strong guarantees of convergence in the task-space and ignoring the corresponding null-space where the robot can be moved freely. We validate these features with experiments on one 9 DoF arm of the robot HRP-5P performing a Cartesian task, and then a dual Cartesian / admittance task.

This article develops model-based grasp planning algorithms. It focuses on industrial end-effectors like grippers and suction cups, and plans grasp configurations considering computer aided design (CAD) models of target objects. The developed algorithms can stably find many high-quality grasps, with satisfying precision and little dependency on the quality of CAD models. The undergoing core technique is superimposed segmentation, which preprocesses a mesh model by peeling it into superimposed facets. The algorithms use the facets to locate contacts and synthesize grasp poses for popular industrial end-effectors. Several tunable parameters are prepared to adapt the algorithms to meet various requirements. The experimental section studies the influence of the tunable parameters and analyzes the cost, precision, and robustness of the proposed algorithms and their planned grasps, with both simulations and real-world systems. Besides, the proposed algorithms are applicable to mesh models reconstructed from point clouds obtained by depth sensors. Some experiments and analysis are also carried out to study and demonstrate the ability.

In this paper we benchmark different types of cameras and evaluate their performance in terms of reliable localization reliability and precision in Visual Simultaneous Localization and Mapping (vSLAM). Such benchmarking is merely found for visual odometry, but never for vSLAM. Existing studies usually compare several algorithms for a given camera. The evaluation methodology we propose is applied to the recent OpenVSLAM framework. The latter is versatile enough to natively deal with perspective, fisheye, 360 cameras in a monocular or stereoscopic setup, an in RGB or RGB-D modalities. Results in various sequences containing light variation and scenery modifications in the scene assess quantitatively the maximum localization rate for 360 vision. In the contrary, RGB-D vision shows the lowest localization rate, but highest precision when localization is possible. Stereo-fisheye trades-off with localization rates and precision between 360 vision and RGB-D vision. The dataset with ground truth will be made available in open access to allow evaluating other/future vSLAM algorithms with respect to these camera types.

The simulation of static friction, and especially the effect of stiction, is cumbersome to perform in discrete-time due to its discontinuity at zero velocity and its switching behavior. However, it is essential to achieve reliable simulations of friction to develop compliant torque control algorithms, as they are much disturbed by this phenomenon. This paper takes as a base an elastoplastic model approach for friction, which is free from chattering and drift. It proposes two closed-form solutions that can be used to reliably simulate the effect of stiction consistently with the physics-based Stribeck model. These solutions consider the nonlinearity and velocity dependency, which are main characteristics of lubricated joints. One is directly inspired by the Stribeck nonlinear terms, and the other is a simplified rational approximation. The reliability of this simulation method is shown in simulation, where the consistency and stability are assessed. We also demonstrate the accuracy of these methods by comparing them to experimental data obtained from a robot joint equipped with a high gear reduction harmonic drive.

This paper presents a manipulation planning algorithm for robots to reorient objects. It automatically finds a sequence of robot motion that manipulates and prepares an object for specific tasks. Examples of the preparatory manipulation planning problems include reorienting an electric drill to cut holes, reorienting workpieces for assembly, and reorienting cargo for packing, etc. The proposed algorithm could plan single- and dual-arm manipulation sequences to solve the problems. The mechanism under the planner is a regrasp graph, which encodes grasp configurations and object poses. The algorithms search the graph to find a sequence of robot motion to reorient objects. The planner is able to plan both single- and dual-arm manipulation. It could also automatically determine whether to use a single arm, dual arms, or their combinations to finish given tasks. The planner is examined by various humanoid robots like Nextage, HRP2Kai, HRP5P, etc., using both simulation and real-world experiments.

This paper focuses on a stabilization control for multi-contact motion which enables a humanoid robot to locomote by realizing dynamic contact transitions on non-flat environment. In the stabilization process of the multi-contact motion, the desired Zero-Moment Point (ZMP) is modified by the position of the Divergent Component of Motion (DCM) error with respect to the 3D Center of Mass (CoM) motion generated from the force distribution ratio. The contact wrench of each end-effector is determined by quadratic optimization considering the centroidal dynamics and contact friction constraints so as to satisfy the modified ZMP. Each end-effector is controlled by optimized force reference through a projection of null space by force distribution ratio. We propose a multi-contact stabilization framework which can be designed not only to generate 3D CoM motion but also the CoM position estimation and the optimal force distribution around the reference ZMP in a unified manner from a balance controller, by using the force distribution ratio. The effectiveness of proposed method is validated by a quadruped locomotion leaning against a vertical wall using the joint position controlled humanoid HRP-5P in a dynamic simulator.

The dynamics of a humanoid robot cannot be correctly described independently from the external forces acting on it. These forces have to be reconstructed to enable the robot to control them or to compensate for them. Force sensors are usually used to measure these forces, but because of their cost, they are often put only on the ankle/feet and possibly the wrists. This paper addresses the issue of the estimation of external forces and moments that apply at any part of a robot without direct force measurements and without torque measurements. The sensors used are the regular force sensors and the IMUs of the robot. The method relies on a model-based estimator able to make the fusion between these sensors and the whole body dynamics. The estimator reconstructs a single state vector containing the floating-base kinematics, a filtered measurement of contact force and an additional estimation external force that we evaluate in this paper. Validation is performed on HRP-2 in a multi-contact motion.

In order to accurately reach a target position while executing a task which imposes occlusion or constraints of the posture, a humanoid robot requires an adaptive locomotion system, which can comprehensively integrate localization, environmental mapping, global locomotion planning and local error correction. In this paper, we propose a method of constructing a perception based locomotion system for a humanoid robot. The major contribution of this paper is solving a problem of the locomotion error caused by the task constraints, by locally compensating footsteps and assessing the need for global footstep re-planning online based on environmental measurements. The proposed system provides an accurate and dense ground point cloud, called HeightField, using plane estimation and space interpolation, and obstacle point cloud for frequent collision avoidance by accumulating laser scans. This environmental perception enables a humanoid robot to plan footsteps globally even in the situation where the sight of the robot is limited and compensate footsteps while estimating landing state during locomotion online with the localization result. We evaluated the practicality of the proposed system by applying it to our humanoid robot carrying a heavy object in a construction site and confirmed that the proposed system contributed to improved locomotion abilities of a humanoid robot engaging in heavy-duty or dangerous tasks.

We propose a control framework for torque controlled humanoid robots that efficiently minimizes the tracking error in a Quadratic Programming (QP)formulated as multiobjective weighted tasks with constraints. It results in an optimal dynamically-feasible reference that can be tracked robustly, with exponential convergence, without joint torque feedback, in the presence of non modelled torque bias and low-frequency bounded disturbances. This is achieved by introducing integral gains in a Lyapunov-stable torque control, which exploit the passivity properties of the dynamical model of the robot and their effect on the dynamic constraints of the QP solver. The robustness of this framework is demonstrated in simulation by commanding our robot, the HRP-5P, to achieve simultaneously several objectives in the configuration and the Cartesian spaces, in the presence of non-modeled static and kinetic joint friction, as well as an uncertain torque scale.

This is an experience report on applying program analysis tools and program monitors to a real-world humanoid robot in simulated and real environments. Humanoid robots, and cyber-physical systems in general, present unique challenges to testing and validation: they have realtime constraints, their runs are typically irreproducible, their tasks are high-level and preclude formal specification, and their software tends to be large and complex. In practice, bugs do cause robots to fail, and methods for analyzing such software has been wanting. This paper presents a case study, in which we find that traditional software bugs like memory errors do cause failures of robots in practice. Dynamic error detectors can be successfully employed to identify such errors - to an extent defined by realtime constraints. Static analysis tools, in their current form, are comparatively of limited use.

For mobile robots, 3D acquisition is required to model the environment. Particularly for humanoid robots, a modeled environment is necessary to plan the walking control. This environment can include both static objects, such as a ground surface with obstacles, and dynamic objects, such as a person moving around the robot. This paper proposes a system for a robot to obtain a sufficiently accurate shape of the environment for walking on a ground surface with obstacles and a method to detect dynamic objects in the modeled environment, which is necessary for the robot to react to sudden changes in the scene. The 3D acquisition is achieved by a projector-camera system mounted on the robot head that uses a structured-light method to reconstruct the shapes of moving objects from a single frame. The acquired shapes are aligned and merged into a common coordinate system using the simultaneous localization and mapping method. Dynamic objects are detected as shapes that are inconsistent with the previous frames. Experiments were performed to evaluate the accuracy of the 3D acquisition and the robustness with regard to detecting dynamic objects when serving as the vision system of a humanoid robot.

Abstract We present a non‐rigid surface registration technique that can align surfaces with sizes and shapes that are different from each other, while avoiding mesh distortions during deformation. The registration is constrained locally as conformal as possible such that the angles of triangle meshes are preserved, yet local scales are allowed to change. Based on our conformal registration technique, we devise an automatic registration and interactive registration technique, which can reduce user interventions during template fitting. We demonstrate the versatility of our technique on a wide range of surfaces.

A controller design to suppress vertical vibration of a position controlled biped walking robot is presented. The system model of structural vibration is estimated from a measurement of frequency response. By using the model, an optimal feedback controller with full state observer is designed. With a consideration of leg support phase, the vibration controller is combined into our walking controller. The effectiveness of our controller is experimentally demonstrated.

This paper presents a basic study on feasibility of usage of humanoid robots as an evaluator of assistive devices, by taking advantage of its anthropomorphic shape. In this new application humanoid are expected to help evaluation through quantitative measures, which is difficult with human subjects, and also to reduce the burden coming from ethical concerns with costly tests by human subjects. Taking a passive supportive wear “Smart Suit Lite” designed to relieve the load at lower back as an example, we have conducted pilot experiments by using the humanoid robot HRP-4C. The motion to be performed by the humanoid is obtained through retargeting technique from measured human lifting motion. The supportive effect is first estimated by simulation taking into account the mechanism of the supportive device. The experimentation of humanoid hardware brought us encouraging results on the basic feasibility of this application, as we observed a clear decrease of the torque for lifting when wearing the device as expected by the simulation.

This paper presents a method for generating a turning motion in a humanoid robot by allowing the feet of the robot to slip on the ground. As humans, we exploit the fact that our feet can slip on the ground, and allowing humanoid robots to realize this same motion is a worthwhile study. In this paper, we propose the hypothesis that a turning motion is caused by the effect of minimizing the power generated by floor friction. A model of rotation from this friction is then described based on our hypothesis. The proposed model suggests that only the trajectory and shape of the robot's feet determine the amount of rotation from a slip, and that the friction coefficient between the floor and the soles of the robot, as well as the velocity of the feet, do not affect the resultant angle. Verification is conducted through an experiment with a humanoid robot known as HRP-2. Next, to obtain the foot motion necessary to realize the desired rotational angle, the inverse problem is solved by confining the trajectory of the center of the foot to an arc shape. This solution is verified through an experiment with another humanoid robot, HRP-4C.

We present the realization of quick turning motion of a humanoid robot on its toes via slipping between its feet and the floor. A rotation model is described on the basis of our hypothesis that turning via slip occurs as a result of minimizing the power caused by floor friction. Using the model, the trajectory of the center of the foot can be generated to realize the desired rotational angle. Toe joints are used to realize quicker turning motion, while avoiding excessive motor load due to frictional torque. Quick slip-turn motion with toe support is successfully demonstrated using a humanoid robot HRP-4C.

This paper presents a reaching motion planning and execution framework tailored for exploration missions by human-operated humanoid robots in hazardous environments such as nuclear plants. This framework offers low-level but practical autonomy that allows the robot to plan and execute simple tasks, such as reaching a target object, within a reasonable amount of time. The human operator benefits from the efficiency of the framework to maneuver the robot without waiting for the planning results for minutes. The efficiency improvement is achieved in the following two phases. In the first phase, a reaching motion is planned quickly through approximation of mass distribution and kinematic structure to apply analytical solutions of inverse kinematics. Supposing that the robot is working in environments not completely known, the proposed planner can use measured voxel maps. In the second phase, the planned path is executed while compensating the approximation error in real time without violating other constraints. We confirm through simulations that a reaching motion for the HRP-2 humanoid with 30 DOFs in a constrained environment with pipes is planned in around one second. The simulation results also validate the efficiency of execution with real-time error compensation.

We present a reactive method for online robot motion replanning in dynamically changing environments by combining path replanning and deformation. Path deformation is newly integrated in our replanning method featured by efficient roadmap reuse and parallel planning and execution. This enhancement allows the planner to deal with more dynamic environments including continuously moving obstacles, by smoothly deforming the path during execution. Simulation results are shown to validate the effectiveness of the proposed method.

This paper presents the development of humanoid robotics platform - 4 (or HRP-4 for short). The high-density implementation used for HRP-4C, the cybernetic human developed by AIST, is also applied to HRP-4. HRP-4 has a total of 34 degrees of freedom, including 7 degrees of freedom for each arm to facilitate object handling and has a slim, lightweight body with a height of 151 [cm] and weight 39 [kg]. The software platform OpenRTM-aist and a Linux kernel with the RT-Preempt patch are used in the HRP-4 software system. Design concepts and mechanisms are presented with its basic specification in this paper.

Hardware improvement of cybernetic human HRP-4C for entertainment is presented in this paper. We coined the word ¿Cybernetic Human¿ to explain a humanoid robot with a realistic head and a realistic figure of a human being. HRP-4C stands for Humanoid Robotics Platform-4 (Cybernetic human). Its joints and dimensions conform to average values of young Japanese females and HRP-4C looks very human-like. We have made HRP-4C present in several events to search for a possibility of use in the entertainment industry. Based on feedback from our experience, we improved its hardware. The new hand, the new foot with active toe joint, and the new eye with camera are introduced.

This paper presents the development of humanoid robotics platform - 4 (or HRP-4 for short). The high-density implementation used for HRP-4C, the cybernetic human developed by AIST, is also applied to HRP-4. HRP-4 has a total of 34 degrees of freedom, including 7 degrees of freedom for each arm to facilitate object handling and has a slim, lightweight body with a height of 151 [cm] and weight 39 [kg]. The software platform OpenRTM-aist and a Linux kernel with the RT-Preempt patch are used in the HRP-4 software system. Design concepts and mechanisms are presented with its basic specification in this paper.

This paper presents preliminary results and analysis on generating turning motion of a humanoid robot by slipping the feet on the ground. Humans unconsciously exploit the fact that our feet slip on the ground; such slip motion is necessary for humanoids so as to realize sophisticated human-like motions. In order to generate the slip motion, we need to predict the amount of slip. We propose the hypothesis that the turning motion is caused by the effect of minimizing the power generated by floor friction. A model of rotation by friction force is described on the basis of our hypothesis. The case that a robot applies the same force on both feet is discussed; then, we extend the discussion to the case of different force distribution. The hypothesis is verified through experiments with a humanoid robot HRP-2.

A novel framework of biped walking stabilization control is introduced. The target robot is a 42 DOF humanoid robot HRP-4C which has a body dimensions close to the average Japanese female. We develop a body posture controller and foot force controllers on the joint position servo of the robot. By applying this posture/force control, we can regard the robot system as a simple linear inverted pendulum with ZMP delay. After a preliminary experiment to confirm the linear dynamics, we design a tracking controller for walking stabilization. It is evaluated in the experiments of HRP-4C walking and turning on a lab floor. The robot can also perform an outdoor walk on an uneven pavement.

This paper proposes a reactive leg motion generation method which integrates geometric constraints into its generation process. In order to react given instructions instantaneously or to keep balance against external disturbances, feasible steps must be generated automatically in real-time for safety. In many cases this feasibility has been realized by using predefined steps or admissible stepping regions. However, these predefinitions are often too conservative or valid only in limited situations. The proposed method considers geometric constraints in addition to joint limits during its generation process and it can utilize the ability of the robot to a maximum extent. It can generate feasible walking pattern in real-time by modifying the swing leg motion and the next landing position at each control cycle. The proposed method is validated by experiments using a humanoid robot HRP-2.

This paper proposes a new framework to recover balance against external forces by combining disturbance suppression and reactive stepping. In the view point of the feedback control, a reactive step can help to diminish the disturbance caused by an external force that should be compensated to maintain balance. In other words, if the adequate step is performed, the feedback controller does not have to compensate all of the external force by itself. Under this concept, we propose an original solution to distribute the compensation between a feedback controller and a reactive step, according to the period of support phase and a disturbance characteristic. We first clearly distinguish between the role of the disturbance suppression and the reactive stepping. Then, based on this distinction, the small disturbance of external force or happening late during the single-support phase, is mainly suppressed by state feedback. The large disturbance which is out of capability by feedback controller and at the beginning of the single-support phase, is absorbed by modifying reactively the next steps. The proposed method is validated through experimental results with the HRP-2 humanoid robot.

This paper proposes a unified optimization framework to solve the time-parameterization problem of humanoid-robot paths. Even though the time-parameterization problem is well known in robotics, the application to humanoid robots has not been addressed. This is because of the complexity of the kinematical structure as well as the dynamical motion equation. The main contribution of this paper is to show that the time parameterization of a statically stablepathto be transformed into a dynamically stabletrajectorywithin the humanoid-robot capacities can be expressed as an optimization problem. Furthermore, we propose an efficient method to solve the obtained optimization problem. The proposed method has been successfully validated on the humanoid robot HRP-2 by conducting several experiments. These results have revealed the effectiveness and the robustness of the proposed method.

We present a novel method for prioritizing both linear equality and inequality systems and provide one algorithm for its resolution. This algorithm can be summarized as a sequence of optimal resolutions for each linear system following their priority order. We propose an optimality criterion that is adapted to linear inequality systems and characterize the resulting optimal sets at every priority level. We have successfully applied our method to plan local motions for the humanoid robot HPR-2. We will demonstrate the validity of the method using an original scenario where linear inequality constraints are solved at lower priority than equality constraints.

In this paper, we generate the biped gait of a humanoid robot that looks like a human's one. To generate the human like motion, we first capture walking motion of a human. Then, we analyze the captured data and obtain several information such as the relationship between the step length and waist height etc. We consider applying these informations to the real humanoid robot. Also, when the human walks, the sway of the waist is smaller than most of the humanoid robot's one. By compensating the angular momentum of the robot and by modifying the ZMP trajectory, we show that sway of the robot's waist can be smaller. We show the effectiveness of the proposed method through simulation and experimental results.

This paper discusses the grasp planning of a multifingered hand attached at the tip of a robotic arm. By using the convex models and the new approximation method of the friction cone, our proposed algorithm can calculate the grasping motion within the reasonable time. For each grasping style used in this research, we define the grasping rectangular convex (GRC). We also define the object convex polygon (OCP) for the grasped object. By considering the geometrical relashionship among these convex models, we determine several parameters needed to define the final grasping configuration. To determine the contact point position satisfying the force closure, we use two approximation models of the friction cone. To save the calculation time, the rough approximation by using the ellipsoid is mainly used to check the force closure. Additionally, approximation by using the convex polyhedral cone is used at the final stage of the planning. The effectiveness of the proposed method is confirmed by some numerical examples.

In this paper, the imitation of human captured motions by a humanoid robot is considered. The main objective is to reproduce an imitated motion which should be as close as possible to the original human captured motion. To achieve this goal, the imitation problem is formulated as an optimization problem and the physical limits of the humanoid robot are considered as constraints. The optimization problem is then solved recursively by using an efficient dynamics algorithm, which allows the calculation of the gradient function with respect to the control parameters analytically. The simulation results using OpenHRP platform, which is a dynamical simulator for humanoid robot motions, have pointed out that the imitated motions preserve the salient characteristics of the original human captured motion. Moreover the optimization procedure converges well thanks to the analytical calculation of the gradient function.

This paper proposes an whole body motion planning method for humanoid robots in which dynamics is integrated. The method consists of two stages. A collision-free and statically stable path is planned in the first stage and it is transformed into a dynamically stable trajectory in the second stage. Contributions of the method is summarized as follows. (1) A local method plans a C1path while avoiding collisions between non-strictly convex objects. (2) The second stage gives the minimum time trajectory by time parameterization under dynamic balance constraints. (3) Any path reshaping for recovering collision-freeness is not required since the second stage doesnpsilat change shape of the path. Effectiveness of the method is examined by applying it to scenarios of a humanoid robot HRP-2.

In this paper, the development of humanoid robot HRP-3 is presented. HRP-3, which stands for Humanoid Robotics Platform-3, is a human-size humanoid robot developed as the succeeding model of HRP-2. One of features of HRP-3 is that its main mechanical and structural components are designed to prevent the penetration of dust or spray. Another is that its wrist and hand are newly designed to improve manipulation. Software for a humanoid robot in a real environment is also improved. We also include information on mechanical features of HRP-3 and together with the newly developed hand. Also included are the technologies implemented in HRP-3 prototype. Electrical features and some experimental results using HRP-3 are also presented.

This paper presents a motion pattern generator of humanoid robots that walks on a flat plane, steps and a rough terrain. It is guaranteed rigorously that the desired contact between a humanoid robot and terrain should be maintained by keeping the contact wrench sum between them inside the contact wrench cone under the sufficient friction assumption. A walking pattern is generated by solving the contact wrench equations and by applying the resolved momentum control.

This paper presents a motion planner of getting up motion using Mahalanobis distance. It is an indispensable function for humanoid robots to get up by itself and some humanoid robots are able to get up by themselves, but the motion can start only from the states specified a prior. The robots have to get up from an arbitrary lying state which may result after an unexpected falling. The proposed method (l) determines the degree of similarity between the current falling state and predefined falling states using Mahalanobis distance, (2) generates a collision-free motion to the most similar state, and (3) plans a sequence of motions using a state transition graph.

This paper presents a development of multi-fingered hand, which is modularized and can be attached to life-size humanoid robots. The developed hand has four fingers with 17 joints, which consist of 13 active joints and 4 linked joints. A miniaturized 6-axes force sensor is newly developed and is mounted on each fingertip for improving the manipulability. A main node controller with I/O, motor drivers, and amplifiers for 6-axes force sensors are also newly developed. These components are equipped in the hand for modularization. The developed hand is designed so as to realize about 8 [N] forces on the pad point of stretched finger, supposing transmission efficiency of drive system is 55 [%]. In this paper, the mechanisms of hand module, its specifications, and electrical system are also introduced.

This paper proposes a method of a real-time gait planning for humanoid robots which can change stride immediately at every step. Based on an analytical solution of an inverted pendulum model, the trajectories of the COG (center of gravity) and the ZMP (zero-moment point) are parameterized by polynomials. Since their coefficients can be efficiently computed with given boundary conditions, this framework can provide a real-time walking pattern generator for humanoid robots. To handle the unexpected result caused by immediate changes of foot placement, we made single support periods as an additional trajectory parameter and the ZMP fluctuation was suppressed by mixing the opposite phase of the ZMP error. The effectiveness of our method is shown by experiments of the humanoid robot HRP-2.

This paper studies an optimal planning of falling motions of a human-sized humanoid robot to reduce the damage of the robot. We developed a human-sized robot HRP-2FX which has a simplified humanoid robot shape with seven d.o.f. and can emulate motions in the sagittal plane of a humanoid robot. An optimal control is applied to generate the falling motion of HRP-2FX to minimize a performance index, and the optimality has been verified by the experiments on HRP-2FX.

We propose a simulation system that achieves realistic and efficient simulations of humanoid robots. This paper focuses on a constraint-based contact force solver and virtual spring-damper joints from among the components of the system. The contact force solver can accurately simulate contacts between rigid bodies including articulated rigid bodies. LCP-like formulation of constraint conditions is solved by an iterative calculation method that extends the Gauss-Seidel method. This paper clarifies how to integrate existing methods to implement a robust and efficient solver. Virtual spring-damper joints are proposed to simulate a shock absorbing mechanism that many biped humanoid robots have in their feet to increase the stability of walking motion. The combination of the rigid contact model and the elastic virtual joints can improve the accuracy of the simulation. The simulation system was verified by experiments using humanoid robot HRP-2, and the results shows the validity of the system.

This paper presents an object active visual search behavior in a 3D environment performed by a HRP-2 humanoid robot. The search is formalized as an optimization problem in which the goal is to maximize the target detection probability while minimizing the energy/distance and time to achieve the task. Natural constraints on the camera parameter space based on the characteristics of the recognition system are used to reduce the dimension of the problem and to speed up the optimization process to achieve a real time behavior. We present simulation and real experimental results using an HRP-2 robot.

This paper proposes a universal stability criterion of the foot contact of legged robots. The proposed method checks if the sum of the gravity and the inertia wrench applied to the COG of the robot, which is proposed to be the stability criterion, is inside the polyhedral convex cone of the contact wrench between the feet of a robot and its environment. The criterion can be used to determine the strong stability of the foot contact when a robot walks on an arbitrary terrain and/or when the hands of the robot are in contact with it under the sufficient friction assumption. The determination is equivalent to check if the ZMP is inside the support polygon of the feet when the robot walks on a horizontal plane with sufficient friction. The criterion can also be used to determine if the foot contact is sufficiently weakly stable when the friction follows a physical law. Therefore, the proposed criterion can be used to judge what the ZMP can, and it can be used in more universal cases

A biped walking pattern generator which allows an additional ZMP control (auxiliary ZMP) is presented. An auxiliary ZMP is realized by an inverse system added to a pattern generator based on the ZMP preview control. To compensate the effect of the auxiliary ZMP, we apply virtual time shifting of the reference ZMP. As an application of the proposed method, a walking control on uneven terrain is simulated. The simulated robot can walk successfully by changing its walking speed as the side effect of the auxiliary ZMP control

In this paper we realize a real-time communication on Ethernet and develop an onbody distributed control system for a humanoid robot, HRP-3P. Real-time communication on Ethernet is realized by (1) a communication method using the data link layer directly and (2) timing control using a real-time operating system ARTLinux. This enables us to reduce the cost of embedded systems and improve developmental efficiency. A CORBA implementation which works on this communication layer is also developed to increase compatibility with existing software. Finally a small-size distributed robot controller is developed for the onbody network of HRP-3P and a distributed I/O system is developed on top of this

This paper presents a motion suspension system to suspend humanoid motion in case of emergency. Once humanoids start their motions in human daily environments, there is a possibility that humanoids will meet with several emergencies such as hurting humans and injuring themselves. Even so, humanoids should be controlled so that they avert such emergencies in real-time. To realize this demand, we propose a method of real-time judgment of emergency prediction by humanoids. We also propose a simple and effective method of real-time pattern generation to force humanoids to stop immediately by one step without falling. To verify the validity of the proposed method, we finally present experimental results using a humanoid robot HRP-2, which include experiments at 2.8 [km/h] walks

A motion planner of emergency stop must make an operating humanoid robot to a stationary state under the emergency signal. It plays an important role in prevention of falling over because humanoid robots fall over easily. Immediately after the emergency signal, it must generate the emergency stop motion in real-time. We modeled a humanoid robot as a simple dynamic system consists of ZMP (zero-moment point) and COG (center of gravity) as its states. The emergency stop motion is generated by a state feedback. We determined optimal feedback gains in terms of the initial conditions and the pole assignment. The proposed method realized a reliable emergency stop with low computational cost. Furthermore, it can easily predict the possibility of the successful emergency stop at any time. The validity of the proposed method is confirmed by an experiment using humanoid robot HRP-2

This paper studies the balance of a humanoid robot carrying a heavy object. Without knowing the mass and the position of the center of gravity of the object, the humanoid robot carries a heavy object stably by using the force sensor information attached at the wrists and the ankles. We first show how to generate the motion of a humamoid robot by taking the force sensor information into consideration. We also show the method for generating the gait pattern by taking the dynamics of the carried object. The effectiveness of the proposed method is shown by experiments.

This paper proposes a method for a humanoid robot to generate 3D model of the environment using a stereo vision, find a movable space using it and plan feasible locomotion online. The model is generated by an accumulation of 3D grid maps which are made from the range data of the field of view obtained by a correlation based stereo vision. The locomotion is planned by an online whole body pattern generator which can modify robot’s waist height, an upper body posture and so on according to the size of the movable space.

This paper describes a generation method for spatially natural biped walking. By limiting the COG (Center of Gravity) motion space to a sculptured surface, the degree of freedom of the COG matches to the number of the ZMP (Zero Moment Point) equations. The COG motion can be uniquely generated along a specified surface satisfying the ZMP constraint with low calculation cost. Spatial and time parts are separable by representing motion surface of the COG as parametric variables. The motion surface defines the relative height of the COG from the landing foot position. Thus, the proposed method reflects geometric information directly to the motion planning without considering walk stability. In this paper, we show two actual examples, walking pattern including mostly stretched knee and going up stairs. The validity of the proposed method is confirmed by simulation. Walking with mostly stretched knee is also shown in experiment using HRP-2.

This paper presents a slip observer towards stabilizing biped walks on a low friction floor. Although biped humanoid robots are expected to easily adapt to environments designed for human, in fact they tend to tip over easily on real environments. For a practical use, it is one of important issues to stabilize a biped walking on an unexpected slippery floor with a low friction. In this paper, we propose the slip observer detecting skids that would occur at walking on unexpected slippery floor. We also propose a basic study of slip stabilizer towards reducing posture rolling caused by skids. Finally, we present experimental results using a humanoid robot HRP-2 to verify the validity of the proposed control scheme.

This paper presents an emergency stop algorithm of a walking humanoid robot. There are many cases which force a walking robot to stop quickly without falling. Since an emergency occurs at unpredictable timing and at any state of robot, the stopping motion must be generated in real-time. To overcome these problems, our emergency stop motion is divided into four phases according to the role of the zero-moment point (ZMP). In each phase, approximate analytical solutions of the center of gravity (COG) dynamics is used to generate the motion. During the single support phase, a landing time and position are determined by evaluating the average velocity of the swing leg and the horizontal position of the COG. During the double support phase, the travel distance of the COG and the ZMP are evaluated. The validity of the proposed method is confirmed by simulation and experiment using a humanoid robot HRP-2.

The goal of this study is developing a biped humanoid robot that can observe a human dance performance and imitate it. To achieve this goal, we propose a task model of lower body motion, which consists of task primitives (what to do) and skill parameters (how to do it). Based on this model, a sequence of task primitives and their skill parameters are detected from human motion, and robot motion is regenerated from the detected result under constraints of a robot. This model can generate human-like lower body motion including various waist motions as well as various stepping motions of the legs. Generated motions can be performed stably on an actual robot supported by its own legs. We used improved robot hardware HRP-2, which has superior features in body weight, actuators, and DOF of the waist. By using the proposed method and HRP-2, we have realized a dance performance of Japanese folk dance by the robot, which is synchronized with a performance of a human grand master on the same stage.

A humanoid robot is a real-time system controlled by a complex computer system that requires huge computing power for perception and planning, high energy efficiency for self-contained control, reduction of physical dimensions, and high reliability. This paper proposes a distributed architecture for the humanoid robot control substituting conventional centralized control architectures. In addition to the parallelism that provides scalable computing power at low clock namely at low energy, the distributed architecture contributes to reliable operations by replacing many fragile analog signal wires with a digital network with redundant routes. In order to accomplish a real-time control over the network, RMTP (responsive multi-threaded processor) for parallel and real-time computation has been newly designed. RMTP can synchronize more than thirty nodes distributed over a robot body in less than 5 micro second with a real time network called the responsive link (RL). Architectures of RMTP, RL and Linux-based real-time system software are presented.

We previously reported the first human-sized humanoid robot that can fall down safely and stand up again. This paper examines the falling motion control by supplementary simulations and presents an improvement of the control by optimizing the control parameters. The UKEMI falling over control of a human-sized humanoid robot is simulated, the simulation results are compared with the experimental results of real humanoid robot.

This work proposes a new style of manipulation by a humanoid robot. Focusing on the task of pushing an object, the foot placement of it is planed in real-time according to the result of manipulation of an object. By controlling the arms using the impedance control, a humanoid robot can push an object stably regardless of the mass of an object. If an object is heavy, a humanoid robot pushes an object with walking slowly, and vice versa. Also, for planning the gait in real-time, we newly propose an analytical method where the newly calculated trajectory of the robot motion is smoothly connected to the current one. The effectiveness of the proposed method is confirmed by simulation and experiment.

This work studies locomotion planning of humanoid robots to pass through narrow spaces. Humanoid robots can alter the style of the locomotion while wheeled robots can not. The proposed method generates a 3D local map from visual information and plans the appropriate locomotion based on the map from biped walking with the variable height and the width and from crawling.

A development of humanoid robot HRP-2 is presented in this paper. HRP-2 is a humanoid robotics platform, which we developed in phase two of HRP. HRP was a humanoid robotics project, which had run by the Ministry of Economy, Trade and Industry (METI) of Japan from 1998FY to 2002FY for five years. The ability of the biped locomotion of HRP-2 is improved so that HRP-2 can cope with uneven surface, can walk at two third level of human speed, and can walk on a narrow path. The ability of whole body motion of HRP-2 is also improved so that HRP-2 can get up by a humanoid robot's own self if HRP-2 tips over safely. In this paper, the appearance design, the mechanisms, the electrical systems, specifications, and features upgraded from its prototype are also introduced.

This paper studies a falling motion control of a human-size humanoid robot when it falls forward. Safe knee landing of the robot is realized as a first step to minimize the damage of the falling over. The landing follows the detection of the falling, the decrease of the knee height after the detection, and the brake of landing speed. It is confirmed that the proposed method can soften the landing impact significantly by the proposed method.

For cooperative work of robots and humans in the real world, a communicative function based on speech is indispensable for robots. To realize such a function in a noisy real environment, it is essential that robots be able to extract target speech spoken by humans from a mixture of sounds by their own resources. We have developed a method of detecting and extracting speech events based on the fusion of audio and video information. In this method, audio information (sound localization using a microphone array) and video information (human tracking using a camera) are fused by a Bayesian network to enable the detection of speech events. The information of detected speech events is then utilized in sound separation using adaptive beam forming. In this paper, some basic investigations for applying the above system to the humanoid robot HRP-2 are reported. Input devices, namely a microphone array and a camera, were mounted on the head of HRP-2, and acoustic characteristics for sound localization/separation performance were investigated. Also, the human tracking system was improved so that it can be used in a dynamic situation. Finally, overall performance of the system was tested via off-line experiments.

This paper shows some preliminary results on the dynamical balance of a humanoid robot grasping an environment. By grasping an environment, it becomes easier for the robot to keep balance. By using the linear programming, a necessary condition for keeping balance of the robot is formulated taking the grasping force into consideration. We show that the occasion exists where the stronger the hand of a humanoid robot grasps the handrail, the larger the region of ZMP for keeping balance becomes. We further show an experimental result of a humanoid robot climbing up a big gap increasing the stability by grasping a handrail.

Biped walking on a low friction floor is analyzed in this paper. For a given walking pattern, we can calculate a necessary friction coefficient which allows the robot to perform the expected motion. To reduce the maximum necessary friction coefficient, a pattern generation based on preview control theory is explained. We also describe a calculation of slip concerned ZMP, which provides a good prediction of falling caused by slips. Finally, we test a walk of 135 [km/h] on low friction environment using a humanoid robot HRP-2. The robot could successfully walk over the slippery area whose friction coefficient is 0.14.

We have developed a software platform, called OpenHRP, for humanoid robotics which consists of a dynamic simulator and motion control library for humanoid robots. This paper attempts to answer a frequently asked question "do the dynamic simulations and the experiments of biped walking of humanoid robots correspond?". Using OpenHRP and humanoid robots HRP-1S and HRP-2P, the comparisons between the simulations and experiments are shown at various aspects.

We introduce a new method of a biped walking pattern generation by using a preview control of the zero-moment point (ZMP). First, the dynamics of a biped robot is modeled as a running cart on a table which gives a convenient representation to treat ZMP. After reviewing conventional methods of ZMP based pattern generation, we formalize the problem as the design of a ZMP tracking servo controller. It is shown that we can realize such controller by adopting the preview control theory that uses the future reference. It is also shown that a preview controller can be used to compensate the ZMP error caused by the difference between a simple model and the precise multibody model. The effectiveness of the proposed method is demonstrated by a simulation of walking on spiral stairs.

This paper presents a humanoid robot that has the same size as a human and that can lie down to the floor and get up from the floor with the robot face upward and downward. We believe that the robot is the first life-size humanoid robot with the capability. The motions are realized by the combination of novel hardware and software. The features of the hardware are a human-like proportion and joints with wide movable ranges including two waist joints. The software segments the motion into the sequence of the contact states between the robot and the floor and assigns an appropriate controller to each transition between the consecutive states. The experimental results are presented.

This paper proposes a new method for the teleoperation of humanoid robots which integrates human operator's intention with robot's autonomy. Getting hints from human conscious and subconscious motion generations, we propose a method which generates whole body motions of a humanoid robot satisfying operator's desired movement of specific points on which the operator focuses as well as the robot's balance. The proposed method is based on a whole body momentum control. We also introduce a function of automatic adjustments of the position of the center of mass and torso orientation in order to expand the reachable area of a humanoid robot. The effectiveness of the method is confirmed by using humanoid robot simulator OpenHRP with physical parameters of real humanoid robot HRP-1S.

We have developed a humanoid robot HRP-2P with a biped locomotion controller, stereo vision software and aural human interface to realize cooperative works by a human and a humanoid robot. The robot can find a target object by the vision, and carry it cooperatively with a human by biped locomotion according to the voice commands by the human. A cooperative control is applied to the arms of the robot while it carries the object, and the walking direction of the robot is controlled by the interactive force and torque through the force/torque sensor on the wrists. The experimental results are presented in the paper.

This paper investigates a method through which human-size humanoid robot can fall over backwards safely. Squatting-extending motion of legs reduce impact of falling and shock-absorbing parts of the robot keep the force at a permissible range. The robot could stand up itself again after falling.

We introduce a method to generate whole body motion of a humanoid robot such that the resulted total linear/angular momenta become specified values. First, we derive a linear equation, which gives to total momentum of a robot from its physical parameters, the base link speed and the joint speeds. Constraints between the legs and the environment are also considered. The whole body motion is calculated from a given momentum reference by using a pseudo-inverse of the inertia matrix. As examples, we generated the kicking and walking motions and tested on the actual humanoid robot HRP-2. This method, the resolved momentum control, gives us a unified framework to generate various maneuvers of humanoid robots.

For real-time walking control of a biped robot, we analyze the dynamics of a three-dimensional inverted pendulum whose motions are constrained onto an arbitrarily defined plane. This analysis leads us a simple linear dynamics, the Three-Dimensional Linear Inverted Pendulum Mode (3D-LIPM). Geometric nature of trajectories under the 3D-LIPM is discussed, and an algorithm for walking pattern generation is presented. Experimental results of real-time walking control of a 12-DOF biped robot HRP-2L using an input device such as a game pad are also shown.

This paper introduces an open architecture humanoid robotics platform (OpenHRP) on which various building blocks of humanoid robotics can be investigated. OpenHRP is a virtual humanoid robot platform with a compatible humanoid robot, and consists of a simulator of humanoid robots and motion control library for them which can also be applied to a compatible humanoid robot as it is. OpenHRP is expected to initiate the exploration of humanoid robotics on an open architecture software and hardware, due to the unification of the controllers and the examined consistency between the simulator and a real humanoid robot.

This paper presents an advanced leg module developed for HRP-2. HRP-2 is a new humanoid robotics platform,. which we have been developing in phase two of HRP. HRP is a humanoid robotics project, which has been launched by Ministry of Economy, Trade and Industry (METI) of Japan from 1998FY to 2002FY for five years. The ability of the biped locomotion of HRP-2 is improved so that HRP-2 can cope with rough terrain in the open air and can prevent the possible damages to a humanoid robot's own self in the event of tipping over. In this paper the mechanisms and specifications of leg module, electrical system, simulation results utilized for deciding specifications, and experimental results are also introduced.

This paper investigates a method to minimize damage to a humanoid robot when it falls over to the ground. The strategy involves controlling the attitude of the robot while it is falling over so that it lands on the ground at one of shock-absorbing parts of the robot. A simulation study has confirmed that the proposed algorithm can make the robot land at specified shock-absorbing parts.

This paper presents an advanced leg module developed for HRP-2, a new humanoid robotics platform, which has been developed in phase two of the Humanoid Robotics Project (HRP), a five year program sponsored by the Ministry of Economy, Trade and Industry of Japan (METI) from 1998FY to 2002FY. The biped locomotion ability of HRP-1, the humanoid robotics platform developed in phase I is to be improved so that HRP-2 can cope with rough terrain and can prevent possible damage to the robot body caused by tipping over. In this paper, the mechanisms and specifications of the leg module, the electrical system, the simulation results utilized for deciding specifications, and some experimental results are presented.

This paper presents a prototype humanoid robotics platform developed for HRP-2. HRP-2 is a new humanoid robotics platform, which we have been developing in phase two of HRP HRP is a humanoid robotics project, which has been launched by Ministry of Economy, Trade and Industry (METI) of Japan from 1998FY to 2002FY for five years. The ability of the biped locomotion of HRP-2 is improved so that HRP-2 can cope with rough terrain in the open air and can prevent the possible damages to a humanoid robot's own self in the event of tipping over. The ability of whole body motion of HRP-2 is also improved so that HRP-2 can get up by a humanoid robot's own self even tough HRP-2 tips over. In this paper, the mechanisms and specifications of developed prototype humanoid robotics platform, and its electrical system are introduced.

A teleoperation system for whole body motions of a humanoid robot using a simple joystick master device is developed. Humanoid robots are physically similar to humans and usually possess a large number of degrees of freedom. By using hints from the shifting of the focus of attention for body motions between the joints of the human body during task executions, we propose a switching command based teleoperation system in which the operator selects only the necessary point of the humanoid robot's body for manipulation. In this paper, we present the implementation of this switching command based teleoperation system and experimental results using this system to teleoperate the humanoid robot HRP-1S developed in the Humanoid Robotics Project of the Ministry of Economy, Trade and Industry of Japan.

This paper describes a rapid development system for humanoid vision-based behavior consisting of both real and virtual robots in a simulation environment. Previous robotics simulators were limited to verify dynamic motions such as walking, or to plan or learn in a simple environment. However, our system is able to simulate vision based behavior, i.e. motion and perception behavior, of a robot as a whole, since it can simulate both dynamics and collisions in the environment. We regard the real-time aspect of the simulation as important so that so that users can develop vision based behavior rapidly and efficiently. We employ a Common Interface API to share behavior software between real and virtual robots. Moreover, visual processing functions such as color extraction and depth map generation are available. As a result, vision based behavior consisting of local map generation, planning and navigation of the humanoid in simulation is presented. We also show that software developed in the simulation environment is applicable to real robots.

In this paper, the developmental software environment for humanoids is introduced. This environment has the following features: 1) a hardware concealment function which saves the trouble with hardware improvements; 2) an incremental expansion function which supports a long-term development by many developers; and 3) a centralised management of body information which enables the robots to share software among different bodies. These features are realized by the plug-in architecture, the universal robot model and its compilers. Using plug-ins enables one to accumulate functions developed by different developers and to switch computer interfaces of the robot body. The universal robot model and compilers promote the use of external software and share software modules among different bodies.

For 3D walking control of a biped robot we analyze the dynamics of a 3D inverted pendulum in which motion is constrained to move along an arbitrarily defined plane. This analysis yields a simple linear dynamics, the 3D linear inverted pendulum mode (3D-LIPM). Geometric nature of trajectories under the 3D-LIPM and a method for walking pattern generation are discussed. A simulation result of a walking control using a 12-DOF biped robot model is also shown.

This paper presents a virtual humanoid robot platform (V-HRP for short) on which we can develop the identical controller for a virtual humanoid robot and its real counterpart. The unification of the controllers for the virtual and real robot has been realized by introducing software adapters for two robots respectively and employing ART-Linux on which real-time processing is available at the user level. Thanks to the unification, the controllers can share softwares with the dynamics simulator of V-HRP, including the parameter parser, kinematics and dynamics computations and the collision detector. This feature can make the development of the controllers more efficient and the developed controllers more reliable.

A humanoid robot "H6" is developed as a platform for the research on perception-action coupling in intelligent behaviour of humanoid type robots. The H6 has the features as follows: 1) a body which has enough DOFs and each joint has enough torque for full body motion, 2) a PC/AT compatible on-board computer which is controlled by RT-linux so that low-level to high-level control is achieved simultaneously, 3) is self-contained and connected to a network via radio ethernet, 4) a 3D vision function can be applied. The H6 is expected to be a common test-bed for experiment and discussion for various aspects of intelligent humanoid robotics.

It is not simple for humanoid to acquire its actions automatically because: 1) it has very wide search space due to many DOF and is not fixed on the ground; 2) complex actions are difficult to describe using indirect evaluation functions; and 3) a humanoid is required to behave naturally like a human does. Kinematics adaptation generates an action outline which is a rough action description from teacher's action (motion capture data) by absorbing a difference of structures between the human and robot. Then a target action is acquired through dynamics adaptation by searching an action which is executable dynamically. Advantages of this framework are that: 1) the target action is described in an intuitive representation, 2) the search space is reduced in the neighborhood space, and 3) the acquired action will be a natural motion.

In this paper, a developmental construction methodology for a humanoid robot system is proposed. "Developmental" means that a versatile robot system is inherently a target to be developed incrementally, repetitively with inheriting results of former generations. Based on this methodology, the developmental system consists of (1) the hardware independent robot model for a skill development, (2) an action space representation for a functional development and (3) a robot body which has an onbody network for a physical development. On this system, five humanoid robots have been developed and locomotion behaviors have been realized.

In this paper, a second generation remote-brained humanoid robot is presented which is developed for research on whole body action. Humanoid robots are important as a platform for the integration of techniques and algorithms acquired from research on manipulators, legged robots and so on. And they have new problems such as how to acquire, memorize, select and carry out various motions which use their whole bodies efficiently. In order to do research on these problems, a good robot body, which has whole body and enough performance for walking and getting up when it falls down is necessary. At the same time, a powerful brain is necessary which can be evolved through the body. As a solution to these demands, remote-brained approach was proposed and several humanoid robots was developed. Using these robots, several researches were done. For example, a brain framework called BeNet, an action acquisition using GA and NN and so on. They were done using a simple wireless connection. Interface between robot brain and its body to concentrate on a high-level problem. However this interface limited actions the robot can do simultaneously. In this paper, this old interface is taken to the next step. New interface has multiple actuator control methods which are switched on demand, and an onbody microprocessor network which controls actuators, measures sensors and interacts with a brain. Finally a new humanoid robot is developed on this interface.

In this paper, we describe the design implementation of this real-time layer for remote-brained robot. "Remote-brained approach" is a paradigm for software research by separating the robot brain from the robot body. The interface from the body to the brain top-level software is also denoted. Furthermore, an application of the reactive motion of humanoid-type robot is described. The real-time facility is fundamentally important for a robot that behaves in the read-world. To accomplish both large scale intelligent software and real-time functionality, we design the system by dividing the brain into a low-level real-time layer and a high-level action decision layer. The action decision layer is put on a WS, and the real-time layer is put on a transputer network.

We present a vision-based real-time control system of a human-form robot in swing motion. The robot has 16 DOF joints and a camera which is designed to do research on human-form robot behaviors. The key idea to develop the vision based system for the robots are the remote-brained approach and real-time visual tracking. In this approach, the robot system is designed to have the brain and the body separate and to connect them by a radio link. The body sends the visual image to the brain and the body receives the motion, references. The basic algorithm to observe the swing motion by vision uses the optical flow generation. The control algorithm for flow-based visual feedback is described.

We have focused on actions of a humanlike robot which has the whole body and realized actions such as "walk", "roll over" and "stand up". In this research, we design and implement the whole body humanlike robot that can perform a "carry objects" action. Keys to realizing a carrying action are (1) the robot system which has a lightweight whole body robot and a powerful brain which can process a multi joint model of the body, (2) the model environment which can generate walking patterns on the horizontal ground, stairs and a slope by moving its centroid and (3) gravity compensation based on modeling of the actuator.

Presents a new type of robot which has two arms and two legs like an ape and is aimed at studying a variety of vision-based behaviors. The robot does not bring its own brain within the body. It leaves the brain in the mother environment and talks with it by radio links. The brain is raised in the mother environment inherited over generations. The key idea of the remote-brained approach is that of interfacing intelligent software systems with real robot bodies through wireless technology. In this framework the robot system can have a powerful vision system in the brain environment. The authors have applied this approach to formation of vision-based dynamic and intelligent behaviors of a multi-limbed mobile robot. In this paper the authors present an apelike robot with the remote-brained environment and describe vision-based experiments carried out with the apelike robot.

Focusing attention on flexibility and intelligent reactivity in the real world, it is more important to build, not a robot that won't fall down, but a robot that can get up if it does fall down. This paper presents research on a two-armed bipedal robot, an apelike robot, which can perform biped walking, rolling over and standing up. The robot consists of a head, two arms, and two legs. The control system of the biped robot is designed based on the remote-brained approach in which a robot does not bring its own brain within the body and talks with it by radio links. This remote-brained approach enables a robot to have both a heavy brain with powerful computation and a lightweight body with multiple joints. The robot can keep balance while standing using tracking vision, detect whether it falls down or not by a set of vertical sensors, and perform a getting up motion by coordinating two arms and two legs. The developed system and experimental results are described with illustrated real examples.

We present a variety of vision-based adaptive and interactive behaviors in mechanical animals. The mechanical animal is a multi-legged robot designed as a remote-brained robots which does not bring its own brain within the body. It leaves the brain in the mother environment and talks with it by radio links. The brain is raised in the mother environment inherited over generations. The key idea of the remote-brained approach is that of interfacing intelligent software systems with real robot bodies through wireless technology. In this framework the robot system can have a powerful vision system in the brain environment. We have applied this approach toward formation of vision-based dynamic and intelligent behaviors of mechanical animals such as doglike robots and apelike robots. In this paper we introduce the remote-brained approach and describe some remote-brained robots and visual processes for adaptive and interactive behaviors with them.>