In order to accurately reach a target position while executing a task which imposes occlusion or constraints of the posture, a humanoid robot requires an adaptive locomotion system, which can comprehensively integrate localization, environmental mapping, global locomotion planning and local error correction. In this paper, we propose a method of constructing a perception based locomotion system for a humanoid robot. The major contribution of this paper is solving a problem of the locomotion error caused by the task constraints, by locally compensating footsteps and assessing the need for global footstep re-planning online based on environmental measurements. The proposed system provides an accurate and dense ground point cloud, called HeightField, using plane estimation and space interpolation, and obstacle point cloud for frequent collision avoidance by accumulating laser scans. This environmental perception enables a humanoid robot to plan footsteps globally even in the situation where the sight of the robot is limited and compensate footsteps while estimating landing state during locomotion online with the localization result. We evaluated the practicality of the proposed system by applying it to our humanoid robot carrying a heavy object in a construction site and confirmed that the proposed system contributed to improved locomotion abilities of a humanoid robot engaging in heavy-duty or dangerous tasks.

This paper presents the development toward a dancing robot that can listen to and dance along with musical performances. One of the key components of this robot is the ability to modify its dance motions with varying tempos, without exceeding motor limitations, in the same way that human dancers modify their motions. In this paper, we first observe human performances with varying musical tempos of the same musical piece, and then analyze human modification strategies. The analysis is conducted in terms of three body components: lower, middle, and upper bodies. We assume that these body components have different purposes and different modification strategies, respectively, for the performance of a dance. For all of the motions of these three components, we have found that certain fixed postures, which we call keyposes, tend to be preserved. Thus, this paper presents a method to create motions for robots at a certain music tempo, from human motion at an original music tempo, by using these keyposes. We have implemented these algorithms as an automatic process and validated their effectiveness by using a physical humanoid robot HRP-2. This robot succeeded in performing the Aizu-bandaisan dance, one of the Japanese traditional folk dances, 1.2 and 1.5 times faster than the tempo originally learned, while maintaining its physical constraints. Although we are not achieving a dancing robot which autonomously interacts with varying music tempos, we think that our method has a vital role in the dancing-to-music capability.

In this paper, we describe three singing information processing systems, VocaListener, VocaListener2, and VocaWatcher, that imitate singing expressions of the voice and face of a human singer. VocaListener can synthesize natural singing voices by analyzing and imitating the pitch and dynamics of the human singing. VocaListener2 imitates temporal timbre changes in addition to the pitch and dynamics. In synchronization with the synthesized singing voices, VocaWatcher can generate realistic facial motions of a humanoid robot, the HRP-4C, by analyzing and imitating facial motions of a human singing that are recorded by a single video camera. These systems that focus on “imitation” are not only promising for representing human-like naturalness, but also useful for providing intuitive control means.

In this paper, we describe VocaWatcher, a novel robot motion generator that enables a humanoid robot to sing with realistic facial expressions and naturally synthesized singing voices. This robot singer is an important and attractive humanoid robot application for the entertainment scene; moreover, it promotes state-of-the-art integration of robot engineering, music processing, and image processing. To overcome the difficulties of generating natural facial expressions that are precisely synchronized with singing voices, VocaWatcher imitates a human singer by analyzing a video clip of a human singing, recorded by a single video camera. VocaWatcher can control mouth, eye, and neck motions by imitating the corresponding human movements, which are estimated without using any markers in the video. It can also synthesize singing voices by imitating the pitch and dynamics of the human singing in the same video.

Hardware improvement of cybernetic human HRP-4C for entertainment is presented in this paper. We coined the word ¿Cybernetic Human¿ to explain a humanoid robot with a realistic head and a realistic figure of a human being. HRP-4C stands for Humanoid Robotics Platform-4 (Cybernetic human). Its joints and dimensions conform to average values of young Japanese females and HRP-4C looks very human-like. We have made HRP-4C present in several events to search for a possibility of use in the entertainment industry. Based on feedback from our experience, we improved its hardware. The new hand, the new foot with active toe joint, and the new eye with camera are introduced.

This paper presents preliminary results and analysis on generating turning motion of a humanoid robot by slipping the feet on the ground. Humans unconsciously exploit the fact that our feet slip on the ground; such slip motion is necessary for humanoids so as to realize sophisticated human-like motions. In order to generate the slip motion, we need to predict the amount of slip. We propose the hypothesis that the turning motion is caused by the effect of minimizing the power generated by floor friction. A model of rotation by friction force is described on the basis of our hypothesis. The case that a robot applies the same force on both feet is discussed; then, we extend the discussion to the case of different force distribution. The hypothesis is verified through experiments with a humanoid robot HRP-2.

A novel framework of biped walking stabilization control is introduced. The target robot is a 42 DOF humanoid robot HRP-4C which has a body dimensions close to the average Japanese female. We develop a body posture controller and foot force controllers on the joint position servo of the robot. By applying this posture/force control, we can regard the robot system as a simple linear inverted pendulum with ZMP delay. After a preliminary experiment to confirm the linear dynamics, we design a tracking controller for walking stabilization. It is evaluated in the experiments of HRP-4C walking and turning on a lab floor. The robot can also perform an outdoor walk on an uneven pavement.

This paper proposes a novel user interface for creating whole body motions of biped humanoid robots just by giving key poses. Although such an interface is popular for CG character animation, there have not been any practical systems that can appropriately handle the kinematic and dynamic conditions required for moving actual biped robots stably without falling down. In our interface, every time a key pose is created, modified or removed, the system immediately processes automatic trajectory adjustment of the feet and the waist so that the key poses and the interpolated motion can always meet the conditions. This enables a user to edit variety of motions in an intuitive and flexible way without paying attention to the conditions. We implemented the interface and confirmed that characteristic whole body motions of a realistic humanoid robot HRP-4C were easily created with it and the actual HRP-4C successfully performed them.

In this paper, we generate the biped gait of a humanoid robot that looks like a human's one. To generate the human like motion, we first capture walking motion of a human. Then, we analyze the captured data and obtain several information such as the relationship between the step length and waist height etc. We consider applying these informations to the real humanoid robot. Also, when the human walks, the sway of the waist is smaller than most of the humanoid robot's one. By compensating the angular momentum of the robot and by modifying the ZMP trajectory, we show that sway of the robot's waist can be smaller. We show the effectiveness of the proposed method through simulation and experimental results.

This paper proposes the kinodynamic gait planning for humanoid robots where both kinematics and dynamics of the system are considered. We can simultaneously plan both the foot-place and the whole-body motion taking the dynamical balance of the robot into consideration. As a dynamic constraint, we consider the differential equation of the robotpsilas CoG. To solve this constraint, we assume two walking pattern generators; the offline and the online ones. We randomly sample the configuration space to search for the path connecting the start and the goal configurations. When sampling the configuration space, three milestones are sequentially connected to the parent milestone. To show the effectiveness of the proposed methods, we show simulation and experimental results where the humanoid robot HRP-2 walks on several environments.

This paper presents a biped humanoid robot that is able to walk on a rough terrain while touching a handrail. The contact wrench sum (CWS for short) is used as the criterion to judge if the contact between the robot and the environment is strongly stable under the sufficient friction assumption, where the contact points are not coplanar and the normal vectors at the points are not identical. It is confirmed that the proposed pattern generator can make the robot walk as desired in dynamics simulations and experiments, and the motions can be improved by a hand position control and using waist joints.

This paper presents a motion pattern generator of humanoid robots that walks on a flat plane, steps and a rough terrain. It is guaranteed rigorously that the desired contact between a humanoid robot and terrain should be maintained by keeping the contact wrench sum between them inside the contact wrench cone under the sufficient friction assumption. A walking pattern is generated by solving the contact wrench equations and by applying the resolved momentum control.

We discuss a ZMP-based running pattern generation for a biped robot equipped with toe springs. Our biped robot HRP-2LT has twelve active DoFs for its legs and two passive DoFs for its toes. The trajectory of the center of mass is designed to realize the specified running motion and the foot trajectories are determined to get proper spring action at lift off phases. They are interpreted into joint angles by using the resolved momentum control. By the simulation and the preliminary experiment, it is shown that the toe springs are effectively used for running and hopping.

This paper presents a motion planner of getting up motion using Mahalanobis distance. It is an indispensable function for humanoid robots to get up by itself and some humanoid robots are able to get up by themselves, but the motion can start only from the states specified a prior. The robots have to get up from an arbitrary lying state which may result after an unexpected falling. The proposed method (l) determines the degree of similarity between the current falling state and predefined falling states using Mahalanobis distance, (2) generates a collision-free motion to the most similar state, and (3) plans a sequence of motions using a state transition graph.

This paper proposes a method of a real-time gait planning for humanoid robots which can change stride immediately at every step. Based on an analytical solution of an inverted pendulum model, the trajectories of the COG (center of gravity) and the ZMP (zero-moment point) are parameterized by polynomials. Since their coefficients can be efficiently computed with given boundary conditions, this framework can provide a real-time walking pattern generator for humanoid robots. To handle the unexpected result caused by immediate changes of foot placement, we made single support periods as an additional trajectory parameter and the ZMP fluctuation was suppressed by mixing the opposite phase of the ZMP error. The effectiveness of our method is shown by experiments of the humanoid robot HRP-2.

This paper studies an optimal planning of falling motions of a human-sized humanoid robot to reduce the damage of the robot. We developed a human-sized robot HRP-2FX which has a simplified humanoid robot shape with seven d.o.f. and can emulate motions in the sagittal plane of a humanoid robot. An optimal control is applied to generate the falling motion of HRP-2FX to minimize a performance index, and the optimality has been verified by the experiments on HRP-2FX.

We propose a simulation system that achieves realistic and efficient simulations of humanoid robots. This paper focuses on a constraint-based contact force solver and virtual spring-damper joints from among the components of the system. The contact force solver can accurately simulate contacts between rigid bodies including articulated rigid bodies. LCP-like formulation of constraint conditions is solved by an iterative calculation method that extends the Gauss-Seidel method. This paper clarifies how to integrate existing methods to implement a robust and efficient solver. Virtual spring-damper joints are proposed to simulate a shock absorbing mechanism that many biped humanoid robots have in their feet to increase the stability of walking motion. The combination of the rigid contact model and the elastic virtual joints can improve the accuracy of the simulation. The simulation system was verified by experiments using humanoid robot HRP-2, and the results shows the validity of the system.

This paper proposes a method to model the modification of upper body motion of dance performance based on the speed of played music. When we observed structured dance motion performed at a normal music playback speed and motion performed at a faster music playback speed, we found that the detail of each motion is slightly different while the whole of the dance motion is similar in both cases. This phenomenon is derived from the fact that dancers omit the details and perform the essential part of the dance in order to follow the faster speed of the music. To clarify this phenomenon, we analyzed the motion differences in the frequency domain, and obtained two insights on the omission of motion details: (1) High frequency components are gradually attenuated depending on the musical speed, and (2) important stop motions are preserved even when high frequency components are attenuated. Based on these insights, we modeled our motion modification considering musical speed and joint limitations that a humanoid robot has. We show the effectiveness of our method via some applications for humanoid robot motion generation.

This paper presents a method to optimize and filter trajectories generated from recorded human motion for a humanoid robot with physical limits. The objective function is responsible for mimicking human trainers, enhancing the possibility for fast convergence, while constraints are used to transform motion within the limit of the capabilities of the humanoid robot. Those constraints for angle, velocity, and dynamic force are represented as B-spline coefficients. An iterative soft-constraint paradigm is proposed to enhance the quality of velocity and force constraints. Collision avoidance is also considered as a constraint. For precision refinement, in regions of high-frequency motion not adequately modeled by an initial splining, B-spline is extensible into a hierarchy so that optimization that meets global criteria can be performed locally. Furthermore, all of the constraints can be used solely as filtering. To use these filters, an effective method to directly decompose a trajectory to a B-spline is also presented

The goal of this study is developing a biped humanoid robot that can observe a human dance performance and imitate it. To achieve this goal, we propose a task model of lower body motion, which consists of task primitives (what to do) and skill parameters (how to do it). Based on this model, a sequence of task primitives and their skill parameters are detected from human motion, and robot motion is regenerated from the detected result under constraints of a robot. This model can generate human-like lower body motion including various waist motions as well as various stepping motions of the legs. Generated motions can be performed stably on an actual robot supported by its own legs. We used improved robot hardware HRP-2, which has superior features in body weight, actuators, and DOF of the waist. By using the proposed method and HRP-2, we have realized a dance performance of Japanese folk dance by the robot, which is synchronized with a performance of a human grand master on the same stage.

From recorded human motion, trajectory optimization of a humanoid robot with physical limits being the key constraint is presented. The optimization's objective function preserves the salient characteristics of the original motion, while constraints are used to transform the motion to the limit of the capabilities of the humanoid robot. It is shown that using constraints to limit generated trajectories to physically realizable motion ensures that limits are being met more precisely than by reducing them using a standard objective function. The use of wavelets vs. B-spline is compared, and it is shown that B-spline data representation has pronounced advantages. Furthermore, in regions of high-frequency motion not adequately modeled by an initial splining, B-spline is extensible into a hierarchy so that optimizations can be performed locally which still meet global criteria. It is shown how to use B-spline coefficients in angle-, velocity-, acceleration-, and dynamic force-constraints. To detect trajectory subsections with excessive error in need of control point adjustment and local re-optimization, not only is a traditional error detector used, but also a B-spline density detector is also presented. Generated motions were tested using our simulation program and the robot HRP-2.

The goal of the study described In this work is to develop a total technology for archiving human dance motions. A key feature of this technology is a dance replay by a humanoid robot. Although human dance motions can be acquired by a motion capture system, a robot cannot exactly follow the captured data because of different body structure and physical properties between the human and the robot. In particular, leg motions are too constrained to be converted from the captured data because the legs must interact with the floor and keep dynamic balance within the mechanical constraints of current robots. To solve this problem, we have designed a symbolic description of leg motion primitives in a dance performance. Human dance actions are recognized as a sequence of primitives and the same actions of the robot can be regenerated from them. This framework is more reasonable than modifying the original motion to adapt the robot constraints. We have developed a system to generate feasible robot motions from a human performance, and realized a dance performance by the robot HRP-1S.

This paper presents a method for matching the frames of the human motions acquired by a motion capture system, and then creating blended (interpolated) motions according to the matching result. This matching method is basically a variation of a dynamic programming (DP) matching but we enhanced it to enable it to detect the timescale parameters. This scaleable dynamic programming (scaleable-DP) can match and evaluate the same class of motions such as walking, running, stepping and their timescale parameters. This approach is adaptable for differences in individuals, such as body sizes and timing of the stop-frames. On the blending pipeline, we first generate the keyframes according to the matching result. The keyframes are generated by considering the spatial and temporal difference of individual motions. After that, transition motions are synthesized between the keyframes. We experimented with our approach by using 15 gait motions and 5 dance motions. The results of these demonstrations show the validity of the proposed algorithm.

The goal of this study is a system for a robot to imitate human dances. This paper describes the process to generate whole body motions which can be performed by an actual biped humanoid robot. Human dance motions are acquired through a motion capturing system. We then extract symbolic representation which is made up of primitive motions: essential postures in arm motions and step primitives in leg motions. A joint angle sequence of the robot is generated according to these primitive motions. Then joint angles are modified to satisfy mechanical constraints of the robot. For balance control, the waist trajectory is moved to acquire dynamics consistency based on desired ZMP. The generated motion is tested on OpenHRP dynamics simulator. In our test, the Japanese folk dance, 'Jongara-bushi', was successfully performed by HRP-1S.

The human body motion synthesis is highly necessary for humanoid robots' motion planning and computer animations. In this paper, new method for generating human-like natural motions based on the motion database acquired by motion capture systems is described. On the analysis step, the acquired motions are divided into some motion segments, and then the characteristic poses and motions are archived as 'motion styles'. The motion style is a kind of the human skill, and it's unique to the motions' scenario, such as the different kinds of dances. On the synthesis step, users direct the key poses of human figures. The system generates the characteristic motions according to the user's directions and motion style database. The experiment result shows that this method can synthesize the realistic 'stylized' motions with this framework.

This paper presents a method for importing human dance motion into humanoid robots through visual observation. The human motion data is acquired from a motion capture system consisting of 8 cameras and 8 PC clusters. Then the whole motion sequence is divided into motion elements and clustered into groups according to the correlation of end-effector trajectories. We call these segments 'motion primitives'. New dance motions are generated by concatenating these motion primitives. We are also trying to make a humanoid dance these original or generated motions using inverse-kinematics and dynamic balancing techniques.