Direct Visual Servoing (DVS) is a technique that controls the robot motion by using the pixel intensities captured by a camera. DVS demonstrates high accuracy at convergence, prompting the development of various methods aimed at expanding its convergence domain.In this paper, we propose a mathematical characterization of the DVS convergence domain with closed-form expressions for the controlled degrees of freedom. From these expressions, we concluded that the extent of the convergence domain is related to the presence of isotropic or defocus blur, a phenomenon that had only been observed previously as a trend in empirical experiments.

In this paper, we propose a trajectory planning framework for a robot that exploits a pre-computed database of end-effector trajectories as the guidance of optimization-based inverse kinematics. We constructed a reachable graph of a robot offline, which represents feasible end-effector paths with corresponding configurations. When performing the online trajectory planning, we applied A* search to the reachable graph to find a feasible path between input start and goal globally in the task space. Its cost function has the separated term dependent on the robot, which comes from the manipulability of configurations preserved in the reachable graph, and that is dependent on the environment. Then, we solve optimization-based inverse kinematics to generate an optimal joint trajectory while utilizing the end-effector trajectory and its corresponding configurations as the guidance to avoid local optimum. We evaluated our framework quantitatively by comparing it with existing methods to confirm that it achieved a high success rate and quality of results while suppressing its computational time. We also qualitatively proved its practicality by applying it to the material handling task in the real-world. This result shows that it improved the performance of the optimization-based inverse kinematics avoiding local optimum and applicability to the different environments of the pre-computed motion database.

In this paper we report the safety-oriented framework for controlling the torque in the case of robots with high reduction gears and having no joint torque feedback. This kind of robots suffer from high joint friction and low backdrivability, requiring high gains and integral feedback, which can be dangerous. Our optimization-based framework includes feasibility and safety features borrowed from position control, and we introduce novel ones. We show how we limit the integral terms using a QP-based anti-windup which produces the optimal torque that maintains the best performances under safety limits. We show also a new controller for null-space compliance, providing strong guarantees of convergence in the task-space and ignoring the corresponding null-space where the robot can be moved freely. We validate these features with experiments on one 9 DoF arm of the robot HRP-5P performing a Cartesian task, and then a dual Cartesian / admittance task.

The simulation of static friction, and especially the effect of stiction, is cumbersome to perform in discrete-time due to its discontinuity at zero velocity and its switching behavior. However, it is essential to achieve reliable simulations of friction to develop compliant torque control algorithms, as they are much disturbed by this phenomenon. This paper takes as a base an elastoplastic model approach for friction, which is free from chattering and drift. It proposes two closed-form solutions that can be used to reliably simulate the effect of stiction consistently with the physics-based Stribeck model. These solutions consider the nonlinearity and velocity dependency, which are main characteristics of lubricated joints. One is directly inspired by the Stribeck nonlinear terms, and the other is a simplified rational approximation. The reliability of this simulation method is shown in simulation, where the consistency and stability are assessed. We also demonstrate the accuracy of these methods by comparing them to experimental data obtained from a robot joint equipped with a high gear reduction harmonic drive.

This paper focuses on a stabilization control for multi-contact motion which enables a humanoid robot to locomote by realizing dynamic contact transitions on non-flat environment. In the stabilization process of the multi-contact motion, the desired Zero-Moment Point (ZMP) is modified by the position of the Divergent Component of Motion (DCM) error with respect to the 3D Center of Mass (CoM) motion generated from the force distribution ratio. The contact wrench of each end-effector is determined by quadratic optimization considering the centroidal dynamics and contact friction constraints so as to satisfy the modified ZMP. Each end-effector is controlled by optimized force reference through a projection of null space by force distribution ratio. We propose a multi-contact stabilization framework which can be designed not only to generate 3D CoM motion but also the CoM position estimation and the optimal force distribution around the reference ZMP in a unified manner from a balance controller, by using the force distribution ratio. The effectiveness of proposed method is validated by a quadruped locomotion leaning against a vertical wall using the joint position controlled humanoid HRP-5P in a dynamic simulator.

The dynamics of a humanoid robot cannot be correctly described independently from the external forces acting on it. These forces have to be reconstructed to enable the robot to control them or to compensate for them. Force sensors are usually used to measure these forces, but because of their cost, they are often put only on the ankle/feet and possibly the wrists. This paper addresses the issue of the estimation of external forces and moments that apply at any part of a robot without direct force measurements and without torque measurements. The sensors used are the regular force sensors and the IMUs of the robot. The method relies on a model-based estimator able to make the fusion between these sensors and the whole body dynamics. The estimator reconstructs a single state vector containing the floating-base kinematics, a filtered measurement of contact force and an additional estimation external force that we evaluate in this paper. Validation is performed on HRP-2 in a multi-contact motion.

In order to accurately reach a target position while executing a task which imposes occlusion or constraints of the posture, a humanoid robot requires an adaptive locomotion system, which can comprehensively integrate localization, environmental mapping, global locomotion planning and local error correction. In this paper, we propose a method of constructing a perception based locomotion system for a humanoid robot. The major contribution of this paper is solving a problem of the locomotion error caused by the task constraints, by locally compensating footsteps and assessing the need for global footstep re-planning online based on environmental measurements. The proposed system provides an accurate and dense ground point cloud, called HeightField, using plane estimation and space interpolation, and obstacle point cloud for frequent collision avoidance by accumulating laser scans. This environmental perception enables a humanoid robot to plan footsteps globally even in the situation where the sight of the robot is limited and compensate footsteps while estimating landing state during locomotion online with the localization result. We evaluated the practicality of the proposed system by applying it to our humanoid robot carrying a heavy object in a construction site and confirmed that the proposed system contributed to improved locomotion abilities of a humanoid robot engaging in heavy-duty or dangerous tasks.

We propose a control framework for torque controlled humanoid robots that efficiently minimizes the tracking error in a Quadratic Programming (QP)formulated as multiobjective weighted tasks with constraints. It results in an optimal dynamically-feasible reference that can be tracked robustly, with exponential convergence, without joint torque feedback, in the presence of non modelled torque bias and low-frequency bounded disturbances. This is achieved by introducing integral gains in a Lyapunov-stable torque control, which exploit the passivity properties of the dynamical model of the robot and their effect on the dynamic constraints of the QP solver. The robustness of this framework is demonstrated in simulation by commanding our robot, the HRP-5P, to achieve simultaneously several objectives in the configuration and the Cartesian spaces, in the presence of non-modeled static and kinetic joint friction, as well as an uncertain torque scale.

This is an experience report on applying program analysis tools and program monitors to a real-world humanoid robot in simulated and real environments. Humanoid robots, and cyber-physical systems in general, present unique challenges to testing and validation: they have realtime constraints, their runs are typically irreproducible, their tasks are high-level and preclude formal specification, and their software tends to be large and complex. In practice, bugs do cause robots to fail, and methods for analyzing such software has been wanting. This paper presents a case study, in which we find that traditional software bugs like memory errors do cause failures of robots in practice. Dynamic error detectors can be successfully employed to identify such errors - to an extent defined by realtime constraints. Static analysis tools, in their current form, are comparatively of limited use.

This paper presents the method to retarget human motion. The method can evaluate the ability of the preservation of the original characteristics of human motion data. It enables to compute the joint trajectories of the human corresponding with the retargeted ones of the robot at the same time, by utilizing the geometric identification technique. The obtained trajectories of a human are the solution to minimize the cost function about motion reproduction. The proposed method is efficient for such applications that the robot needs to mimic human motion without modifying the detailed features of the original movement of each body segment. The results of the retargeted motions to a humanoid robot are shown.

For mobile robots, 3D acquisition is required to model the environment. Particularly for humanoid robots, a modeled environment is necessary to plan the walking control. This environment can include both static objects, such as a ground surface with obstacles, and dynamic objects, such as a person moving around the robot. This paper proposes a system for a robot to obtain a sufficiently accurate shape of the environment for walking on a ground surface with obstacles and a method to detect dynamic objects in the modeled environment, which is necessary for the robot to react to sudden changes in the scene. The 3D acquisition is achieved by a projector-camera system mounted on the robot head that uses a structured-light method to reconstruct the shapes of moving objects from a single frame. The acquired shapes are aligned and merged into a common coordinate system using the simultaneous localization and mapping method. Dynamic objects are detected as shapes that are inconsistent with the previous frames. Experiments were performed to evaluate the accuracy of the 3D acquisition and the robustness with regard to detecting dynamic objects when serving as the vision system of a humanoid robot.

A controller design to suppress vertical vibration of a position controlled biped walking robot is presented. The system model of structural vibration is estimated from a measurement of frequency response. By using the model, an optimal feedback controller with full state observer is designed. With a consideration of leg support phase, the vibration controller is combined into our walking controller. The effectiveness of our controller is experimentally demonstrated.

This paper presents the development of humanoid robotics platform - 4 (or HRP-4 for short). The high-density implementation used for HRP-4C, the cybernetic human developed by AIST, is also applied to HRP-4. HRP-4 has a total of 34 degrees of freedom, including 7 degrees of freedom for each arm to facilitate object handling and has a slim, lightweight body with a height of 151 [cm] and weight 39 [kg]. The software platform OpenRTM-aist and a Linux kernel with the RT-Preempt patch are used in the HRP-4 software system. Design concepts and mechanisms are presented with its basic specification in this paper.

Hardware improvement of cybernetic human HRP-4C for entertainment is presented in this paper. We coined the word ¿Cybernetic Human¿ to explain a humanoid robot with a realistic head and a realistic figure of a human being. HRP-4C stands for Humanoid Robotics Platform-4 (Cybernetic human). Its joints and dimensions conform to average values of young Japanese females and HRP-4C looks very human-like. We have made HRP-4C present in several events to search for a possibility of use in the entertainment industry. Based on feedback from our experience, we improved its hardware. The new hand, the new foot with active toe joint, and the new eye with camera are introduced.

This paper presents the development of humanoid robotics platform - 4 (or HRP-4 for short). The high-density implementation used for HRP-4C, the cybernetic human developed by AIST, is also applied to HRP-4. HRP-4 has a total of 34 degrees of freedom, including 7 degrees of freedom for each arm to facilitate object handling and has a slim, lightweight body with a height of 151 [cm] and weight 39 [kg]. The software platform OpenRTM-aist and a Linux kernel with the RT-Preempt patch are used in the HRP-4 software system. Design concepts and mechanisms are presented with its basic specification in this paper.

This paper presents preliminary results and analysis on generating turning motion of a humanoid robot by slipping the feet on the ground. Humans unconsciously exploit the fact that our feet slip on the ground; such slip motion is necessary for humanoids so as to realize sophisticated human-like motions. In order to generate the slip motion, we need to predict the amount of slip. We propose the hypothesis that the turning motion is caused by the effect of minimizing the power generated by floor friction. A model of rotation by friction force is described on the basis of our hypothesis. The case that a robot applies the same force on both feet is discussed; then, we extend the discussion to the case of different force distribution. The hypothesis is verified through experiments with a humanoid robot HRP-2.

A novel framework of biped walking stabilization control is introduced. The target robot is a 42 DOF humanoid robot HRP-4C which has a body dimensions close to the average Japanese female. We develop a body posture controller and foot force controllers on the joint position servo of the robot. By applying this posture/force control, we can regard the robot system as a simple linear inverted pendulum with ZMP delay. After a preliminary experiment to confirm the linear dynamics, we design a tracking controller for walking stabilization. It is evaluated in the experiments of HRP-4C walking and turning on a lab floor. The robot can also perform an outdoor walk on an uneven pavement.

This paper proposes a reactive leg motion generation method which integrates geometric constraints into its generation process. In order to react given instructions instantaneously or to keep balance against external disturbances, feasible steps must be generated automatically in real-time for safety. In many cases this feasibility has been realized by using predefined steps or admissible stepping regions. However, these predefinitions are often too conservative or valid only in limited situations. The proposed method considers geometric constraints in addition to joint limits during its generation process and it can utilize the ability of the robot to a maximum extent. It can generate feasible walking pattern in real-time by modifying the swing leg motion and the next landing position at each control cycle. The proposed method is validated by experiments using a humanoid robot HRP-2.

This paper proposes a new framework to recover balance against external forces by combining disturbance suppression and reactive stepping. In the view point of the feedback control, a reactive step can help to diminish the disturbance caused by an external force that should be compensated to maintain balance. In other words, if the adequate step is performed, the feedback controller does not have to compensate all of the external force by itself. Under this concept, we propose an original solution to distribute the compensation between a feedback controller and a reactive step, according to the period of support phase and a disturbance characteristic. We first clearly distinguish between the role of the disturbance suppression and the reactive stepping. Then, based on this distinction, the small disturbance of external force or happening late during the single-support phase, is mainly suppressed by state feedback. The large disturbance which is out of capability by feedback controller and at the beginning of the single-support phase, is absorbed by modifying reactively the next steps. The proposed method is validated through experimental results with the HRP-2 humanoid robot.

In this paper, we generate the biped gait of a humanoid robot that looks like a human's one. To generate the human like motion, we first capture walking motion of a human. Then, we analyze the captured data and obtain several information such as the relationship between the step length and waist height etc. We consider applying these informations to the real humanoid robot. Also, when the human walks, the sway of the waist is smaller than most of the humanoid robot's one. By compensating the angular momentum of the robot and by modifying the ZMP trajectory, we show that sway of the robot's waist can be smaller. We show the effectiveness of the proposed method through simulation and experimental results.

This paper proposes the kinodynamic gait planning for humanoid robots where both kinematics and dynamics of the system are considered. We can simultaneously plan both the foot-place and the whole-body motion taking the dynamical balance of the robot into consideration. As a dynamic constraint, we consider the differential equation of the robotpsilas CoG. To solve this constraint, we assume two walking pattern generators; the offline and the online ones. We randomly sample the configuration space to search for the path connecting the start and the goal configurations. When sampling the configuration space, three milestones are sequentially connected to the parent milestone. To show the effectiveness of the proposed methods, we show simulation and experimental results where the humanoid robot HRP-2 walks on several environments.

This paper presents a biped humanoid robot that is able to walk on a rough terrain while touching a handrail. The contact wrench sum (CWS for short) is used as the criterion to judge if the contact between the robot and the environment is strongly stable under the sufficient friction assumption, where the contact points are not coplanar and the normal vectors at the points are not identical. It is confirmed that the proposed pattern generator can make the robot walk as desired in dynamics simulations and experiments, and the motions can be improved by a hand position control and using waist joints.

This paper presents a motion pattern generator of humanoid robots that walks on a flat plane, steps and a rough terrain. It is guaranteed rigorously that the desired contact between a humanoid robot and terrain should be maintained by keeping the contact wrench sum between them inside the contact wrench cone under the sufficient friction assumption. A walking pattern is generated by solving the contact wrench equations and by applying the resolved momentum control.

We discuss a ZMP-based running pattern generation for a biped robot equipped with toe springs. Our biped robot HRP-2LT has twelve active DoFs for its legs and two passive DoFs for its toes. The trajectory of the center of mass is designed to realize the specified running motion and the foot trajectories are determined to get proper spring action at lift off phases. They are interpreted into joint angles by using the resolved momentum control. By the simulation and the preliminary experiment, it is shown that the toe springs are effectively used for running and hopping.

This paper presents a motion planner of getting up motion using Mahalanobis distance. It is an indispensable function for humanoid robots to get up by itself and some humanoid robots are able to get up by themselves, but the motion can start only from the states specified a prior. The robots have to get up from an arbitrary lying state which may result after an unexpected falling. The proposed method (l) determines the degree of similarity between the current falling state and predefined falling states using Mahalanobis distance, (2) generates a collision-free motion to the most similar state, and (3) plans a sequence of motions using a state transition graph.

This paper proposes a method of a real-time gait planning for humanoid robots which can change stride immediately at every step. Based on an analytical solution of an inverted pendulum model, the trajectories of the COG (center of gravity) and the ZMP (zero-moment point) are parameterized by polynomials. Since their coefficients can be efficiently computed with given boundary conditions, this framework can provide a real-time walking pattern generator for humanoid robots. To handle the unexpected result caused by immediate changes of foot placement, we made single support periods as an additional trajectory parameter and the ZMP fluctuation was suppressed by mixing the opposite phase of the ZMP error. The effectiveness of our method is shown by experiments of the humanoid robot HRP-2.

This paper studies an optimal planning of falling motions of a human-sized humanoid robot to reduce the damage of the robot. We developed a human-sized robot HRP-2FX which has a simplified humanoid robot shape with seven d.o.f. and can emulate motions in the sagittal plane of a humanoid robot. An optimal control is applied to generate the falling motion of HRP-2FX to minimize a performance index, and the optimality has been verified by the experiments on HRP-2FX.

In this paper, we plan the collision free motion for walking pattern generation of a humanoid robot. Our motion planner can take into account several features of the walking pattern generator. We first run the walking pattern generator by considering the contact wrench applied to the robot and monitor the collision among the links and the environments. Then, we plan the collision free motion for the period of time causing the collision. In our motion planner, we can consider the constraint condition which are the functions of time. Also, for keeping balance of the robot, we plan the motion with keeping the horizontal position of the COG as well as the position/orientation of the feet/hand. The effectiveness of the proposed method is confirmed by simulation and experiment.

This paper proposes a universal stability criterion of the foot contact of legged robots. The proposed method checks if the sum of the gravity and the inertia wrench applied to the COG of the robot, which is proposed to be the stability criterion, is inside the polyhedral convex cone of the contact wrench between the feet of a robot and its environment. The criterion can be used to determine the strong stability of the foot contact when a robot walks on an arbitrary terrain and/or when the hands of the robot are in contact with it under the sufficient friction assumption. The determination is equivalent to check if the ZMP is inside the support polygon of the feet when the robot walks on a horizontal plane with sufficient friction. The criterion can also be used to determine if the foot contact is sufficiently weakly stable when the friction follows a physical law. Therefore, the proposed criterion can be used to judge what the ZMP can, and it can be used in more universal cases

A biped walking pattern generator which allows an additional ZMP control (auxiliary ZMP) is presented. An auxiliary ZMP is realized by an inverse system added to a pattern generator based on the ZMP preview control. To compensate the effect of the auxiliary ZMP, we apply virtual time shifting of the reference ZMP. As an application of the proposed method, a walking control on uneven terrain is simulated. The simulated robot can walk successfully by changing its walking speed as the side effect of the auxiliary ZMP control

This paper presents a motion suspension system to suspend humanoid motion in case of emergency. Once humanoids start their motions in human daily environments, there is a possibility that humanoids will meet with several emergencies such as hurting humans and injuring themselves. Even so, humanoids should be controlled so that they avert such emergencies in real-time. To realize this demand, we propose a method of real-time judgment of emergency prediction by humanoids. We also propose a simple and effective method of real-time pattern generation to force humanoids to stop immediately by one step without falling. To verify the validity of the proposed method, we finally present experimental results using a humanoid robot HRP-2, which include experiments at 2.8 [km/h] walks

A motion planner of emergency stop must make an operating humanoid robot to a stationary state under the emergency signal. It plays an important role in prevention of falling over because humanoid robots fall over easily. Immediately after the emergency signal, it must generate the emergency stop motion in real-time. We modeled a humanoid robot as a simple dynamic system consists of ZMP (zero-moment point) and COG (center of gravity) as its states. The emergency stop motion is generated by a state feedback. We determined optimal feedback gains in terms of the initial conditions and the pole assignment. The proposed method realized a reliable emergency stop with low computational cost. Furthermore, it can easily predict the possibility of the successful emergency stop at any time. The validity of the proposed method is confirmed by an experiment using humanoid robot HRP-2

This paper studies the balance of a humanoid robot carrying a heavy object. Without knowing the mass and the position of the center of gravity of the object, the humanoid robot carries a heavy object stably by using the force sensor information attached at the wrists and the ankles. We first show how to generate the motion of a humamoid robot by taking the force sensor information into consideration. We also show the method for generating the gait pattern by taking the dynamics of the carried object. The effectiveness of the proposed method is shown by experiments.

This paper proposes a method for a humanoid robot to generate 3D model of the environment using a stereo vision, find a movable space using it and plan feasible locomotion online. The model is generated by an accumulation of 3D grid maps which are made from the range data of the field of view obtained by a correlation based stereo vision. The locomotion is planned by an online whole body pattern generator which can modify robot’s waist height, an upper body posture and so on according to the size of the movable space.

This paper describes a generation method for spatially natural biped walking. By limiting the COG (Center of Gravity) motion space to a sculptured surface, the degree of freedom of the COG matches to the number of the ZMP (Zero Moment Point) equations. The COG motion can be uniquely generated along a specified surface satisfying the ZMP constraint with low calculation cost. Spatial and time parts are separable by representing motion surface of the COG as parametric variables. The motion surface defines the relative height of the COG from the landing foot position. Thus, the proposed method reflects geometric information directly to the motion planning without considering walk stability. In this paper, we show two actual examples, walking pattern including mostly stretched knee and going up stairs. The validity of the proposed method is confirmed by simulation. Walking with mostly stretched knee is also shown in experiment using HRP-2.

This paper presents a slip observer towards stabilizing biped walks on a low friction floor. Although biped humanoid robots are expected to easily adapt to environments designed for human, in fact they tend to tip over easily on real environments. For a practical use, it is one of important issues to stabilize a biped walking on an unexpected slippery floor with a low friction. In this paper, we propose the slip observer detecting skids that would occur at walking on unexpected slippery floor. We also propose a basic study of slip stabilizer towards reducing posture rolling caused by skids. Finally, we present experimental results using a humanoid robot HRP-2 to verify the validity of the proposed control scheme.

This paper presents an emergency stop algorithm of a walking humanoid robot. There are many cases which force a walking robot to stop quickly without falling. Since an emergency occurs at unpredictable timing and at any state of robot, the stopping motion must be generated in real-time. To overcome these problems, our emergency stop motion is divided into four phases according to the role of the zero-moment point (ZMP). In each phase, approximate analytical solutions of the center of gravity (COG) dynamics is used to generate the motion. During the single support phase, a landing time and position are determined by evaluating the average velocity of the swing leg and the horizontal position of the COG. During the double support phase, the travel distance of the COG and the ZMP are evaluated. The validity of the proposed method is confirmed by simulation and experiment using a humanoid robot HRP-2.

The goal of this study is developing a biped humanoid robot that can observe a human dance performance and imitate it. To achieve this goal, we propose a task model of lower body motion, which consists of task primitives (what to do) and skill parameters (how to do it). Based on this model, a sequence of task primitives and their skill parameters are detected from human motion, and robot motion is regenerated from the detected result under constraints of a robot. This model can generate human-like lower body motion including various waist motions as well as various stepping motions of the legs. Generated motions can be performed stably on an actual robot supported by its own legs. We used improved robot hardware HRP-2, which has superior features in body weight, actuators, and DOF of the waist. By using the proposed method and HRP-2, we have realized a dance performance of Japanese folk dance by the robot, which is synchronized with a performance of a human grand master on the same stage.