In this paper, we propose a method of visual recognition in cooperation with actions for automatic handling of clothing by a robot. Difficulty in visual recognition of clothing largely depends on the observed shape of the clothing. Therefore, strategy of actively making clothing into the shape easier to recognize should be effective. First, after clothing is observed by a trinocular stereo vision system, it is checked whether the observation gives enough information to recognize the clothing shape robustly or not. If not, proper “recognition-aid” actions, such as rotating and/or spreading the clothing, are automatically planned based on the visual analysis of the current shape. After executing the planned action, the clothing is observed again to recognize. The effect of the action of spreading clothes was demonstrated through experimental results using an actual humanoid.

In this paper, we propose a deformable-model-driven method to recognize the state of hanging clothes using three-dimensional (3D) observed data. For the task to pick up a specific part of the clothes, it is indispensable to obtain the 3D position and posture of the part. In order to robustly obtain such information from 3D observed data of the clothes, we take a deformable-model-driven approach[4], that recognizes the clothes state by comparing the observed data with candidate shapes which are predicted in advance. To carry out this approach despite large shape variation of the clothes, we propose a two-staged method. First, small number of representative 3D shapes are calculated through physical simulations of hanging the clothes. Then, after observing clothes, each representative shape is deformed so as to fit the observed 3D data better. The consistency between the adjusted shapes and the observed data is checked to select the correct state. Experimental results using actual observations have shown the good prospect of the proposed method.

In this paper, we propose a strategy for a dual-arm robot to pick up a specific part of clothing with one hand while holding the item of clothing with its other hand. Due to the large deformability of clothing, the handling requirements differ from those required for the handling of rigid objects. In the case of holding a specific part of clothing, large deformation leads to a large variety of positions and orientations of the target part, requiring flexibility of both visual recognition and motion control. On the other hand, since the clothing can flexibly curve over the hand, a relatively large range of suitable actions is allowed for grasping the clothing. By considering these characteristics, the following three-stage strategy is proposed. First, the state of the clothing is recognized from visual observation of the clothing using a deformable-model [1]. Then, the theoretically optimal position and orientation of the hand for handling a specific part of the clothing is calculated based on the recognition results. Finally, the position and orientation of the hand is modified by considering the executable motion range of the dual arms. Preliminary experimental results using actual observations of a humanoid robot were used to validate the effectiveness of the proposed strategy.

In multimodal tele-cooperation as considered in this paper two humans in distant locations jointly perform a task requiring multimodal including haptic feedback. One human operator teleoperates a remotely placed humanoid robot which is collocated with the human cooperator. Time delay in the communication channel as destabilizing factor is one of the multiple challenges associated with such a tele-cooperation setup. In this paper we employ a control architecture with force-position exchange accounting for the admittance type of the haptic input device and the telerobot, which both are position-based admittance controlled. Llewellynpsilas stability criteria are employed for the parameter tuning of the virtual impedances in the presence of time delay. The control strategy is successfully validated in an intercontinental tele-cooperation experiment with the humanoid telerobot HRP-2 located in Japan/Tsukuba and a multimodal human-system-interface located in Germany/Munich, see also the corresponding video submission. The proposed setup gives rise to a large number of exciting new research questions to be addressed in the future.

The video shows an intercontinental cooperative telemanipulation task, whereby the operator site is located in Munich, Germany and the teleoperator site in Tsukuba, Japan. The human operator controls a remotely located teleoperator, which performs a task in the remote environment. Hereby the human operator is assisted by another person located at the remote site. The task consists in jointly grasping an object, moving it to a new position and finally releasing it, see Fig. 1.

This paper introduces a framework for whole-body motion generation integrating operator's control and robot's autonomous functions during online control of humanoid robots. Humanoid robots are biped machines that usually possess multiple degrees of freedom (DOF). The complexity of their structure and the difficulty in maintaining postural stability make the whole-body control of humanoid robots fundamentally different from fixed-base manipulators. Getting hints from human conscious and subconscious motion generations, the authors propose a method of generating whole-body motions that integrates the operator's command input and the robot's autonomous functions. Instead of giving commands to all the joints all the time, the operator selects only the necessary points of the humanoid robot's body for manipulation. This paper first explains the concept of the system and the framework for integrating operator's command and autonomous functions in whole-body motion generation. Using the framework, autonomous functions were constructed for maintaining postural stability constraint while satisfying the desired trajectory of operation points, including the feet, while interacting with the environment. Finally, this paper reports on the implementation of the proposed method to teleoperate two 30-DOF humanoid robots, HRP-1S and HRP-2, by using only two 3-DOF joysticks. Experiments teleoperating the two robots are reported to verify the effectiveness of the proposed method.

This paper deals with the generation of motion pattern for humanoid robots vertical jump. The study concentrates on the landing phase of the jump which is the most demanding for the system because of high discontinuity of acceleration at impact time. We first decompose the motion in two functions. The first one starts just before landing and consists in maximizing the value of the discontinuity by reducing the feet/ground velocity. We set the feet, COM and feet/COM velocities according to the supposed height of the jump. At impact time, the second function uses measured data of ground reaction force (GRF) from the ankle force sensors to quickly stop the system while keeping its vertical equilibrium. The influence of each function and their respective parameters is discussed and analyzed, and their validity is tested using the model of an existing humanoid platform

The wide potential applications of humanoid robots require that the robots can walk in complex environments and overcome various obstacles. To this end, we address the problem of humanoid robots stepping over obstacles in this paper. We focus on two aspects, which are feasibility analysis and motion planning. The former determines whether a robot can step over a given obstacle, and the latter discusses how to step over, if feasible, by planning appropriate motions for the robot. We systematically examine both of these aspects. In the feasibility analysis, using an optimization technique, we cast the problem into global optimization models with nonlinear constraints, including collision-free and balance constraints. The solutions to the optimization models yield answers to the possibility of stepping over obstacles under some assumptions. The presented approach for feasibility provides not only a priori knowledge and a database to implement stepping over obstacles, but also a tool to evaluate and compare the mobility of humanoid robots. In motion planning, we present an algorithm to generate suitable trajectories of the feet and the waist of the robot using heuristic methodology, based on the results of the feasibility analysis. We decompose the body motion of the robot into two parts, corresponding to the lower body and upper body of the robot, to meet the collision-free and balance constraints. This novel planning method is adaptive to obstacle sizes, and is, hence, oriented to autonomous stepping over by humanoid robots guided by vision or other range finders. Its effectiveness is verified by simulations and experiments on our humanoid platform HRP-2.

In this paper, we address the problem of how a humanoid robot can step over a given obstacle. Obstacle stepping-over has two aspects, namely, feasibility analysis and motion planning. The former determines whether the robot can step over the obstacle, and the latter discusses how to realize the stepping-over, if it is feasible, by trajectory planning. The paper focuses on the latter. Specifically, based on our previous analysis of feasibility, we present a novel algorithm to plan suitable trajectories for obstacle stepping-over, taking into account two basic requirements. The first requirement is to avoid any collision between the robot and the obstacle, and the second to maintain stability or balance of the robot. To meet them, we decompose the whole body motion of the robot into two parts, corresponding to the upper body motion and the lower body, respectively. We first plan collision-free trajectories of the feet and the waist for lower body motion, and then adjust upper body motion by resolved momentum control to guarantee the robot stability. This novel planning method is adaptive to obstacle sizes and hence oriented to autonomous stepping-over of humanoid robots guided by vision or other range finders. Its effectiveness is shown by simulation and experiment on our humanoid platform HRP-2.

The establishment of an intuitive and effective whole body operation method is essential for the utilization of humanoid robots in real world tasks. Here we introduce a method for humanoid robot foot teleoperation, as a part of our whole body teleoperation system using only a simple joystick device. This paper explains a framework for real-time foot operation which incorporates the operator's foot command and robot's autonomy in maintaining balance. We describe the algorithm of an autonomous function which shifts the position of the robot's center of mass interactively based on the operator's command and the current feet condition of the robot. We report on successful experimental results of teleoperating a real humanoid robot HRP-2 using the proposed method.

It is believed that humanoid robots have better mobility than other mobile robots. However, little work has been reported on this mobility for nontrivial motion such as walking on rough terrains and overcoming obstacles. In this paper, we address the problem of obstacle overcoming by stepping-over, focusing on the feasibility analysis. That is, given an obstacle to overcome, we determine if the robot can step over it. In our analysis, we present a technique of global optimization under constraints to obtain the maximum height or width of an obstacle that the robot can step over while maintains the stability and avoids any collision between the robot legs and the obstacle. This analysis is a basis of motion planning for the robots to step over obstacles. It also provides an approach for evaluating the capability of obstacle overcoming of humanoid robots.

This paper proposes a new method for the teleoperation of humanoid robots which integrates human operator's intention with robot's autonomy. Getting hints from human conscious and subconscious motion generations, we propose a method which generates whole body motions of a humanoid robot satisfying operator's desired movement of specific points on which the operator focuses as well as the robot's balance. The proposed method is based on a whole body momentum control. We also introduce a function of automatic adjustments of the position of the center of mass and torso orientation in order to expand the reachable area of a humanoid robot. The effectiveness of the method is confirmed by using humanoid robot simulator OpenHRP with physical parameters of real humanoid robot HRP-1S.

This paper presents a whole body teleoperation system for a humanoid robot integrating operator's intention and the robot's autonomy. Instead of giving command to all the joints of the body, we have been proposing a switching command based teleoperation system of which the operator selects only the necessary points of the robot's body to manipulate depending on the required tasks. We have also proposed a whole body motion generation method, which satisfies both, and the desired movement of specific points of the robot's body and the robot's balance constraint. This paper first explains the concept of the proposed teleoperation system and the autonomous functions for maintaining balance and expanding the reachable area of a humanoid robot. Then the paper reports on the experimental results using the proposed methods to teleoperate a real humanoid robot HRP-1S to perform balanced whole body motions satisfying operator's input command.

A teleoperation system for whole body motions of a humanoid robot using a simple joystick master device is developed. Humanoid robots are physically similar to humans and usually possess a large number of degrees of freedom. By using hints from the shifting of the focus of attention for body motions between the joints of the human body during task executions, we propose a switching command based teleoperation system in which the operator selects only the necessary point of the humanoid robot's body for manipulation. In this paper, we present the implementation of this switching command based teleoperation system and experimental results using this system to teleoperate the humanoid robot HRP-1S developed in the Humanoid Robotics Project of the Ministry of Economy, Trade and Industry of Japan.