In this study, we propose and implement a method for grasping only the topmost layer of stacked fabrics with a dual-arm robot and placing it at a target position. The proposed method employs a three-finger robot hand consisting of one adhesive finger and two non-adhesive grasping fingers to grasp only the top fabric layer. After grasping, feature matching is performed to determine the rotation angle and translation vector required to align the fabric with the target position. The fabric is then moved while being pressed by the fingers. The proposed method was implemented on a robot and validated through experiments using five different fabrics with varying structures, surface materials, masses, and thicknesses, thereby confirming its effectiveness.

When a robot manipulates a flexible object, a delicate grasp is required. Slip is an important indicator that can enable a robotic manipulator to delicately grasp an object. Therefore, we created a 3×3 tactile sensor matrix covered with a elastomer grip, intended to be used on a robotic fingertip, to detect slip. We obtained tactile sensor values corresponding to the slip and nonslip states in advance and created a training dataset with several patterns. Random Forest Classifiers were trained on the datasets, and the results were compared. All nine sensor elements in the sensor matrix could detect slips with practical accuracy in the real-time robot experiments. In time periods wherein no slip occurred, no nonslip states were erroneously determined as slip states under any of the tested conditions. The slip detector created in this study was demonstrated to be applicable to types of rope that were not used to train the detector.

Service robots have developed considerably in recent years. This study aims to propose two approaches for controlling the motion of a service robot as it pours liquid precisely from an unknown container into another unknown container without the need of any external tools. To realize this task, we must resolve two sub-tasks, which are determining the poured volume and controlling the pouring action. Our first proposal concentrates on the target container. The poured volume is calculated using a model of target container and the height of liquid in the target container. The action is controlled using a proportional-derivative controller, which considers the angular speed of the pouring container as a process variable and the poured volume as a control variable. Our second method concentrates on the pouring container. The poured volume is calculated using the relation between the angle of the pouring container and poured volume. The action is controlled with a simple proportional controller that takes the angular speed of the pouring container as a process variable and target angle as a control variable. A point cloud is used to model the two containers. These two methods were implemented in a dualarm robot system for testing, and the results show that both methods are effective for controlling precise pouring tasks.

We have been developing a paradigm that we call learning-from-observation for a robot to automatically acquire a robot program to conduct a series of operations, or for a robot to understand what to do, through observing humans performing the same operations. Since a simple mimicking method to repeat exact joint angles or exact end-effector trajectories does not work well because of the kinematic and dynamic differences between a human and a robot, the proposed method employs intermediate symbolic representations, tasks, for conceptually representing what-to-do through observation. These tasks are subsequently mapped to appropriate robot operations depending on the robot hardware. In the present work, task models for upper-body operations of humanoid robots are presented, which are designed on the basis of Labanotation. Given a series of human operations, we first analyze the upper-body motions and extract certain fixed poses from key frames. These key poses are translated into tasks represented by Labanotation symbols. Then, a robot performs the operations corresponding to those task models. Because tasks based on Labanotation are independent of robot hardware, different robots can share the same observation module, and only different task-mapping modules specific to robot hardware are required. The system was implemented and demonstrated that three different robots can automatically mimic human upper-body operations with a satisfactory level of resemblance.

This paper proposes a motion-planning method for a wrapping-with-fabric task. To wrap, robots require not only offhand paths (which represent the movement of a grasping point on fabric) but also actual robot motion. When a robot handles fabric, it must consider the range of motion and collisions. To overcome such limitations, the robot must pass fabric inter- and intra-hand. The proposition is based on the reliability of its movements. The method may be used to plan a combination of inter- and intra-hand passing motions to generate the basic movements for wrapping along an offhand path. Experiments show how, by using the proposed planning method, a dual-arm robot can wrap an L-shaped pipe.

This paper proposes an architecture with a two-layered structure for implementing an air-hockey robot that performs tactically in the deployment of entertainment robots. The two-layered architecture is comprised of two components: a tactical component that governs tactics and a skill component that deals with individual shots. The skill layer shoots a puck autonomously by selecting an appropriate way of shooting while considering what is physically feasible. The tactical layer controls the behavior of the skill layer by adjusting a probability function for selecting shooting methods. In this study, we implemented the architecture and conducted experiments to demonstrate its usefulness.

This paper describes emotional body expressions by flying robots for displaying emotions. Subjective evaluation experiments were performed to clarify the relation between movement parameters and subjective feeling of the expressions. Emotional body expressions were designed by eight people. The expressions were evaluated by another group of twelve people, who answered a questionnaire. The elements of emotional body expressions were investigated using the Lasso method. The results indicate movements that are most effective for displaying a particular emotion to a human. Such expressions are expected to enhance communication between the flight robots and humans.

This paper describes the implementation of a twisting skill to a multi-DOF end effector for manipulating linear deformable objects (LDOs). When manipulating of LDOs such as rope knotting, an important factor is to control torsion. However, controlling torsion is difficult using only the robot arm movement because of the range limit of a manipulator and the danger of collision with the environment such as a table and objects to be tied. On the other hand, implementing the movement into a robot hand as in-hand manipulation means that the robot-arm movement can be compact. In this study, we extract what is needed in a “twisting skill”, implement them into multi-DOF end effectors mounted on a dual-arm robot, and evaluate the skill. Then, we demonstrate rope manipulation including a knotting task and verify the effect by using a twisting skill. Our results show that the proposed method can be used to control the twisted angle of the rope with the required precision. Furthermore, using the proposed twisting skill, when placing a rope on a table top, the accuracy and reproducibility are improved compared with the case when the skill is not used. Finally, using the proposed method, clove hitches are successfully tied.

In the present paper, we propose a method for in-air knotting by a dual-arm, multi-finger robot. Most previous studies about knotting by robots assume that the rope is placed on a table. However, in-air knotting requires more skillful hand manipulations, such as swapping ropes between the right and left hands. Therefore, we herein focus on hand motion and first extract the essential hand motions for knotting, referred to herein as skill motions, by observing human knotting procedures. Next, hardware with the capability of executing these skill motions is developed. Finally, experiments are conducted to confirm that several types of knots can be tied in the air using the proposed method.

In recent years, the research of humanoid robots that replace human tasks in emergency situations have been widely studied. Currently, many approaches are automate dedicated hardware for each mission. But, at the environment where situation changes, operation by humanoid robot is effective to operate equipments which designed for human. Ultimately, automation is ideal, but under the present circumstances, teleoperation of humanoid robot is effective for corresponding changes of situation. An intuitive interface is required for effectively controlling the humanoid robot from a distant place. Recently, the interfaces that map the human motion to the humanoid robot have become popular because of the development of the motion recognition systems. However, the humanoid robot and human beings have different joint structure, physical ability and weight balance. It is not practical to map the motion directly. There is also the issue of time delay between the operator and the robot. Therefore, it is desirable that the operator performs global judgments and the robot runs semi-autonomously in the local environment. In this paper we propose a method to remotely operate the humanoid robot by the task model. Our method describes human behavior abstractly by the task model and mapped this abstract expressions to humanoid robots, and overcome difference of structure of body. In this work, we operate lever of buggy-type vehicles as a example of mapping using the task model.

This paper describes a method for generating hand trajectories for a robot to enable it to place a rope on a table in a target shape. In many aspects of rope-work, e.g., making knots, hitches, and bends, it is important to control not only the topological relations (e.g., a tangle of string), but also to control the shape (e.g., the position, size, and so on). The shape of the rope on a table is the starting stage of the rope-work in that it affects the result of the rope-work, e.g., the position and size of a knot. Thus, it is necessary to generate hand trajectories automatically according to the target rope-shape when executing rope-work with a robot arm. To generate trajectories from a tabletop target shape, it is necessary to consider a positional relationship between the robot hand and the touchdown point, which is the point that the rope is placed on the table. In this study we propose a rope-shape model which can be used in limited situations, and derive hand trajectories from the model in a simple manner. The parameter needed for the model is easily identified. An experiment of making a loop was conducted to verify the effectiveness and limitations of the proposed method. As further verification, the method was applied to making a clove hitch using a dual-arm robot. In the starting stage of tying a clove hitch, two loops were created in a specified shape, and the clove hitch was successfully tied.

Conventional automatic doors cannot distinguish between people wishing to pass through the door and people passing by the door, so they often open unnecessarily. This paper describes an intelligent door system that observes people near a door so that the door opens at the proper time and only for people who have the intention of passing through it. To achieve this, we developed a novel laser range scanner that scans the surrounding environment three dimensionally, and formulated lightweight algorithms for the sensor. The experimental results show that the sensor can control a door according to the speed of people within the required specifications and that it correctly identifies people passing by the door.

In this paper, we present a humanoid robot which extracts and imitates the person-specific differences in motions, which we will call style. Synthesizing human-like and stylistic motion variations according to specific scenarios is becoming important for entertainment robots, and imitation of styles is one variation which makes robots more amiable. Our approach extends a learning from observation (LFO) paradigm which enables robots to understand what a human is doing and to extract reusable essences to be learned. The focus is on styles in the domain of LFO and the representation of them using the reusable essences. In this paper, we design an abstract model of a target motion defined in LFO, observe human demonstrations through the model, and formulate the representation of styles in the context of LFO. Then we introduce a framework of generating robot motions that reflect styles which are automatically extracted from human demonstrations. To verify our proposed method we applied it to a ring toss game, and generated robot motions for a physical humanoid robot. Styles from each of three random players were extracted automatically from their demonstrations, and used for generating robot motions. The robot imitates the styles of each player without exceeding the limitation of its physical constraints, while tossing the rings to the goal.

This paper presents the development toward a dancing robot that can listen to and dance along with musical performances. One of the key components of this robot is the ability to modify its dance motions with varying tempos, without exceeding motor limitations, in the same way that human dancers modify their motions. In this paper, we first observe human performances with varying musical tempos of the same musical piece, and then analyze human modification strategies. The analysis is conducted in terms of three body components: lower, middle, and upper bodies. We assume that these body components have different purposes and different modification strategies, respectively, for the performance of a dance. For all of the motions of these three components, we have found that certain fixed postures, which we call keyposes, tend to be preserved. Thus, this paper presents a method to create motions for robots at a certain music tempo, from human motion at an original music tempo, by using these keyposes. We have implemented these algorithms as an automatic process and validated their effectiveness by using a physical humanoid robot HRP-2. This robot succeeded in performing the Aizu-bandaisan dance, one of the Japanese traditional folk dances, 1.2 and 1.5 times faster than the tempo originally learned, while maintaining its physical constraints. Although we are not achieving a dancing robot which autonomously interacts with varying music tempos, we think that our method has a vital role in the dancing-to-music capability.

The goal of this work is to teach a robot to regrasp an object using knowledge obtained from human demonstration. This paper presents a task model that represents a human regrasping movement. The task model is based on the topological information and comprised of four task primitives. Human regrasping movement is recognised and represented as a sequence of these task primitives by the proposed recognition algorithm. The proposed method then maps each task primitive to the target robot hand using knowledge obtained from human demonstration. The experimental result verified the proposed task model by executing the regrasping movement on the real robot hand.

This paper describes a localization technique for a mobile robot with unknown obstacles, such as pedestrians. It is important that a robot can run in environment freely in order to perform goods conveyance efficiently. In order to run in environment and deliver goods to a target position, self-position presumption is important. However, In real environment, robot may be unable to run to a target position, because the robot cannot localize the self-position correctly with unknown objects, such as people and goods. In this paper, this problem is solved by utilizing the robust position localization technique based on the observed space information to a delivery robot's navigation. Furthermore, by building the system which can set a robot's run path easily, flexible path creation and autonomously running are realized. we verified the validity of the proposed method by the localization experiment and the goods delivery demonstration in real environment. The system which localizes self-position with 0.1m error even in the environment where an unknown object exists, and performs goods delivery to a target position was realized.

This paper describes a localization technique for a mobile robot with unknown obstacles, such as pedestrians. Many robust localization technnique with unknown obstacles by using a particle filter and laser scanner have been propsed. In particular, Tomizawa's method(Space observation model) is effective. However, this method has problem that it does not consider the fixed object occlusion. Thus, in the environment where thin wall (this is example), the robot position cannot be localized well. In this research, we solve this problem by rebulding the space information which is observed intrinsically in predicted position by considerinmg the fixed object occlusion. From above, we realize robot position localization with more high accuracy than previous method, by solving the problem of previous method and improving the previous method. We evaluate the validity of the proposed method by a simulation and an experiment in a real environment.

Humanoid robots have discrepancies between the postures simulated on models and those of actual humanoid bodies. To correct this problem, this paper proposes a method in which a robot observes its own posture in a mirror. This method does not need an additional time-consuming setup, such as calibration of external cameras. We attach a reference point to a known position on the robot body. By observing markers and the reference point in the same view, we can estimate the position of the markers accurately based on the geometric relationship between the mirror, an object, and its mirror reflection. We evaluate the proposed method by experiments of measuring and correcting the posture of an actual robot body to attain the target posture.

This paper proposes a new strategy for making a knot with a general-purpose arm. The strategy is divided into four stages: First, a human-performed knotting action is analyzed to acquire some motion primitives. Second, based on these databases, the robot is taught to tie a knot. Third, key points on the stored trajectories are extracted. These key points are considered in a general case, and then used to implement the manipulation skills. Finally, knotting task is generated by synthesis of the manipulation skills. An overhand knot was made by the actual robot for the validity of the proposed method. This paper also describes a system for manipulating deformable objects (e.g. knotting a rope).

This paper introduces a novel method to teach a robot to regrasp an object based on the Programming by Demonstration paradigm. In this paradigm, a robot observes a human performing a regrasping task via various sensors to recognise crucial information in order to reproduce the task using its own hand. The main contribution is in the proposal of a representation technique that can analyse a human regrasping movement and reproduce this movement in a robot hand. The technique is based on tangle topology where both hand and manipulated object are considered as strands. This allows a regrasping movement to be considered as an alteration of the tangle relationship between the strands (hand and the object) over time. Human regrasping movements are analysed and reproduced in multi-fingered robot hands in a grasp simulation to demonstrate the efficiency of the proposed method.

In this paper, we propose a method to achieve temporal scaling of leg motions as a fundamental technique for a music feedback system of a dancing humanoid robot. We asked dancers to perform dance motion at normal musical tempo and faster musical tempos and observed how dancers modified performance for given musical tempos. The obtained insights from the observation are 1) a dancer needs to preserve leg postures that are important to emphasize dance expression, 2) there is a priority to determine what features of leg motion can be adjusted, and 3) stylistic leg motion resembles normal step motion if dancers cannot follow fast musical tempo completely. Based on these insights, we generate leg motion appropriately adjusted for changing musical tempo while maintaining balance. We validated our method via simulation experiments with a humanoid robot HRP-2.

This paper presents a novel method for robust dance motion structure detection. In the japanese folk dance domain, teachers created illustrations of dance poses. These poses characterize the most important movements of a dance. So far there is no simple and reliable extraction method which can extract all poses as shown in these drawings. We use these poses for the Task Model (TM) in the context of Learning from Observation (LFO). LFO which is a well known technique for successful human to robot motion mapping, consists of tasks (what to do) and skills (how to do). We propose a novel approach, to extract special motions from a dance, called turning motions useful for skill mapping in the LFO paradigm. Furthermore, we use a modified version of this approach, to detect all poses as shown in the drawings, called turning poses. To achieve this we observe both forearms at the same time and analyze their movement in different 2-D coordinate planes. We evaluate the parameters with and without a weighting function where we minimize acceleration, velocity and power. We successfully demonstrate this novel method using two very different japanese folk dances and discuss further implications of this work in respect to the LFO paradigm and dances of other domains.

This paper presents visual perception discovered in high-level manipulator planning for a robot to reproduce the procedure involved in human painting. First, we apply a technique of 2D object segmentation that considers region similarity as an objective function and edge as a constraint with artificial intelligent used as a criterion function. The system can segment images more effectively than most of existing methods, even if the foreground is very similar to the background. Second, we propose a novel color perception model that shows similarity to human perception. The method outperforms many existing color reduction algorithms. Third, we propose a novel global orientation map perception using a radial basis function. Finally, we use the derived model along with the brush's position- and force-sensing to produce a visual feedback drawing. Experiments show that our system can generate good paintings including portraits.

This paper introduces a Multi Factor Tensor(MFT) model to recognize motion styles and person identities in dance sequences. We apply a musical information analysis method in segmenting the motion sequence relevant to the key poses and the musical rhythm. We define a task model considering the repeated motion segments, where the motion is decomposed into person invariant factor task and person dependant factor style. We capture the motion data of different people for a few cycles, segment it using the musical analysis approach, normalize the segments using a vectorization method, and realize our MFT model. The experiments are conducted according to two approaches. Various experiments that we conduct to evaluate the potential of the recognition ability of our proposed approaches and the results demonstrate the high accuracy of our model. The recognition results and the motion decomposition will be used in further extending the motion generation process in various styles and for different tasks.

This paper presents visual perception discovered in high-level manipulator planning for a robot to reproduce the procedure involved in human painting. First, we propose a technique of 3D object segmentation that can work well even when the precision of the cameras is inadequate. Second, we apply a simple yet powerful fast color perception model that shows similarity to human perception. The method outperforms many existing interactive color perception algorithms. Third, we generate global orientation map perception using a radial basis function. Finally, we use the derived foreground, color segments, and orientation map to produce a visual feedback drawing. Our main contributions are 3D object segmentation and color perception schemes.

This paper proposes a method to model the modification of upper body motion of dance performance based on the speed of played music. When we observed structured dance motion performed at a normal music playback speed and motion performed at a faster music playback speed, we found that the detail of each motion is slightly different while the whole of the dance motion is similar in both cases. This phenomenon is derived from the fact that dancers omit the details and perform the essential part of the dance in order to follow the faster speed of the music. To clarify this phenomenon, we analyzed the motion differences in the frequency domain, and obtained two insights on the omission of motion details: (1) High frequency components are gradually attenuated depending on the musical speed, and (2) important stop motions are preserved even when high frequency components are attenuated. Based on these insights, we modeled our motion modification considering musical speed and joint limitations that a humanoid robot has. We show the effectiveness of our method via some applications for humanoid robot motion generation.

We propose a method of maintaining balance for a human-like character against large perturbations. The method enables a human-like model to maintain its balance with active whole-body motion, such as rotating its arms, bending down, and taking a step, if necessary. First, we capture the human motions of maintaining balance and abstract essential mechanisms from these motions. Next, we construct a model of maintaining balance that has a simple structure, such as an inverted pendulum. This model has two modes of maintaining balance: keeping the feet on the ground, and stepping. In this paper, the stepping mode is mainly described. Finally, we generate whole-body motion based on the model against several perturbations, and we discuss the validity of our method

This paper presents a method to optimize and filter trajectories generated from recorded human motion for a humanoid robot with physical limits. The objective function is responsible for mimicking human trainers, enhancing the possibility for fast convergence, while constraints are used to transform motion within the limit of the capabilities of the humanoid robot. Those constraints for angle, velocity, and dynamic force are represented as B-spline coefficients. An iterative soft-constraint paradigm is proposed to enhance the quality of velocity and force constraints. Collision avoidance is also considered as a constraint. For precision refinement, in regions of high-frequency motion not adequately modeled by an initial splining, B-spline is extensible into a hierarchy so that optimization that meets global criteria can be performed locally. Furthermore, all of the constraints can be used solely as filtering. To use these filters, an effective method to directly decompose a trajectory to a B-spline is also presented

In this paper, we propose a new method for biped humanoids to compensate for large amounts of angular momentum induced by strong external perturbations applied to the body during gait motion. Such angular momentum can easily cause the humanoid to fall down onto the ground. We use an Angular Momentum inducing inverted Pendulum Model (AMPM), which is an enhanced version of the 3D linear inverted pendulum model to model the robot dynamics. Because the AMPM allows us to explicitly calculate the angular momentum generated by the ground reaction force, it is possible to calculate a counteracting motion that compensates for the angular momentum generated by external perturbations in real-time.

From recorded human motion, trajectory optimization of a humanoid robot with physical limits being the key constraint is presented. The optimization's objective function preserves the salient characteristics of the original motion, while constraints are used to transform the motion to the limit of the capabilities of the humanoid robot. It is shown that using constraints to limit generated trajectories to physically realizable motion ensures that limits are being met more precisely than by reducing them using a standard objective function. The use of wavelets vs. B-spline is compared, and it is shown that B-spline data representation has pronounced advantages. Furthermore, in regions of high-frequency motion not adequately modeled by an initial splining, B-spline is extensible into a hierarchy so that optimizations can be performed locally which still meet global criteria. It is shown how to use B-spline coefficients in angle-, velocity-, acceleration-, and dynamic force-constraints. To detect trajectory subsections with excessive error in need of control point adjustment and local re-optimization, not only is a traditional error detector used, but also a B-spline density detector is also presented. Generated motions were tested using our simulation program and the robot HRP-2.

We present a new inverse kinematics method that utilizes the motion data for realtime control and editing. The key idea is to extract parameters necessary for inverse kinematics from the motion data. These parameters are the weight matrix, which determines the motion of the redundant joints, and the transformation functions that define the motion of the end effectors. The user can control the motion by dragging a body segment using a mouse, and the method calculates the new motion using the precomputed parameters. The method enables interactive editing, warping, and retargeting character motions.

In this paper, we propose a new method to generate C/sup 2/ continuous gait motion for biped robots. The method is based on the enhanced inverted pendulum mode, which can easily handle angular momentum around the center of gravity. Using our method, it is possible to plan motion paths for biped robots without discontinuity in the acceleration even during switching from single support phase to double support phase, and vice versa.

The postural balance system is one of the most fundamental functions for humanoid robot control. In this paper, we propose a new feedback balance control system for the human body. This system can manipulate large perturbations. It finds the optimal motion for maintaining balance in the 3D space without receiving any feed-forward input beforehand. Two different strategies are adopted for the optimization: the quadratic programming method and the PD control. Simulation results are compared with real human motion; many common features such as rotating arms are observed.