This paper presents a novel method for detecting and estimating contact forces only from human motions using machine learning techniques. Knowing the location of the contacts with the environment and the magnitude of the exerted force is critical for dynamic human motion analysis. However, their annotation is usually made manually from captured motion data especially in case of multiple contacts even if the data includes force measurement. Moreover, most existing human motion datasets do not include contact force. To overcome these bottlenecks, we introduce a network that leverages vector-quantized variational autoencoder (VQ-VAE) and self-attention that learns a small set of discrete feature values representing various contact states. These feature values, called contact codes, allow human motions to be converted to contact states and resulting forces. By applying an optimization for contact estimation with a reduced set of manual annotations, the existence of contacts can be automatically determined, which is essential information for dynamic analysis. We validated the effectiveness and potential usefulness of the proposed method with a human walking gait dataset, by converting the human motions into contact sequences and forces and applying the estimated contacts to dynamic motion analysis.

Anticipating a future scenario where the robot initiates its own actions and behaves voluntarily when collaborating with humans, our research focuses on human understanding and perception of unanticipated robot actions during physical human-robot interaction. While the current literature searches for key factors that make the human-robot collaboration successful, the question of how people experience the robot’s unanticipated action as cooperative or uncooperative seems to remain open. We designed a game-based experiment (N = 35) where the participant played a “catch-falling-coins” game by moving a robotic arm. Our experiment introduced unanticipated robot actions in an “active session” where the robot targeted higher-valued coins without first informing the participants. Through semi-structured interviews and statistical analysis of questionnaires (Big Five Personality Test, SAM, NARS and CH33), we examined the participants’ understanding of the robot’s “intention” and their positive or negative perception of the robot as cooperative or uncooperative. Among the participants who understood that the robot’s “intention” was to catch the higher-valued coins, the majority of them reported a positive perception of the robot (cooperative or helpful) while this was not the case among those who did not understand the robot’s intention. We also observed relevant relationships between some personality traits and a person’s understanding of the robot’s intention. Qualitative analysis of the interviews allowed us to structure the process of perception change during the game into three phases: confusion, investigation, and adaptation. We believe that our research contributes to the study of human perception, and particularly to the relationship between a human’s understanding of unanticipated robot actions and their positive or negative perception of the robot.

Teleoperation of humanoid robots enables the integration of the cognitive skills and domain expertise of humans with the physical capabilities of humanoid robots. The operational versatility of humanoid robots makes them the ideal platform for a wide range of applications when teleoperating in a remote environment. However, the complexity of humanoid robots imposes challenges for teleoperation, particularly in unstructured dynamic environments with limited communication. Many advancements have been achieved in the last decades in this area, but a comprehensive overview is still missing. This survey article gives an extensive overview of humanoid robot teleoperation, presenting the general architecture of a teleoperation system and analyzing the different components. We also discuss different aspects of the topic, including technological and methodological advances, as well as potential applications.

Unanticipated physical actions from the robot on humans [active physical human–robot interaction (pHRI)] may be inevitable with the deployment of robots in human-populated environments. However, it is still unclear how humans would perceive such actions and how the robot should execute them in a physically and psychologically safe manner. The objective of this article is to explore the possibility of quantifying the humans’ physical and mental state during an active physical interaction with a robot, by means of a laboratory experiment. We hypothesize that the active robot actions could cause measurable alterations in users’ data, which could be related to their perceptions and personalities. In the experiment, the user plays a visual game using the robot, which has a hidden task that results in active physical actions on the user. We collect data from physical and physiological sensors, and the perceptions and personalities via questionnaires and a semi-structured interview. Statistical analysis and clustering of the data collected from a total of 35 participants showed the relationships between participants’ physical and physiological data and their age, gender, perception, and personalities. Further developments based on these exploratory outcomes can be used to implement an active pHRI controller that can account for both the physical and the mental state of users.

A significant problem in brain-computer interface (BCI) research is decoding - obtaining required information from very weak noisy electroencephalograph signals and extracting considerable information from limited data. Traditional intention decoding methods, which obtain information from induced or spontaneous brain activity, have shortcomings in terms of performance, computational expense and usage burden. Here, a new methodology called prediction error decoding was used for motor imagery (MI) detection and compared with direct intention decoding. Galvanic vestibular stimulation (GVS) was used to induce subliminal sensory feedback between the forehead and mastoids without any burden. Prediction errors were generated between the GVS-induced sensory feedback and the MI direction. The corresponding prediction error decoding of the front/back MI task was validated. A test decoding accuracy of 77.83-78.86% (median) was achieved during GVS for every 100[Formula: see text]ms interval. A nonzero weight parameter-based channel screening (WPS) method was proposed to select channels individually and commonly during GVS. When the WPS common-selected mode was compared with the WPS individual-selected mode and a classical channel selection method based on correlation coefficients (CCS), a satisfactory decoding performance of the selected channels was observed. The results indicated the positive impact of measuring common specific channels of the BCI.

Recent advances in deep learning have shown high success in obtaining the 6-DoF pose of rigid objects. However, most works rely on a pre-existing dataset and do not tackle the data gathering part. The time-consuming and tedious tasks required to build datasets are, to a large extent, what is keeping these techniques from being more widely used in practical applications. We present a whole pipeline from data gathering to pose recognition and an example application of robot grasping. For our data gathering method we require as minimum user intervention as possible and, even without using depth information or 3D models, by using a novel RGB-only Neural Network design we are able to obtain results very close to the state of the art. We call this method Affordable Pose Estimation (APE).

Deformable objects are very common around us in our daily life. Because they have infinitely many degrees of freedom, they present a challenging problem in robotics. Inspired by practical industrial applications, we present in this paper our research on using a humanoid robot to take a long, thin and flexible belt out of a bobbin and pick up the bending part of the belt from the ground. By proposing a novel non-prehensile manipulation strategy "scraping" which utilizes the friction between the gripper and the surface of the belt, efficient manipulation can be achieved. In addition, a 3D shape detection algorithm for deformable objects is used during manipulation process. By integrating the novel "scraping" motion and the shape detection algorithm into our multi-objective QP-based controller, we show experimentally humanoid robots can complete this complex task.

This paper presents an inverse kinematics (IK) method which can control future velocities and accelerations for multi-body systems. The proposed IK method is formulated as a quadratic programing (QP) that optimizes future joint trajectories. The features of the proposed IK are: (1) the evaluation of accelerations at future time instances, (2) the trajectory representation that can implicitly integrate the time integral formula into QP, (3) the computation of future Jacobian matrices based on the comprehensive theory of differential kinematics proposed in our previous work. Those features enable a stable and fast IK computation while evaluating the future accelerations. We also conducted thorough numerical studies to show the efficiency of the proposed method.

The paper aims to develop a methodology of transferring the knowledge obtained from the experiments of laboratory animals to human musculoskeletal system. To achieve the goal, we propose a method for estimating the homologous posture of the mammalian skeletal system corresponding to the human body posture. We hypothesize the homology of bone geometry between mammalian species implies that of biomechanical functions. The method relies on this homology and determines the homologous postures according to the anatomical landmarks of bone geometry. This paper shows the results of the analysis on homologous postures between the human and mouse skeletal models to validate our hypothesis. A pilot study also introduces comparison of mechanical functions between the two models by using the homologous postures.

Motor functions of the biological system has been forged through 4 billion years evolution. From a neurorobotics view, it is important not only to know how well it works, but also how it fails. To quantitatively describe early onset symptoms of a neurodegenerative disease, we analyzed phenotypes of genetically engineered Huntington disease (HD) model mice, which reveal progressive impaired motor functions. We devised a simple yet sensitive paradigm called the crystalized motion profile (CMP), by which we successfully detected subtle difference between normal and abnormal mice in terms of whole-body level motor coordination. Our long-term objective is to remodel human mind and body to regain impaired motor and cognitive functions with ageing. To do so, we are developing a soft neurorobotic suit that provides integrated cognitive and physical interventions to users. Our analysis on the HD model mice is important as the first step to bridge between molecular mechanisms (altered genetic code) and the macroscopic neuro-musculoskeletal model. With this, we can extrapolate from knowledge of non-human mammals to human to derive the remodeling.

A motion planning algorithm computes the motion of a robot by computing a path through its configuration space. To improve the runtime of motion planning algorithms, we propose to nest robots in each other, creating a nested quotient-space decomposition of the configuration space. Based on this decomposition we define a new roadmap-based motion planning algorithm called the Quotient-space roadMap Planner (QMP). The algorithm starts growing a graph on the lowest dimensional quotient space, switches to the next quotient space once a valid path has been found, and keeps updating the graphs on each quotient space simultaneously until a valid path in the configuration space has been found. We show that this algorithm is probabilistically complete and outperforms a set of state-of-the-art algorithms implemented in the open motion planning library (OMPL).

We propose multi-view and volumetric convolutional neural networks (ConvNets) for 3D shape recognition, which combines surface normal and height fields to capture local geometry and physical size of an object. This strategy helps distinguishing between objects with similar geometries but different sizes. This is especially useful for enhancing volumetric ConvNets and classifying 3D scans with insufficient surface details. Experimental results on CAD and real-world scan datasets showed that our technique outperforms previous approaches.

This paper presents a novel optimization method for generating persistently exciting trajectories for inertial parameters identification of a robot. The exciting performance of the trajectories is usually evaluated by the condition number of the regressor matrix, which appears in the linear regression model for identification. In this paper, the efficient formulation is presented to directly compute the gradient of the condition number with respect to joint trajectory parameters, by deriving the derivative of the singular values and regressor matrices. Direct gradient computation can enhance computational performance of optimization, which is essential for large DOF systems under many physical consistent conditions such as humanoid robots. The proposed method is validated by generating several trajectories for the humanoid robot HRP-4.

This paper presents a novel method for retargeting human motions onto a humanoid robot. The method solves the following three simultaneous problems: the geometric parameter identification that morphs the human model to the robot model, motion planning for a robot, and the inverse kinematics of the human motion-capture data. Simultaneous solutions can imitate the original motion more accurately than conventional approaches, which solve the problems sequentially. The proposed method can reconstruct the human motion within the physical constraints imposed by robot dynamics. A reconstruction step enables quantitative analysis of the retargeting results through direct comparison with the original human motion. The method can also provide the precise morphing function as well as subject-specific models, which can handle the different body dimensions of human subjects. This new framework is suitable for applications that require an accurate generation of human-like motions with quantitative evaluation criteria, such as humanoid robots that evaluate assistive devices. Experimental tests of the proposed method were performed with humanoid robot HRP-4.

In this paper, we introduce a nonlinear dimensionality reduction (NLDR) technique that can construct a low-dimensional embedding efficiently and accurately with low embedding distortions. The key idea is to divide NLDR into nonlinearity reduction and linear dimensionality reduction, which simplifies the overall NLDR process. Nonlinearity reduction is based on the elastic shell model that measures the in-plane stretching and bending energy. With this model, we minimize the curvature of the data, which is the source of nonlinearity, while preserving the original intrinsic property (i.e., local lengths) as-much-as possible. We discretize and linearize our nonlinearity reduction model such that it leads to an iterative deformation technique that alternates between two steps in order to flatten a manifold: the curvature minimization step that solves a bi-Laplace system and the local length restoration step that solves a Poisson system. We propose an efficient optimization technique for the both steps using a direct solver based on Cholesky decomposition, which exploits the fact that the system matrices stay constant; during iterations, we reuse the factorizations that are obtained once at the beginning and perform back substitutions only. Since our algorithm relies only on local geometric properties, it can accurately embed the data with complicated topology. Experimental results show that our algorithm is faster than the most of other state-of-the-art algorithms and preserves local areas and angles better than previous approaches.

We propose a method for estimation of humanoid and human links' inertial parameters. Our approach formulates the problem as a hierarchical quadratic program by exploiting the linear properties of rigid body dynamics with respect to the inertia parameters. In order to assess our algorithm, we conducted experiments with a humanoid robot and a human subject. We compared ground reaction forces and moments estimated from force measurements with those computed using identified inertia parameters and movement information. Our method is able to accurately reconstruct ground reaction forces and force moments. Moreover, our method is able to estimate correctly masses of the robots links and to accurately detect additional masses placed on the human subject during the experiments.

This paper presents the method to retarget human motion. The method can evaluate the ability of the preservation of the original characteristics of human motion data. It enables to compute the joint trajectories of the human corresponding with the retargeted ones of the robot at the same time, by utilizing the geometric identification technique. The obtained trajectories of a human are the solution to minimize the cost function about motion reproduction. The proposed method is efficient for such applications that the robot needs to mimic human motion without modifying the detailed features of the original movement of each body segment. The results of the retargeted motions to a humanoid robot are shown.

The mass parameters of robots influence performances of model-based control and validation of the simulation results. The mass parameters provided by CAD data are usually rough approximation of the true parameters. Therefore several methods for estimation of those parameters have been proposed. Their precision depends on the used motion, called optimal exciting trajectories. This paper describes a new approach to determine humanoid robot exciting trajectories for mass parameters identification. The method was inspired by the studies done in the field of human mass parameters identification, and it is based on observation of condition numbers of sub-regressor matrices created from the columns of the regressor matrix. The method has been experimentally applied to identify mass parameters of HRP-2 and HRP-4 humanoid robots. The proposed method is able to reconstruct ground reaction forces and force moments more accurately than parameters obtained from CAD data.

We present a nonrigid shape matching technique for establishing correspondences of incomplete 3D surfaces that exhibit intrinsic reflectional symmetry. The key for solving the symmetry ambiguity problem is to use a point-wise local mesh descriptor that has orientation and is thus sensitive to local reflectional symmetry, e.g. discriminating the left hand and the right hand. We devise a way to compute the descriptor orientation by taking the gradients of a scalar field called the average diffusion distance (ADD). Because ADD is smoothly defined on a surface, invariant under isometry/scale and robust to topological errors, the robustness of the descriptor to non-rigid deformations is improved. In addition, we propose a graph matching algorithm called iterative spectral relaxation which combines spectral embedding and spectral graph matching. This formulation allows us to define pairwise constraints in a scale-invariant manner from k-nearest neighbor local pairs such that non-isometric deformations can be robustly handled. Experimental results show that our method can match challenging surfaces with global intrinsic symmetry, data incompleteness and non-isometric deformations.

Failure detection and correction is essential in robust systems. In robotics, failure detection has focused on traditional parts assembly, tool breakage, and threaded fastener assembly. However, not much work has focused on sub-mode failure classification. This is an important step in order to provide accurate failure recovery. Our work implemented a novel failure characterization scheme for cantilever snap assemblies. The approach identified exemplars that characterized salient features for specific deviations from a nominal trajectory. Then, a rule based approach with statistical measures was used to identify failure and classify failure sub-modes. Failure sub-mode classification was evaluated by using a reliability measure. Our work classified failure deviations with 88% accuracy. Varying success was experienced in correlating failure deviation modes. Cases with only 1-deviation had 86% accuracy, cases with 2-deviations had 67% accuracy, and cases with 3 deviations had 55% accuracy. Our work is an important step in failure characterization of complex geometrical parts and serves as a stepping stone to enact failure recovery.

Our work builds largely on Nagasaka's stabilizer in multi-contact motion [1]. Using a sequence of contact stances from an offline multi-contact planner, we use first a Model Predictive Controller to generate a dynamic trajectory of the center of mass, then a whole-body closed-loop model-based controller to track it at best. Relatively to Nagasaka's work, we allow frame changes of the preferred force, provide a heuristic to compute the timing of the transition from purely geometrical features and investigate the synchronization problem between the reduced-model preview control and the whole-body controller. Using our framework, we generate a wide range of 3D motions, while accounting for predictable external forces, which includes transporting objects. Simulation scenarios are presented and obtained results are analyzed and discussed.

This paper describes the identification of HRP-2 foot's dynamics. It is expected that a humanoid robot will work in the same environment as man. For that purpose, safety of operation is important. Although a simulator is used for confirming safe conditions of operation, an error may arise in a dynamic parameter by the robot of a simulator, and an actual robot. In this paper, it identified about the viscoelasticity of the sole bush for impact absorption at the time of the walk of humanoid robot HRP-2. We used some simple active motions and composed these motions. We identified parameter using composed motions. We finally compare the identified parameters using the experimental results and simulator results.

Abstract We present a non‐rigid surface registration technique that can align surfaces with sizes and shapes that are different from each other, while avoiding mesh distortions during deformation. The registration is constrained locally as conformal as possible such that the angles of triangle meshes are preserved, yet local scales are allowed to change. Based on our conformal registration technique, we devise an automatic registration and interactive registration technique, which can reduce user interventions during template fitting. We demonstrate the versatility of our technique on a wide range of surfaces.

This paper presents a basic study on feasibility of usage of humanoid robots as an evaluator of assistive devices, by taking advantage of its anthropomorphic shape. In this new application humanoid are expected to help evaluation through quantitative measures, which is difficult with human subjects, and also to reduce the burden coming from ethical concerns with costly tests by human subjects. Taking a passive supportive wear “Smart Suit Lite” designed to relieve the load at lower back as an example, we have conducted pilot experiments by using the humanoid robot HRP-4C. The motion to be performed by the humanoid is obtained through retargeting technique from measured human lifting motion. The supportive effect is first estimated by simulation taking into account the mechanism of the supportive device. The experimentation of humanoid hardware brought us encouraging results on the basic feasibility of this application, as we observed a clear decrease of the torque for lifting when wearing the device as expected by the simulation.

In this paper, we present a tactile approach to grasp large and unknown objects, which can not be easily manipulated with a single end-effector or two-handed grasps, with the whole upper body of a humanoid robot. Instead of conventional joint level force sensing, we equip the robot with various patches of HEX-o-SKIN - a self-organizing, multi-modal cellular artificial skin. Low-level controllers, one allocated to each sensor cell, utilize a self-explored inverted jacobian-like sensory-motor map to directly transfer tactile stimulation into reactive arm motions, altering basic grasping trajectories to the need of the current object. A high-level state machine guides those low-level controllers during the different states of the grasping action. Desired contact points, and key poses for the trajectory generation, are taught through forceless tactile stimulation. First experiments on a position controlled robot, an HRP-2 humanoid, demonstrate the feasibility of our approach. Our paper contributes to the first realization of a self-organizing tactile sensor-behavior mapping on a full-sized humanoid robot, which enables: 1) a new general approach for grasping unknown objects with the whole-body; and 2) a novel way of teaching behaviors using pre-contact tactile sensing.

This paper presents a reaching motion planning and execution framework tailored for exploration missions by human-operated humanoid robots in hazardous environments such as nuclear plants. This framework offers low-level but practical autonomy that allows the robot to plan and execute simple tasks, such as reaching a target object, within a reasonable amount of time. The human operator benefits from the efficiency of the framework to maneuver the robot without waiting for the planning results for minutes. The efficiency improvement is achieved in the following two phases. In the first phase, a reaching motion is planned quickly through approximation of mass distribution and kinematic structure to apply analytical solutions of inverse kinematics. Supposing that the robot is working in environments not completely known, the proposed planner can use measured voxel maps. In the second phase, the planned path is executed while compensating the approximation error in real time without violating other constraints. We confirm through simulations that a reaching motion for the HRP-2 humanoid with 30 DOFs in a constrained environment with pipes is planned in around one second. The simulation results also validate the efficiency of execution with real-time error compensation.

Snap assembly automation remains a challenging task. While progress is being made in localization of parts, force controllers, and control strategies, little work has been done to help the robot reason about its current state, such that if necessary, the robot can assume corrective actions to accomplish the task. Error prone situations caused by the unexpected motion of parts, localization errors, jamming or wedging, cannot be solved by force controllers alone. For this reason we propose a snap assemblies verification system for cantilever-snap fasteners. The verification works in concert with a control strategy that makes use of constraint designs embedded in the snap parts' physical design. The constrained assembly motion generates similar sensory-signal patterns across trials that facilitates force signal discrimination into higher level abstractions of intuitive behavior. This work's contribution is the design of a hierarchical taxonomy for cantilever-snap verification based on increasingly abstract layers that encode relative-change in the task's force signatures. A five-layered taxonomy is built on the concept that relative-change patterns can be classified through a small category set and aided by contextual information. The verification system yielded human apropos intuitive categorizations of task behavior for every state and effectively determined the assembly result. This simple yet effective approach will be expanded to perform probabilistic online system verification to aid in fault tolerance and the automation of cantilever-based snap assemblies.

In this paper, we propose a novel and coherent framework for fast footstep planning for legged robots on a flat ground with 3-D obstacle avoidance. We use swept volume approximations that are computed offline in order to considerably reduce the time spent in collision checking during the online planning phase, in which a rapidly exploring random tree variant is used to find collision-free sequences of half-steps (which are produced by a specific walking pattern generator). Then, an original homotopy is used to smooth the sequences into natural motions, gently avoiding the obstacles. The results are experimentally validated on the robot HRP-2.

We present a new biped walking pattern generator based on "half-steps". Its key features are a) a 3-dimensional parametrization of the input space, and b) a simple homotopy that efficiently smooths the walking trajectory corresponding to a fixed sequence of steps. We show how these features can be ideally combined in the framework of sampling-based footstep planning. We apply our approach to the robot HRP-2 and are able to quickly produce smooth and dynamically stable trajectories that are solutions to a difficult problem of footstep planning.

We present a reactive method for online robot motion replanning in dynamically changing environments by combining path replanning and deformation. Path deformation is newly integrated in our replanning method featured by efficient roadmap reuse and parallel planning and execution. This enhancement allows the planner to deal with more dynamic environments including continuously moving obstacles, by smoothly deforming the path during execution. Simulation results are shown to validate the effectiveness of the proposed method.

In this paper we demonstrate an original equivalence between footstep planning problems, where discrete sequences of steps are searched for, and the more classical problem of motion planning for a 2D rigid shape, where a continuous collision-free path has to be found. This equivalence enables a lot of classical motion planning techniques (such as PRM, RRT, etc.) to be applied almost effortlessly to the specific problem of footstep planning for a humanoid robot.

We present here an original approach to test the feasibility of footsteps for a given walking pattern generator. It is based on a new approximation algorithm intended to cope with this specific problem. The result obtained is used on the robot HRP-2, and enables it to guess a step feasibility 40,000 times faster (in 9μs) than with the normal verification process. As a consequence some advance is made towards fast online motion (re)planning based on a continuous set of possible steps.

This paper introduces a visual servoing scheme for humanoid walking. Though most of the existing approaches follow a perception-decision-action scheme, we close the loop so that the control is robust to model error. Our approach is based on a new reactive pattern generator which modifies, at the control level, the footsteps, the center of mass and the center of pressure trajectories for the center of mass to track a reference velocity. And, in this paper, the reference velocity is directly given by a visual servoing control law. Since, the HRP-2 walk induces a sway motion that disturbs the regulation of the visual control law, we introduce a control law allowing convergence in the image space and taking into account this sway motion.

This paper proposes a reactive leg motion generation method which integrates geometric constraints into its generation process. In order to react given instructions instantaneously or to keep balance against external disturbances, feasible steps must be generated automatically in real-time for safety. In many cases this feasibility has been realized by using predefined steps or admissible stepping regions. However, these predefinitions are often too conservative or valid only in limited situations. The proposed method considers geometric constraints in addition to joint limits during its generation process and it can utilize the ability of the robot to a maximum extent. It can generate feasible walking pattern in real-time by modifying the swing leg motion and the next landing position at each control cycle. The proposed method is validated by experiments using a humanoid robot HRP-2.

This paper proposes a new computation method to solve semi-infinite optimization problems for motion planning of robotic systems. Usually, this problem is solved by means of time-grid discretization of the continuous constraints. Unfortunately, discretization may lead to unsafe motions since there is no guarantee of constraint satisfaction between time samples. First, we show that constraints such as joint position and velocity do not need time-discretization to be checked. Then, we present the computation method based on Taylor polynomials to evaluate more complex constraints over time-intervals. This method also applies to continuous equality constraints, to continuous maximum derivative constraint, and to compute the cost function.

This paper proposes a new framework to recover balance against external forces by combining disturbance suppression and reactive stepping. In the view point of the feedback control, a reactive step can help to diminish the disturbance caused by an external force that should be compensated to maintain balance. In other words, if the adequate step is performed, the feedback controller does not have to compensate all of the external force by itself. Under this concept, we propose an original solution to distribute the compensation between a feedback controller and a reactive step, according to the period of support phase and a disturbance characteristic. We first clearly distinguish between the role of the disturbance suppression and the reactive stepping. Then, based on this distinction, the small disturbance of external force or happening late during the single-support phase, is mainly suppressed by state feedback. The large disturbance which is out of capability by feedback controller and at the beginning of the single-support phase, is absorbed by modifying reactively the next steps. The proposed method is validated through experimental results with the HRP-2 humanoid robot.

We address practical issues to develop reactive motion planning method capable of replanning the path online when the environment changes during the execution. By introducing planning and execution threads running in parallel, the robot can keep moving even during the replanning process as long as the motion is safe. The proposed method can be applied to discontinuous input of environmental changes that may occur due to incomplete perception. We then address a roadmap reuse method for efficient replanning to make use of the increasing knowledge about the environment, by introducing working and learning roadmaps. A general interface with robot motion controller is also defined so that the method can be applied to various types of robots. The proposed method is validated through planning simulations with moving obstacles.

This paper proposes a unified optimization framework to solve the time-parameterization problem of humanoid-robot paths. Even though the time-parameterization problem is well known in robotics, the application to humanoid robots has not been addressed. This is because of the complexity of the kinematical structure as well as the dynamical motion equation. The main contribution of this paper is to show that the time parameterization of a statically stablepathto be transformed into a dynamically stabletrajectorywithin the humanoid-robot capacities can be expressed as an optimization problem. Furthermore, we propose an efficient method to solve the obtained optimization problem. The proposed method has been successfully validated on the humanoid robot HRP-2 by conducting several experiments. These results have revealed the effectiveness and the robustness of the proposed method.

We present a novel method for prioritizing both linear equality and inequality systems and provide one algorithm for its resolution. This algorithm can be summarized as a sequence of optimal resolutions for each linear system following their priority order. We propose an optimality criterion that is adapted to linear inequality systems and characterize the resulting optimal sets at every priority level. We have successfully applied our method to plan local motions for the humanoid robot HPR-2. We will demonstrate the validity of the method using an original scenario where linear inequality constraints are solved at lower priority than equality constraints.

A motion planning method for humanoid robots to pass under obstacles is proposed. The proposed motion planner can calculate a goal configuration and connect it with an initial configuration in a collision-free dynamically stable motion. The method can generate not only a body motion but also the footstep sequence. The effectiveness of the proposed method was validated by experiments with the humanoid robot HRP-2.

A method of regrasp planning for humanoid robot manipulation is proposed. We adopt pivoting manipulation for the humanoid robot to move a bulky object without lifting in a stable and dexterous manner. In order to carry the object to a desired place, the humanoid should sometimes move through narrow areas surrounded by obstacles. We propose a roadmap multiplexing planning to allow the robot to leave the object near narrow places and to regrasp it from another position to continue carrying. We utilize visibility probabilistic roadmap (PRM) method as a preprocessing to capture the critical configurations for regrasping. Then a diffusion method is employed to plan the overall manipulation path including regrasping. The proposed method is verified through planning simulation including whole-body motions.

Abstract This paper deals with whole‐body motion planning and dynamic control for humanoid from two aspects: locomotion including manipulation and reaching. In the first part, we address a problem of simultaneous locomotion and manipulation planning that combines a geometric and kinematic motion planner with a dynamic humanoid motion generator. The second part deals with whole‐body reaching tasks by using a generalized inverse kinematics (IK) method to fully exploit the high redundancy of the humanoid robot. Through experiments using humanoid platform HRP‐2 No. 14 installed at LAAS‐CNRS, we first verify the validity of each method. An integrated experiment is then presented that unifies the both results via visual perception to execute an object‐fetching task. Copyright © 2009 John Wiley & Sons, Ltd.

In this paper, the imitation of human captured motions by a humanoid robot is considered. The main objective is to reproduce an imitated motion which should be as close as possible to the original human captured motion. To achieve this goal, the imitation problem is formulated as an optimization problem and the physical limits of the humanoid robot are considered as constraints. The optimization problem is then solved recursively by using an efficient dynamics algorithm, which allows the calculation of the gradient function with respect to the control parameters analytically. The simulation results using OpenHRP platform, which is a dynamical simulator for humanoid robot motions, have pointed out that the imitated motions preserve the salient characteristics of the original human captured motion. Moreover the optimization procedure converges well thanks to the analytical calculation of the gradient function.

This paper emphasizes on the capacity of a humanoid robot to perform tasks that are difficult for other types of robots. It deals with manipulation of bulky objects. Such tasks require complicated manipulations involving the whole-body and fine coordination between legs, arms and torso motions. We introduce here a whole-body motion planner that allows a humanoid robot to autonomously plan a pivoting strategy that accounts for the various constraints: collision avoidance, legs-arms coordination and stability control. Based on a previous result by the authors [1] proving the small-time controllability of a pivoting system, the planner is proven to inherit from the probabilistic completeness of the samplingbased motion planning method it is built on. The geometric and kinematic capacity of the proposed planner is mainly demonstrated through simulations and experiments.

This paper proposes an whole body motion planning method for humanoid robots in which dynamics is integrated. The method consists of two stages. A collision-free and statically stable path is planned in the first stage and it is transformed into a dynamically stable trajectory in the second stage. Contributions of the method is summarized as follows. (1) A local method plans a C1path while avoiding collisions between non-strictly convex objects. (2) The second stage gives the minimum time trajectory by time parameterization under dynamic balance constraints. (3) Any path reshaping for recovering collision-freeness is not required since the second stage doesnpsilat change shape of the path. Effectiveness of the method is examined by applying it to scenarios of a humanoid robot HRP-2.

We propose a general and practical planning framework for generating 3-D collision-free motions that take complex robot dynamics into account. The framework consists of two stages that are applied iteratively. In the first stage, a collision-free path is obtained through efficient geometric and kinematic sampling-based motion planning. In the second stage, the path is transformed into dynamically executable robot trajectories by dedicated dynamic motion generators. In the proposed iterative method, those dynamic trajectories are sent back again to the first stage to check for collisions. Depending on the application, temporal or spatial reshaping methods are used to treat detected collisions. Temporal reshaping adjusts the velocity, whereas spatial reshaping deforms the path itself. We demonstrate the effectiveness of the proposed method through examples of a space manipulator with highly nonlinear dynamics and a humanoid robot executing dynamic manipulation and locomotion at the same time.

The current research analyses and demonstrates how spoken language can be used by human users to communicate with the HRP-2 humanoid to program the robot's behavior in a cooperative task. The task involves the humans and the HRP-2 working together to assemble a piece of furniture. The objectives of the system are to 1) Allow the human to impart knowledge of how to accomplish a cooperative task to the robot, i.e. to program the robot, in the form of a sensory-motor action plan. 2) To do this in a semi-natural and real-time manner using spoken language. In this framework, a system for spoken language programming (SLP) is presented, and experimental results are presented from this prototype system. In Experiment 1, the human programs the robot to assist in assembling a small table. In Experiment 2, the generalization of the system is demonstrated as the user programs the robot to assist in taking the table apart. The SLP is evaluated in terms of the changes in efficiency as revealed by task completion time and number of command operations required to accomplish the tasks with and without SLP. Lessons learned are discussed, along with plans for improving the system, including developing a richer base of robot action and perception predicates that will allow the use of richer language. We thus demonstrate - for the first time - the capability for a human user to tell a humanoid what to do in a cooperative task so that in real time, the robot performs the task, and acquires new skills that significantly facilitate the cooperative human-robot interaction.

In this paper, we plan the collision free motion for walking pattern generation of a humanoid robot. Our motion planner can take into account several features of the walking pattern generator. We first run the walking pattern generator by considering the contact wrench applied to the robot and monitor the collision among the links and the environments. Then, we plan the collision free motion for the period of time causing the collision. In our motion planner, we can consider the constraint condition which are the functions of time. Also, for keeping balance of the robot, we plan the motion with keeping the horizontal position of the COG as well as the position/orientation of the feet/hand. The effectiveness of the proposed method is confirmed by simulation and experiment.

Pivoting manipulation has such advantages as dexterity and safety over other methods to move bulky or heavy objects. In this paper we aim to show that a polyhedral object can be displaced to arbitrary position and orientation on a plane (i.e. such a pivoting system is controllable). More than that we show it is small time controllable, i.e. the reachable space from a starting point contains always a neighbor no matter how cluttered the environment is. As a consequence of this analysis, we propose a steering method to plan a manipulation path to be performed by a humanoid robot: first we use a classical nonholonomic path planner that accounts for the robot motion constraints, and then we transform that path into a sequence of pivoting operations. While the feasibility of elementary pivoting tasks has been already experienced by the humanoid robot HRP-2, we present here the very first simulations of the plans generated by our steering method.

In this paper we address smooth and collision-free whole-body motion planning for humanoid robots. A two-stage iterative planning framework is introduced where geometric motion planner and dynamic pattern generator interacts by exchanging the trajectory, to obtain 3D whole-body dynamic motions simultaneous tasks including locomotion, in complex environments. We propose a practical method for smooth motion reshaping to avoid collisions in generated dynamic motion. Based on motion editing techniques in computer graphics animation, smooth collision-avoiding motion is generated through trajectory deformation. The validity of the proposed reshaping method is verified by computer simulations and experiments using humanoid platform HRP-2

Pivoting manipulation can be an alternative to pushing operation when the floor is not flat enough, or when the object to manipulate is too heavy. The technique of pivoting is used by humans to move large and bulky furniture from one place to another. The decomposition of the task of pivoting has already been studied, in particular with the use of two fingers of a robotic arm. This work intends to apply the technique of pivoting using an humanoid platform. The robot should be able to pivot the object and to walk with it to displace it to a specific remote location. The research achievements proposed here take place in a more long term objective aimed at improving the dexterity and the autonomy of humanoid robots. As a matter of fact, such robots should be able to handle objects and move around in the environment in an autonomous way. This paper describes the algorithm designed to perform the displacement of a large object using the pivoting technique. It also presents the results of the dynamic simulation and the results of the real hardware experiment of the HRP-2 platform performing the task.

This paper addresses a multi-level exploitation of degree of freedom (DOF), for humanoid motion planning based on a randomized method. The improvement of autonomy and mobility is required so that humanoid robot can perform tasks in various environments. Although a humanoid robot has many DOFs, all of them do not have to be controlled depending on the situation and the required tasks. Utilizing rapidly-exploring random trees (RRTs) as an efficient planning tool, a method of multi-level DOF exploitation is developed that adjusts the controlled DOFs according to the detected environmental situation. That allows the planner to deal with only the necessary sets of DOFs instead of exploring the search space of all the DOFs, which leads to efficient humanoid motion planning. The proposed method is verified through simulations using software platform OpenHRP and HRP-2 humanoid robot model.

A modular robot has a distributed mechanical composition which can make various configurations and also make locomotion in a wide variety of configurations. Modular robots are thought to be useful in extreme or unknown environments by adaptively changing their shape and locomotion patterns. As for locomotion, two types can be used; one is whole-body fixed-configuration locomotion and the other is locomotion by self-reconfiguration. In this paper we deal with the former type of locomotion which is realized by coordinated joint actuation. So far, proposed control methods for whole-body locomotion by modular robots have been based on predefined locomotion sequences. However, locomotion based on predefined sequences cannot adapt to changing terrain conditions such as uphill, downhill, slippery and sticky grounds. To solve such problems, we propose a distributed control mechanism using a CPG controller which enables adaptive locomotion by modular robots. Besides the real-time CPG control we introduce a decentralized control mechanism for detecting the situation that the robot is stuck and initiating transformation to another shape for recovering the situation. The results of various hardware experiments by 4-legged structure prove the feasibility of the method for adaptive locomotion and transformation by our M-TRAN II modules.

A method for behavior planning is presented for a modular robot that applies a randomized planner to coherent structure of the robot. To cope with difficulty in planning of many degrees of freedom (DOFs) of a modular robot, coherent structure is introduced in terms of control system and robot configuration. As the control system, a simple phase synchronization mechanism is introduced that can control the robot with many DOFs with reduced number of parameters. Together with symmetrical configuration, this control system generates various dynamic motions. In order to plan the behaviors of the modular robot determined by the parameters of the control system, we adopt a randomized planner called rapidly exploring random trees (RRTs). This can benefit from a number of advantages of RRT planner, including simple implementation, uniform search, and applicability to a dynamic system with differential constraints. By exploring parameter space of the coherent control system, behaviors including dynamic motions can be planned. We have applied the proposed planner to M-TRAN modular robot to demonstrate the effectiveness of the proposed method through preliminary simulation results.

Locomotion, one of the most basic robotic functions, has been widely studied for several types of robots. As for self-reconfigurable modular robots, there are two types of locomotion; one type is realized as a series of self-reconfiguration and the other is realized as a whole body motion such as walking and crawling. Even for the latter type of locomotion, designing control method is more difficult than ordinary robots. This is because the module configuration includes many degrees of freedom and there are a wide variety of possible configurations. We propose an offline method to generate a locomotion pattern automatically for a modular robot in an arbitrary module configuration, which utilizes a neural oscillator as a controller of the joint motor and evolutionary computation method for optimization of the neural oscillator network, which determines the performance of locomotion. We confirm the validity of the method by software simulation and hardware experiments.

We have been developing a self-reconfigurable modular robotic system (M-TRAN) which can make various 3-D configurations and motions. In the second prototype (M-TRAN II), various improvements are integrated in order to realize complicated reconfigurations and versatile whole body motions. Those are a reliable connection/detachment mechanism, on-board multi-computers, high speed inter-module communication system, low power consumption, precise motor control, etc. Programing environments are also integrated to design self-reconfiguration processes, to verify motions in dynamics simulation, and to realize distributed control on the hardware. Hardware design, developed software and experiments are presented in this paper.

We have developed a modular robotic system (M-TRAN), which can change its configuration by itself. By using many DOFs of mechanism and self-reconfiguration capability, it can realize several types of motion, and can make various configurations. We have made two models of the system (M-TRAN I & II). In M-TRAN II, various improvements are integrated such as onboard multi-computers, reliable inter-module communication system, low power consumption, precise motor control, etc. Its hardware design, basic experiments and examples of motion are presented in this paper.

We have proposed a self-reconfigurable robotic module, which has a very simple structure. The system is capable of not only building a static structure, but also generating a dynamic robotic motion. We have also developed a simulator for the motion planning. In this paper, we present details of the mechanical and electrical designs of the developed module and its control system architecture. Experiments using ten modules demonstrate the robotic configuration change, crawling locomotion and three types of quadruped locomotion.

This paper addresses motion planning of a homogeneous modular robotic system. The modules have self-reconfiguration capability so that a group of the modules can construct a robotic structure. Motion planning for self-reconfiguration is a kind of computationally difficult problem because of many combinatorial possibilities of modular configuration and the restricted degrees of freedom of the module; only two rotation axes per module. We will show a motion planning method for a class of multimodule structures. It is based on global planning and local motion scheme selection that is effective to solve the complicated planning problem.

In this paper we describe the hardware design of a novel self-reconfigurable robotic system. We have classified previous studies on self-reconfigurable robotic systems into "lattice type" composed of spatially symmetric modules and "string type" like snake robots. The proposed system has both the advantages of simple operation of self-reconfiguration of the former and motion generation ability of the latter. Its simple structure and reliable operation allows us to construct large 3D self-reconfigurable structure which functions as a robotic system such as a legged walking machine. We have examined its basic mechanical functions and verified its reliable operation of self-reconfiguration.

Presents a miniaturized self-reconfigurable modular robotics system using shape memory alloy (SMA). The system is designed so that various shapes can be actively formed by a group of identical mechanical units. The unit realizes rotational motion by using an actuator mechanism composed of two SMA torsion coil springs which generate sufficient motion range and torque for reconfiguration. The fundamental functions of the system are tested by experiments. Applicability of the developed unit model to a 3-D self-reconfigurable system is also discussed.

We propose a self-assembly and self-repair method for a homogeneous distributed mechanical system. We focus on a category of distributed systems composed of numbers of identical units which can dynamically change connections among themselves. Each unit has an onboard microprocessor, and local communication between neighboring units is possible. We discuss a distributed method for a group of such units to metamorphose from an arbitrary configuration into a desired configuration through cooperation by the units. This process, called self-assembly, is realized by identical software on each unit with local inter-unit communication. An extension of self-assembly, self-repair, is also examined. In this process, an occasional cut-off of an arbitrary part of the system is assumed. When some part of the system detects damage, the whole system degenerates and reconstructs itself. Computer simulations show the feasibility of self-assembly and self-repair.

A three-dimensional, self-reconfigurable structure is proposed. The structure is a fully distributed system composed of many identical 3-D units. Each unit has functions of changing local connection, information processing, and communication among neighborhood units. Groups of units cooperate to change their connection so that the shape of the whole solid structure transforms into an arbitrary shape. Also, the structure can repair itself by rejecting faulty units, replacing them with spare units. This kind of self-maintainability is essential to structure's longevity in hazardous or remote environments such as space or deep sea where human operators cannot approach. We have designed and built a prototype unit to examine the feasibility of the 3-D self-reconfigurable concept. The design of the unit, method of reconfiguration, hardware implementation, and results of preliminary experiments are shown. In the last part of the paper, distributed software for self-reconfiguration is discussed.

A three-dimensional self-reconfigurable structure made of identical units is proposed. Each unit has six arms on the surface of its base cube which can connect to neighboring units mechanically. By the connection, cubic lattice structure is formed. A unit can carry its neighbor unit from one node of the lattice to another by rotating its arm by 90 degrees. Repeating this movement, the structure can reconfigure itself to realize various 3D structures. General process of reconfiguration were proposed for this system. Four units were made and basic motions of self-reconfiguration were verified.

A distributed reconfiguration method is proposed for a 3D reconfigurable machine, composed of many identical mechanical units. The method aims to enable the machine to transform itself into desired structure from an arbitrary initial configuration. The proposed method is implemented in such a way that each unit has identical software, so that any unit can play any role in the system. It is also featured by a stochastic relaxation process, which allows the system to converge to a given target structure by searching for a proper unit motion over many degrees of freedom. Furthermore, the method is extended for the structure to reconfigure itself dynamically according to the environment. The effectiveness of the method is confirmed by computer simulations.

A distributed formation control method is proposed for a modular mechanical system. We have developed a totally decentralized system composed of many homogeneous mechanical units which are designed to change their connective configuration using only local information. The control method proposed in this paper enables the systems to re-organize themselves so that various configurations can be formed in a robust way. Computer simulations and experiments are carried out to show its effectiveness.

This paper presents a novel design methodology of local communication system for cooperation in distributed mobile robot systems. Our goal is to design a local communication system so as to transmit task information to necessary robots in minimum time without excessive propagation. In this paper, we propose a layered methodology, i.e. design from spatial and temporal aspects based on analysis of information diffusion by local communication between robots. The spatial design gives the optimal communication area minimizing transmission time for various cooperative tasks. In the temporal design, we derive the information announcing time to prevent excessive information diffusion. Finally, the simulations and experiments demonstrate that the design methodology is effective in constructing an efficient local communication system.

In this paper, we propose an off-line planning algorithm for cooperative tasks of multiple mobile robots. Sweeping means a motion that a robot covers a 2-dimensional area by its effector. Sweeping of a whole work area is fundamental and essential task of mobile robots. For efficient cooperation, setting appropriate burden onto each robot is very important because interference of robots and overlaps of their effecters make efficiency low. The cost of sweeping depends on both sweeping ability of a robot and a shape of a work area to be swept. Evaluation and distribution of the cost are most important issues. In the proposed algorithm, the cost is evaluated by means of length on which robot should move. We introduce both edges of the configuration space and Voronoi diagram so as to compute paths in the whole area. We generate the a tour for traversing all the paths by applying the algorithm of the Chinese Postman Problem. According to the cost evaluation, appropriate paths of the tour are assigned to each robot. The efficiency of the proposed algorithm is verified by simulations and an experiment.

One of the major emerging problems of distributed mobile robotic system is what kind of inter-robot communication to use, because of increasing robots integrated in the system. This paper aims to give an analytical view of this issue. The efficiency of local and global communication is compared based on the analysis of information transmission time to multiple robots. We will show that local communication is effective in environments where cooperative tasks are executed by multiple mobile robots in distributed fashion.

The authors aim at grasping and handling a large object by cooperating multiple mobile robots with various sizes, moving cost, and load capacity. In this paper the authors focus on the problems of deciding an appropriate grasping arrangement before handling operations. The authors propose this method to avoid bad situations, such as making an object fall down, or some robots overloaded in grasping and handling. The algorithm proposed includes two optimization problems; one is the "decision of initial robot arrangement," the other is the "decision of final robot arrangement." The difference between these optimizations is that the mass center of the object is recognized or not. The penalty index is defined to minimize the energy the system consumes and to maximize an index of stability. The authors have confirmed that the robots moved to the optimal arrangements in computer simulations and experiments.

When many mobile robots should achieve cooperation, a local communication system is considered appropriate from the standpoint of the cost and capacity of communication. This paper presents the optimization of the efficiency of local communication in environments where many mobile robots send out information stochastically. The optimal communication area is derived by minimizing the transmission waiting time calculated using the probability of successful information transmission. Computer simulations have been undertaken to verify the analytical results.

Proposes an algorithm of dividing a work area into small pieces in order to make multiple mobile robots cooperate efficiently. Searching of a whole work area is the most fundamental and the most essential task of mobile robots. The authors expect that the cost of searching is shared by cooperation of multiple robots. In the proposed algorithm, searching motions of robots are represented by paths. Both edges of the configuration space and the Voronoi diagram are introduced so as to compute paths in the whole area. The authors generate a tour of the paths to traverse all the paths using the algorithm of the Chinese postman problem. The cost is estimated as the total length of the paths. By means of the cost evaluation, the appropriate paths are assigned to each robot. The efficiency of the proposed algorithm is verified by simulations.

This paper deals with a motion planning of mobile robots during transferring a large object cooperatively by a group of multiple mobile robots (a robot group). This problem has the following characteristics: 1) a real time planning is essential because mobile robots move in an open and dynamic area in comparison to articulated robots; 2) both the object and the robot group need to avoid collision against obstacles; and 3) the object needs to be grasped stably by the robot group. The authors propose the following approach to solve this complicated problem: 1) regrasping strategy of the object is introduced in order avoid obstacles and to transfer the object stably; and 2) the motion planning is divided into two steps: the object and that of the robot group. The former step is accomplished by extending the virtual impedance method, which was proposed by authors. The latter step is realized by using nonlinear programming method, that optimizes a penalty index indicating the performance for obstacle avoidance and for stable grasping. Effectiveness of the proposed method is verified by a simulation of transferring a large circle-shaped object with three robots.

Local communication system is considered appropriate when many mobile robots should achieve cooperation, from the standpoint of cost and capacity of communication. This paper presents a design method of the optimal communication range for efficient local communication system. The analyses of optimization are made by minimizing the information transition time, first in the case of transmission to an arbitrary robot, and next, to multiple robots. Computer simulations have been undertaken to verify the analytical results.

For the cooperation in a large system with many mobile robots, local communication system is considered appropriate in terms of the cost and capacity of communication. The behavior of robots has a respectable effect on the efficiency of communication in such a local communication system and it is essential to know what kind of behavior robots should take to realize efficient information transmission for cooperative tasks. We introduce a simple group behavior for the purpose of improving the communication efficiency. This paper analyses the effect of group behavior on the communication performance, and derives differential equations describing information diffusion among robot groups. The optimal group size to transmit information to a desired number of robots is obtained front these equations. The effectiveness of the analysis is verified by computer simulation. We also show a self-organization algorithm for group forming designed for local communication system.>

This paper proposes a new algorithm for the manipulation of a rigid object by cooperative control of multiple flexible manipulators. Assuming that the flexibility exists only at the force sensor attached to the tip of the manipulator, each manipulator is modeled as a rigid manipulator with six springs at the tip. A cooperative control algorithm considering the flexibility is proposed. After the differential equation of the error vector of the system is derived, the method of determining the feedback gain matrices is shown for linearizing the system. To verify the efficiency of the algorithm, the experiments are done on carrying an object held by two parallel flexible beams.