General purpose simulators provide cheap training data to learn complex robotic skills. However, the transition from simulation to reality is often very challenging for the agent. One major issue is the delay on the physical robot that may deteriorate the performance of the deployed agent. Furthermore, once a successfully trained learning-based control policy is available, re-purposing the knowledge acquired by the agent to enable a structurally distinct agent to perform the same task is hazardous if done naively. In this work, we address the above issues with a single method, the DA-UNN (Delay Aware Universal Notice Network), which decomposes the knowledge into robot-specific and task-specific modules for fast transfer. Our framework deals with delays immanent to physical systems in order to improve sim2real transfer. We evaluate the efficiency of our approach using simulated and actual robots on a dynamic manipulation task where delay management is crucial.

International audience

International audience

Being able to learn and transfer skills from one agent to another is a fundamental feature in constructing even more intelligent behaviors. In this paper, we introduce a new kind of architecture and information pipeline that aims to enable the transmission of skills from one robot to one or several others. The Universal Notice Network (UNN) originality lies in the fact that it clearly distinguishes knowledge necessary to solve the task from the agent intrinsic perceptions and capabilities, hence increasing its reusability and its potential transmission to other agents. In various experiments, focusing on manipulation and comanipulation tasks in original environments, we demonstrate the capabilities of the proposed method that takes advantage of reinforcement learning algorithms and domain knowledge, such as forward geometric model and inverse kinematics. In particular, we show that a learned UNN through the interactions of an agent with its environment is transmissible to other agents, conserving a similar perfomance level.

We propose three novel methods to evaluate a distance function for robotic motion planning based on semiinfinite programming (SIP) framework; these methods include golden section search (GSS), conservative advancement (CA) and a hybrid of GSS and CA. The distance function can have a positive and a negative value, each of which corresponds to the Euclidean distance and penetration depth, respectively. In our approach, each robot's link is approximated and bounded by a capsule shape, and the distance between some selected link pairs is continuously evaluated along the joint's trajectory, provided by the SIP solver, and the global minimum distance is found. This distance is fed into the SIP solver, which subsequently suggests a new trajectory. This process is iterated until no negative distance is found anywhere in the links of the robot.We have implemented the three distance evaluation methods, and experimentally validated that the proposed methods effectively and accurately find the global minimum distances to generate a self-collision-free motion for the HRP-2 humanoid robot. Moreover, we demonstrate that the hybrid method outperforms other two methods in terms of computational speed and reliability.

This paper introduces effective numerical methods for the planning and fast replanning of safe motions to ensure the safety, balance, and integrity of humanoid robots over the whole motion duration. Our safe methods do not depend on, nor are connected to, any type of modeling or constraints. To plan safe motions, certain constraints have to be satisfied over a continuous interval of time. Classical methods revert to time-grid discretization, which can be risky for the robot. We introduce a hybrid method to plan safe motions, which combines a classical unsafe method with a verification step that checks constraint violation and computes excess by the usage of interval analysis. When the robot meets unexpected situations, it has to replan a new motion, which is often too time consuming. Hence, we introduce a new method to rapidly replan safe motions, i.e., in less than 2 s CPU time. It computes offline feasible subsets in the vicinity of safe motions and finds online a solution in these subsets without actually recomputing the nonlinear constraints. Our methods are validated by the use the HOAP-3 robot, where the motions are run with no balance controller.

This paper proposes a new computation method to solve semi-infinite optimization problems for motion planning of robotic systems. Usually, this problem is solved by means of time-grid discretization of the continuous constraints. Unfortunately, discretization may lead to unsafe motions since there is no guarantee of constraint satisfaction between time samples. First, we show that constraints such as joint position and velocity do not need time-discretization to be checked. Then, we present the computation method based on Taylor polynomials to evaluate more complex constraints over time-intervals. This method also applies to continuous equality constraints, to continuous maximum derivative constraint, and to compute the cost function.

Motion databasing is an important topic in robotics research. Humanoid robots have a large number of degrees of freedom and their motions have to satisfy a set of constraints (balance, maximal joint torque velocity and angle values). Thus motion planning cannot efficiently be done on-line. The computation of optimal motions is performed off-line to create databases that transform the problem of large computation time into a problem of large memory space. Motion planning can be seen as a Semi-Infinite Programming problem (SIP) since it involves a finite number of variables over an infinite set of constraints. Most methods solve the SIP problem by transforming it into a finite programming one using a discretization over a prescribed grid. We show that this approach is risky because it can lead to motions which may violate one or several constraints. Then we introduce our new method for planning safe motions. It uses Interval Analysis techniques in order to achieve a safe discretization of the constraints. We show how to implement this method and use it with state-of-the-art constrained optimization packages. Then, we illustrate its capabilities for planning safe motions dedicated to the HOAP-3 humanoid robot.

Optimal motions are usually used as joint reference trajectories for repetitive or complex motions. In the case of soccer robots, the kicking motion is usually a benchmark motion, computed off-line, without taking into account the current position of the robot or the direction of the goal. Moreover, robots must react quickly to any situation, even if not expected, and cannot spend time to generate a new optimal motion by the classical way. Therefore, we propose a new method for fast motion re-planning based on an off-line computation of a feasible sub-set of the motion parameters, using Interval Analysis.