Learning the inverse dynamics of robots directly from data, adopting a black-box approach, is interesting for several real-world scenarios where limited knowledge about the system is available. In this article, we propose a black-box model based on Gaussian process (GP) regression for the identification of the inverse dynamics of robotic manipulators. The proposed model relies on a novel multidimensional kernel, calledLagrangian Inspired Polynomial(LIP) kernel. The LIP kernel is based on two main ideas. First, instead of directly modeling the inverse dynamics components, we model as GPs the kinetic and potential energy of the system. The GP prior on the inverse dynamics components is derived from those on the energies by applying the properties of GPs under linear operators. Second, as regards the energy prior definition, we prove a polynomial structure of the kinetic and potential energy, and we derive a polynomial kernel that encodes this property. As a consequence, the proposed model allows also to estimate the kinetic and potential energy without requiring any label on these quantities. Results on simulation and on two real robotic manipulators, namely a 7 DOF Franka Emika Panda, and a 6 DOF MELFA RV4FL, show that the proposed model outperforms state-of-the-art black-box estimators based both on Gaussian processes and neural networks in terms of accuracy, generality, and data efficiency. The experiments on the MELFA robot also demonstrate that our approach achieves performance comparable to fine-tuned model-based estimators, despite requiring less prior information. The code of the proposed model is publicly available.

In this paper, we investigate the use of ModelBased Reinforcement Learning (MBRL) for pick-and-throw tasks, which is a variation of the pick-and-place task. In pick-and-throw applications, objects are thrown instead of placed to increase time efficiency and enlarge the workspace reachable by the robot. The effectiveness of analytical methods for this task is limited by the complex system dynamics and the disturbances due to communication and control. Recently, the application of data-driven solutions, in particular Model-Free Reinforcement Learning (MFRL), has attracted attention. In this paper, we explore the use of a MBRL algorithm to solve the pick-and-throw task. Experiments carried out in simulation show the potentialities of our approach, in terms of accuracy and data efficiency, both with respect to analytical methods and a MFRL approach.

In this article, we present a model-based reinforcement learning (MBRL) algorithm namedMonte Carlo probabilistic inference for learning control(MC-PILCO). This algorithm relies on Gaussian processes (GPs) to model the system dynamics and on a Monte Carlo approach to estimate the policy gradient. This defines a framework in which we ablate the choice of the components, which are the selection of the cost function, the optimization of policies using dropout, and an improved data efficiency through the use of structured kernels in the GP models. The combination of the aforementioned aspects affects dramatically the performance of MC-PILCO. Numerical comparisons in a simulated cart–pole environment show that MC-PILCO exhibits better data efficiency and control performance w.r.t. state-of-the-art GP-based MBRL algorithms. Finally, we apply MC-PILCO to real systems, considering, in particular, systems with partially measurable states. We discuss the importance of modeling both the measurement system and the state estimators during policy optimization. The effectiveness of the proposed solutions has been tested in simulation and on two real systems, which are the Furuta pendulum and the ball-and-plate rig.

Robotics systems are becoming more and more autonomous and reconfigurable. In this context, the design of algorithms capable of deriving kinematics and dynamics models directly from data could be particularly useful. In this article, we present an algorithm that learns a forward kinematics model of a robot starting from a time series of visual observations. Our strategy can be applied to any robot with serial kinematics composed of revolute and prismatics joints. First, the algorithm identifies the robot kinematic structure, i.e., a high-level description of the robot geometry that defines the connections between the rigid-bodies composing the robot. Then, the algorithm derives the forward kinematics relying on a Gaussian process (GP) model. More precisely, the GP model is based on a polynomial kernel, defined exploiting the kinematic structure previously identified. The effectiveness of the proposed solution has been tested via extensive Monte Carlo simulations, as well as via experiments on a real UR10 robot.

This paper presents a problem of model learning for the purpose of learning how to navigate a ball to a goal state in a circular maze environment with two degrees of freedom. The motion of the ball in the maze environment is influenced by several non-linear effects such as dry friction and contacts, which are difficult to model physically. We propose a semiparametric model to estimate the motion dynamics of the ball based on Gaussian Process Regression equipped with basis functions obtained from physics first principles. The accuracy of this semiparametric model is shown not only in estimation but also in prediction at n-steps ahead and its compared with standard algorithms for model learning. The learned model is then used in a trajectory optimization algorithm to compute ball trajectories. We propose the system presented in the paper as a benchmark problem for reinforcement and robot learning, for its interesting and challenging dynamics and its relative ease of reproducibility.