Dynamic movement primitives (DMPs) have proven to be an effective movement representation for motor skill learning. In this paper, we propose a new approach for training deep neural networks to synthesize dynamic movement primitives. The distinguishing property of our approach is that it can utilize a novel loss function that measures the physical distance between movement trajectories as opposed to measuring the distance between the parameters of DMPs that have no physical meaning. This was made possible by deriving differential equations that can be applied to compute the gradients of the proposed loss function, thus enabling an effective application of backpropagation to optimize the parameters of the underlying deep neural network. While the developed approach is applicable to any neural network architecture, it was evaluated on two different architectures based on encoder-decoder networks and convolutional neural networks. Our results show that the minimization of the proposed loss function leads to better results than when more conventional loss functions are used.

In this paper we propose a new approach for learning perception-action couplings. We show that by collecting a suitable set of raw images and the associated movement trajectories, a deep encoder-decoder network can be trained that takes raw images as input and outputs the corresponding dynamic movement primitives. We propose suitable cost functions for training the network and describe how to calculate their gradients to enable effective training by back-propagation. We tested the proposed approach both on a synthetic dataset and on a widely used MNIST database to generate handwriting movements from raw images of digits. The calculated movements were also applied for digit writing with a real robot.

Encoding desired motions into dynamic movement primitives (DMPs) is a common way for generating compact task representations that are able to handle sensor-based goal adaptations. At the same time, a robot should not only express adaptive motion capabilities at planning level, but use also contact wrench feedback in the adaptation and learning process of the DMP. Despite first approaches exist in this direction, no fully integrated approach has been proposed so far. In this paper, we introduce a new class of admittance-coupled DMPs that addresses environmental changes by including contact wrench feedback dynamics into the DMP formalism. Moreover, a novel iterative learning approach is devised that is based on monitoring the overall system passivity analysis in terms of reference power tracking. Simulations and experimental results with the Kuka LWR robot maintaining a non-rigid contact with the environment (wiping a surface) are shown for supporting the validity of our approach.

Autonomous trajectory generation through generalization requires a database of motion, which can be difficult and time consuming to obtain. In this paper, we propose a method for autonomous expansion of a database for the generation of compliant and accurate motion, achieved through the framework of compliant movement primitives (CMPs). These combine task-specific kinematic and corresponding feed-forward dynamic trajectories. The framework allows for generalization and modulation of dynamic behavior. Inspired by human sensorimotor learning abilities, we propose a novel method that can autonomously learn task-specific torque primitives (TPs) associated to given kinematic trajectories, encoded as dynamic movement primitives. The proposed algorithm is completely autonomous, and can be used to rapidly generate and expand the CMP database. Since CMPs are parameterized, statistical generalization can be used to obtain an initial TP estimate of a new CMP. Thereby, the learning rate of new CMPs can be significantly improved. The evaluation of the proposed approach on a Kuka LWR-4 robot performing a peg-in-hole task shows fast TP acquisition and accurate generalization estimates in real-world scenarios.

Continuous and on-line adaptation of robotic trajectories is one of the key properties of robotic control policies that make robots useful in unstructured environments, where the conditions of the external world are not predefined or stationary. Modification of robotic trajectories through human intervention, in order to make them more suitable to the user, is often termed as robotic coaching. In the manner of a tutor, the user shows to the robot which part of the motion to change and how. Predefined gestures acquired with visual systems and direct physical interaction are two possibilities to realize robotic coaching. In this paper we study what are the desired user features and which of the three tested coaching methods is deemed most favorable by a group of 7 subjects. The three methods we explore all build on on-line coaching of dynamic motion primitives, but are based on different feedback signals, i. e., visual or force feedback, and different low-level robot control approaches, i. e., a stiff or a compliant robot. The experiment required coaching of a KUKA LWR-4 robot arm while wiping a flat surface, so that it followed two simple patterns. The survey presented in this paper aims at providing the designers of human-robot coaching interfaces with answers on the feasibility, advantages and drawbacks of the three methods for on-line coaching analyzed in this paper: 1. visual feedback, 2. force feedback with a stiff robot, and 3. position feedback with a compliant robot.

Human-demonstrated motion transferred to a robotic platform often needs to be adapted to the current state of the environment or to modified task requirements. Adaptation, i. e. learning of a modified behavior, needs to be fast to enable quick utilization of the robot either in industry or in future household-assistant tasks. In this paper we show how to accelerate trajectory adaptation based on learning of coupling terms in the framework of dynamic movement primitives (DMPs). Our method applies ideas from feedback error learning to iterative learning control (ILC). By taking into account the actual physical constraints of the synchronous motion - through synchronization of both positions (or forces) and velocities - it is not only a more faithful representation of actual real-world processes, but it also accelerates the speed of convergence. To show the applicability of the approach in the framework of DMPs, we tested it on a formulation which encodes an initial discrete motion, followed by a periodic behavior, all in a single system. Modifications of the original discrete-periodic formulation now also allow for the use of DMP temporal scaling property. In the paper we also show how the DMP coupling can be implemented in joint space, whereas the measured forces and previous approaches always remained in the task space. We applied our approach to an example dual-arm synchronization task on Sarcos humanoid robot CB-i.

Just as their discrete counterparts, periodic or rhythmic dynamic motion primitives allow easily modulated and robust motion generation, but for periodic tasks. In this paper we present an approach for modulating periodic dynamic movement primitives based on force feedback, allowing for rich motor behavior and skills. We propose and evaluate the combination of feedback and learned feed-forward terms to fully adapt the motions of a robot in order to achieve a desired force interaction with the environment. For the learning we employ the notion of repetitive control, which can effectively minimize the error of behavior towards a given reference. To demonstrate the approach, we show results of simulated and real world experiments on a compliant humanoid robot COMAN. We show the initial results of utilizing the approach to control a pedal-racer, a demanding balance toy best described as a hybrid between a skateboard and a bicycle.

The creation and adaptation of motor behaviors is an important capability for autonomous robots. In this paper we propose an approach for altering existing robot behaviors online, where a human coach interactively changes the robot motion to achieve the desired outcome. Using hand gestures, the human coach can specify the desired modifications to the previously acquired behavior. To preserve a natural posture while performing the task, the movement is encoded in the robot's joint space using periodic dynamic movement primitives. The coaching gestures are mapped to the robot joint space via robot Jacobian and used to create a virtual force field affecting the movement. A recursive least squares technique is used to modify the existing movement with respect to the virtual force field. The proposed approach was evaluated on a simulated three degrees of freedom planar robot and on a real humanoid robot, where human coaching gestures were captured by an RGB-D sensor. Although our focus was on rhythmic movements, the developed approach is also applicable to discrete (point-to-point) movements.

In this paper we address the problem of accurate trajectory tracking while ensuring compliant robotic behaviour for periodic tasks. We propose an approach for on-line learning of task-specific dynamics, i.e. task specific movement trajectories and corresponding force/torque profiles. The proposed control framework is a multi-step process, where in the first step a human tutor shows how to perform the desired periodic task. A state estimator based on an adaptive frequency oscillator combined with dynamic movement primitives is employed to extract movement trajectories. In the second step, the movement trajectory is accurately executed in the controlled environment under human supervision. In this step, the robot is accurately tracking the acquired movement trajectory, using high feedback gains to ensure accurate tracking. Thus it can learn the corresponding force/torque profiles, i. e. task-specific dynamics. Finally, in the third step, the movement is executed with the learned feedforward task-specific dynamic model, allowing for low position feedback gains, which implies compliant robot behaviour. Thus, it is safe for interaction with humans or the environment. The proposed approach was evaluated on a Kuka LRW robot performing object manipulation and crank turning.

The framework of dynamic movement primitives (DMPs) contains many favorable properties for the execution of robotic trajectories, such as indirect dependence on time, response to perturbations, and the ability to easily modulate the given trajectories, but the framework in its original form remains constrained to the kinematic aspect of the movement. In this paper, we bridge the gap to dynamic behavior by extending the framework with force/torque feedback. We propose and evaluate a modulation approach that allows interaction with objects and the environment. Through the proposed coupling of originally independent robotic trajectories, the approach also enables the execution of bimanual and tightly coupled cooperative tasks. We apply an iterative learning control algorithm to learn a coupling term, which is applied to the original trajectory in a feed-forward fashion and, thus, modifies the trajectory in accordance to the desired positions or external forces. A stability analysis and results of simulated and real-world experiments using two KUKA LWR arms for bimanual tasks and interaction with the environment are presented. By expanding on the framework of DMPs, we keep all the favorable properties, which is demonstrated with temporal modulation and in a two-agent obstacle avoidance task.

Direct transfer of human motion trajectories to humanoid robots does not result in dynamically stable robot movements due to the differences in human and humanoid robot kinematics and dynamics. We developed a system that converts human movements captured by a low-cost RGB-D camera into dynamically stable humanoid movements. The transfer of human movements occurs in real-time. As need arises, the developed system can smoothly transition between unconstrained movement imitation and imitation with balance control, where movement reproduction occurs in the null space of the balance controller. The developed balance controller is based on an approximate model of the robot dynamics, which is sufficient to stabilize the robot during on-line imitation. However, the resulting movements cannot be guaranteed to be optimal because the model of the robot dynamics is not exact. The initially acquired movement is therefore subsequently improved by model-free reinforcement learning, both with respect to the accuracy of reproduction and balance control. We present experimental results in simulation and on a real humanoid robot.

The framework of dynamic movement primitives allows the generation of discrete and periodic trajectories, which can be modulated in various aspects. We propose and evaluate a novel modulation approach that includes force feedback and thus allows physical interaction with objects and the environment. The proposed approach also enables the coupling of independently executed robotic trajectories, simplifying the execution of bimanual and tightly coupled cooperative tasks. We apply an iterative learning control algorithm to learn a coupling term, which is applied to the original trajectory in a feed-forward fashion. The coupling term modifies the trajectory in accordance to either the desired position or external force. The strengths of the approach are shown in bimanual or two-agent obstacle avoidance tasks, where no higher level cognitive reasoning or planning are required. Results of simulated and real-world experiments on the ARMAR-III humanoid robot in interaction and object lifting tasks, and on two KUKA LWR robots in a bimanual setting are presented.

We address the problem of extracting the fundamental frequency of an arbitrary periodic or quasi-periodic signal for application in robotic tasks. We focus on controlling periodic robotic movement by extracting the frequency of human movement by using surface electromyography (EMG), a technique by which muscle action potentials are gathered by electrodes placed on the skin. However, since the EMG signal is quasi-periodic with a lot of frequency components and noise, it is difficult to determine the frequency and phase of the measured limb motion. We propose to use nonlinear dynamical systems capable of extracting the frequency and the phase from an unknown periodic signal with an arbitrary waveform. The method uses a whole Fourier series representation in a feedback loop. It is capable of extracting the frequency and the phase of an unknown periodic signal in real-time, without any additional signal processing or preprocessing. Combining this method with an output dynamic system based on dynamic movement primitives, which generate the desired trajectory, allows synchronization between human muscles actions and some other system actions like robot motion or electrical neuromuscular stimulator activation.

Acquisition of new sensorimotor knowledge by imitation is a promising paradigm for robot learning. To be effective, action learning should not be limited to direct replication of movements obtained during training but must also enable the generation of actions in situations a robot has never encountered before. This paper describes a methodology that enables the generalization of the available sensorimotor knowledge. New actions are synthesized by the application of statistical methods, where the goal and other characteristics of an action are utilized as queries to create a suitable control policy, taking into account the current state of the world. Nonlinear dynamic systems are employed as a motor representation. The proposed approach enables the generation of a wide range of policies without requiring an expert to modify the underlying representations to account for different task-specific features and perceptual feedback. The paper also demonstrates that the proposed methodology can be integrated with an active vision system of a humanoid robot. 3-D vision data are used to provide query points for statistical generalization. While 3-D vision on humanoid robots with complex oculomotor systems is often difficult due to the modeling uncertainties, we show that these uncertainties can be accounted for by the proposed approach.