The ability to apply a previously-learned skill (e.g., pushing) to a new task (context or object) is an important requirement for new-age robots. An attempt is made to solve this problem in this paper by proposing a deep meta-imitation learning framework comprising of an attentive-embedding net-work and a control network, capable of learning a new task in an end-to-end manner while requiring only one or a few visual demonstrations. The feature embeddings learnt by incorporating spatial attention is shown to provide higher embedding and control accuracy compared to other state-of-the-art methods such as TecNet [7] and MIL [4]. The interaction between the embedding and the control networks is improved by using multiplicative skip-connections and is shown to overcome the overfitting of the trained model. The superiority of the proposed model is established through rigorous experimentation using a publicly available dataset and a new dataset created using PyBullet [36]. Several ablation studies have been carried out to justify the design choices.

This paper is concerned with the motor skill learning from human demonstrations using the framework of dynamic regressive models (DRMs). The DRM-based motion planner is preferred as it generates the end-effector trajectory dynamically based on the current state of the end-effector. Within existing frameworks, a single DRM can learn a single motion profile. In addition, such learned DRMs from the data may not be stable. This paper addresses both these issues in a comprehensive manner. In this paper a single DRM has been used to encode human demonstrations involving multitask profiles and multiple task-equilibriums which is novel. We have introduced the idea that the learned DRM will generate human-like stable motion if the energy dissipation rate (EDR) of the generated trajectory follows that of the human demonstration. Thus, the DRM structure has been modified by adding a continuous guiding signal which can be called as the control signal. This signal has been derived using control theoretic principle to ensure asymptotic stability while maintaining the EDR equivalent to that of the human demonstration. The asymptotic stability of the learned DRM has been established by involving a nonmonotonic Lyapunov function consisting of first derivative of a quadratic function and the energy function associated with the DRM. The proposed framework can be learned using many existing regression techniques in this paper Gaussian mixture regression, locally weighted projection regression, and support vector regression techniques have been used successfully. During the pick and place tasks, human demonstrations involving multiple task profiles and multiple task-equilibriums are generated using a 7 DOF commercial robot manipulator. Experimental validations show that the DRMs learned using these three regression schemes are able to guide the robot along the multitask profiles in a stable manner.

This paper proposes an end-to-end self-supervised feature representation network named Attentive Task-Net or AT-Net for video-based task imitation. The proposed AT-Net incorporates a novel multi-level spatial attention module to highlight spatial features corresponding to the intended task demonstrated by the expert. The neural connections in AT-Net ensure the relevant information in the demonstration is amplified and the irrelevant information is suppressed while learning task-specific feature embeddings. This is achieved by a weighted combination of multiple intermediate feature maps of the input image at different stages of the CNN pipeline. The weights of the combination are given by the compatibility scores, predicted by the attention module for respective feature maps. The AT-Net is trained using a metric learning loss which aims to decrease the distance between the feature representations of concurrent frames from multiple view points and increase the distance between temporally consecutive frames. The AT-Net features are then used to formulate a reinforcement learning problem for task imitation. Through experiments on the publicly available Multi-view pouring dataset, it is demonstrated that the output of the attention module highlights the task-specific objects while suppressing the rest of the background. The efficacy of the proposed method is further validated by qualitative and quantitative comparison with a state-of-the-art technique along with intensive ablation studies. The proposed method is implemented to imitate a pouring task where an RL agent is learned with the AT-Net in Gazebo simulator. Our findings show that the AT-Net achieves 6.5% decrease in alignment error along with a reduction in the number of training iterations by almost 155k over the state-of-the-art while satisfactorily imitating the intended task.

In this paper, we propose an unsupervised deep learning framework with Bayesian inference for improving the accuracy of per-pixel depth prediction from monocular RGB images. The proposed framework predicts confidence map along with depth and pose information for a given input image. The depth hypotheses from previous frames are propagated forward and fused with the depth hypothesis of the current frame by using Bayesian inference mechanism. The ground truth information required for training the confidence map prediction is constructed using image reconstruction loss thereby obviating the need for explicit ground truth depth information used in supervised methods. The resulting unsupervised framework is shown to outperform the existing state-of-the-art methods for depth prediction on the publicly available KITTI outdoor dataset. The usefulness of the proposed framework is further established by demonstrating a real-world robotic pick-and-place application where the pose of the robot end-effector is computed using the depth predicted from an eye-in-hand monocular camera. The design choices made for the proposed framework is justified through extensive ablation studies.

This paper looks into the problem of online 2D bin packing where the objective is to place an incoming object in a way so as to maximize the overall packing density inside the bin. Unlike off-line methods, the online methods do not make use of information about the sequence of future objects that are going to arrive and hence, are comparatively difficult to solve. A deep reinforcement learning framework based on Double DQN is proposed to solve this problem that takes an image showing the current state of the bin as input and gives out the pixel location where the incoming object needs to be placed as the output. The reward function is defined in such a way so that the system learns to place an incoming object adjacent to the already placed items so that the maximum grouped empty area is retained for future placement. The resulting approach is shown to outperform existing state-of-the-art-method for 2D online packing and can easily be extended to 3D online bin packing problems.

This paper presents a feature agnostic and model-free visual servoing (VS) technique using deep reinforcement learning (DRL) which exploits two new architectures of experience replay buffer in deep deterministic policy gradient (DDPG). The proposed architectures are significantly fast and converge in a few numbers of steps. We use the proposed method to learn an end-to-end VS with eye-in-hand configuration. In traditional DDPG, the experience replay memory is randomly sampled for training the actor-critic network. This results in a loss of useful experiences when the buffer contains very few successful examples. We solve this problem by proposing two new replay buffer architectures: (a) min-heap DDPG (mH-DDPG) and (b) dual replay buffer DDPG (dR-DDPG). The former uses a min-heap data structure to implement the replay buffer whereas the latter uses two buffers to separate “good” examples from the “bad” examples. The training data for the actor-critic network is created as a weighted combination of the two buffers. The proposed algorithms are validated in simulation with the UR5 robotic manipulator model. It is observed that as the number of good experiences increases in the training data, the convergence time decreases. We find 27.25% and 43.25% improvements in the rate of convergence respectively by mH-DDPG and dR-DDPG over state-of-the-art DDPG.

Transferring skills to roUots through human demonstrations is an interesting problem. Locally generated demonstrations of reaching motion, given by a human teacher are generally encoded in a dynamical model. Stability of this encoding system demands great attention while learning the model parameters. In that context, we present a new architecture of dynamical system to learn movement primitives from multiple demonstrations exploiting a fuzzy Lyapunov function (FLF). We assume that there exists a natural Lyapunov function (LF) that associates the demonstrations. The proposed FLF tries to approximate that LF. First, the dynamics of the demonstrations are encoded in a regressive model, learnt using Gaussian mixture regression with EM algorithm. Then the FLF is searched involving the learnt dynamics in an optimization process. The FLF in turn helps to learn a fuzzy controller. Our architecture is new in a sense that it combines the probabilistic model with a fuzzy controller to create a globally asymptotically stable motion model. The proposed algorithm can simultaneously learn position and orientation profiles in a single model. The algorithm is experimentally validated on a commercially available manipulator and also compared with a state-of-the-art technique.

This paper presents a single-network adaptive critic-based controller for continuous-time systems with unknown dynamics in a policy iteration (PI) framework. It is assumed that the unknown dynamics can be estimated using the Takagi-Sugeno-Kang fuzzy model with arbitrary precision. The successful implementation of a PI scheme depends on the effective learning of critic network parameters. Network parameters must stabilize the system in each iteration in addition to approximating the critic and the cost. It is found that the critic updates according to the Hamilton-Jacobi-Bellman formulation sometimes lead to the instability of the closed-loop systems. In the proposed work, a novel critic network parameter update scheme is adopted, which not only approximates the critic at current iteration but also provides feasible solutions that keep the policy stable in the next step of training by combining a Lyapunov-based linear matrix inequalities approach with PI. The critic modeling technique presented here is the first of its kind to address this issue. Though multiple literature exists discussing the convergence of PI, however, to the best of our knowledge, there exists no literature, which focuses on the effect of critic network parameters on the convergence. Computational complexity in the proposed algorithm is reduced to the order of (Fz)(n-1) , where n is the fuzzy state dimensionality and Fz is the number of fuzzy zones in the states space. A genetic algorithm toolbox of MATLAB is used for searching stable parameters while minimizing the training error. The proposed algorithm also provides a way to solve for the initial stable control policy in the PI scheme. The algorithm is validated through real-time experiment on a commercial robotic manipulator. Results show that the algorithm successfully finds stable critic network parameters in real time for a highly nonlinear system.

Kinematic motor behaviour of robots can be encoded using dynamical systems. These dynamical systems are learnt from human demonstrations to generalize human like motions. The size of the demonstration space involving a robot usually being large, it is not possible to provide all the demonstrations for robot's learning. Hence, it requires an efficient learning architecture that is able to generalize for unseen contexts. In the proposed algorithm, we model the movements, shown in the human demonstrations, as non-linear multivariate dynamics using mixture of Gaussians. Generally, in non-linear multivariate modelling approach pertaining to programming by demonstration, requires structured demonstrations i.e. it always requires to have a fixed and unique equilibrium point during the learning phase. The proposed method can relax these constraints and has the following advantage over the existing work: first, it would be possible to learn from any demonstration which is not constrained to have always the same equilibrium point; second, it would be possible to capture the variations in movement patterns depending upon the position of the equilibrium point. The proposed algorithm has been implemented using Barrett WAM and experimental results have been compared with existing approach.

A near optimal control technique for robotic manipulator with completely unknown dynamics is described in this work. Obtaining the optimal control law u* depends on solving Hamilton Jacobi Bellman equation but getting an analytic solution is not possible for unknown models. It is shown that instead of solving HJB equation analytically, the optimal control law can be obtained through learning of a Single Network Adaptive Critic (SNAC). The generic nonlinear model of manipulator dynamics is represented as Takagi-Sugeno-Kang fuzzy combination of local linear models. A stabilizing fixed gain controller is designed for the TSK fuzzy system using an unconventional Lyapunov function that is used to represent the value function. Stable Lyapunov P(i)matrices are selected using the Genetic Algorithm (GA) Toolbox in Matlab. This approach avoids the learning of initial cost that can be accumulated by an existing controller. The critic is trained to approximate the optimal cost J* by renewing the policy in iterations. Validation of the proposed technique is done through simulation on a robotic manipulator model. Results show the effectiveness of the presented work.