Differential Dynamic Programming (DDP) has become a well established method for unconstrained trajectory optimization. Despite its several applications in robotics and controls, however, a widely successful constrained version of the algorithm has yet to be developed. This paper builds upon penalty methods and active-set approaches towards designing a Dynamic Programming-based methodology for constrained optimal control. Regarding the former, our derivation employs a constrained version of Bellman’s principle of optimality, by introducing a set of auxiliary slack variables in the backward pass. In parallel, we show how Augmented Lagrangian methods can be naturally incorporated within DDP, by utilizing a particular set of penalty-Lagrangian functions that preserve second-order differentiability. We demonstrate experimentally that our extensions (individually and combinations thereof) enhance significantly the convergence properties of the algorithm, and outperform previous approaches on a large number of simulated scenarios.

We propose a sampling-based trajectory optimization methodology for constrained problems. We extend recent works on stochastic search to deal with box control constraints, as well as nonlinear state constraints for discrete dynamical systems. Regarding the former, our strategy is to optimize over truncated parameterized distributions on control inputs. Furthermore, we show how non-smooth penalty functions can be incorporated into our framework to handle state constraints. Simulations on cartpole and quadcopter show that our approach outperforms previous methods on constrained sampling-based optimization, in terms of quality of solutions and convergence speed.

We present a trajectory optimization approach to reinforcement learning in continuous state and action spaces, called probabilistic differential dynamic programming (PDDP). Our method represents systems dynamics using Gaussian processes (GPs), and performs local dynamic programming iteratively around a nominal trajectory in Gaussian belief spaces. Different from model-based policy search methods, PDDP does not require a policy parameterization and learns a time-varying control policy via successive forward-backward sweeps. A convergence analysis of the iterative scheme is given, showing that our algorithm converges to a stationary point globally under certain conditions. We show that prior model knowledge can be incorporated into the proposed framework to speed up learning, and a generalized optimization criterion based on the predicted cost distribution can be employed to enable risk-sensitive learning. We demonstrate the effectiveness and efficiency of the proposed algorithm using nontrivial tasks. Compared with a state-of-the-art GP-based policy search method, PDDP offers a superior combination of learning speed, data efficiency, and applicability.

The majority of the works on grasping consider both object as well as robot hand parameters to be accurately known and do not take into account the constraints imposed by the robot hand. In this paper, a complete methodology is proposed that handles the grasping problem under a wide range of uncertainties. Initially, we search for an acceptable posture that provides robustness against positioning inaccuracies and maximizes the ability of the robot hand to exert forces on the object. Subsequently, in order to secure the grasp stability, we also deal with the determination of sufficient contact forces. Finally, an appropriate tactile sensor setup, mounted on the robot hand, allow us to reduce the magnitude of uncertainty regarding the grasping parameters. The efficiency of our approach is validated through extensive experimental paradigms using a 15 DoF DLR/HIT II robotic hand attached at the end effector of a 7 DoF Mitsubishi PA10 robotic manipulator.

In this paper, we propose a complete methodology for deriving task-specific force closure grasps for multifingered robot hands under a wide range of uncertainties. Given a finite set of external disturbances representing the task to be executed, the concept of Q distance is introduced in a novel way to determine an efficient grasp with a task compatible hand posture (i.e., configuration and contact points). Our approach takes, also, into consideration the mechanical and geometric limitations imposed by the robotic hand design and the object to be grasped. In addition, incorporating our recent results on grasping [1], the ability of the robot hand to exert the required contact forces is maximized and robustness against positioning inaccuracies and object uncertainties is established. Finally, the efficiency of our approach is verified through an experimental study on the 15 DoF DLR/HIT II robotic hand attached at the end effector of the 7 DoF Mitsubishi PA10 robotic manipulator.