We present a novel second-order trajectory optimization algorithm based on Stein Variational Newton's Method and Maximum Entropy Differential Dynamic Programming. The proposed algorithm, called Stein Variational Differential Dynamic Programming, is a kernel-based extension of Maximum Entropy Differential Dynamic Programming that combines the best of the two worlds of sampling-based and gradient-based optimization. The resulting algorithm avoids known drawbacks of gradient-based dynamic optimization in terms of getting stuck at local minima, while it overcomes limitations of sampling-based stochastic optimization in terms of introducing undesirable stochasticity when applied in online fashion. To test the efficacy of the proposed algorithm, experiments are conducted in Model Predictive Control mode. The experiments include comparisons with unimodal and multimodal Maximum Entropy Differential Dynamic Programming as well as Model Predictive Path Integral Control and its multimodal and Stein Variational extensions. The results demonstrate the superior performance of the proposed algorithms and confirm the hypothesis that there is a middle ground between sampling-and gradient-based optimization that is indeed beneficial for dynamic optimization.

The control of granular materials, which are found in many industrial applications, is a challenging open research problem. Granular material systems are complex-behavior (as they could have solid-, fluid-, and gas-like behaviors) and high-dimensional (as they could have many grains/particles with at least 3 DOF in 3D) systems. Recently, a machine learning-based Graph Neural Network (GNN) simulator has been proposed to learn the underlying dynamics. In this paper, we perform optimal control of a rigid body-driven granular material system whose dynamics is learned by a GNN model trained by reduced data generated via a physics-based simulator and Principal Component Analysis (PCA). We use Differential Dynamic Programming (DDP) to obtain optimal control commands that can form granular particles into a target shape. The model and results are shown to be relatively fast and accurate. The control commands are also applied to the ground truth model, i.e., physics-based simulator, to further validate the approach.

This article proposes two decentralized multiagent optimal control methods that combine the computational efficiency and scalability of differential dynamic programming (DDP) and the distributed nature of the alternating direction method of multipliers (ADMM). The first one, nested distributed DDP, is a three-level architecture, which employs ADMM for consensus, an augmented Lagrangian layer for local constraints and DDP as the local optimizer. The second one, merged distributed DDP, is a two-level architecture that addresses both consensus and local constraints with ADMM, further reducing computational complexity. Both frameworks arefully decentralizedsince all computations are parallelizable among the agents and only local communication is necessary. Simulation results that scale up to thousands of cars and hundreds of drones demonstrate the effectiveness of the algorithms. Superior scalability to large-scale systems against other DDP and sequential quadratic programming methods is also illustrated. Finally, hardware experiments on a multirobot platform verify the applicability of the methods. A video with all results is provided in the supplementary material.

Differential Dynamic Programming (DDP) has become a well established method for unconstrained trajectory optimization. Despite its several applications in robotics and controls, however, a widely successful constrained version of the algorithm has yet to be developed. This paper builds upon penalty methods and active-set approaches towards designing a Dynamic Programming-based methodology for constrained optimal control. Regarding the former, our derivation employs a constrained version of Bellman’s principle of optimality, by introducing a set of auxiliary slack variables in the backward pass. In parallel, we show how Augmented Lagrangian methods can be naturally incorporated within DDP, by utilizing a particular set of penalty-Lagrangian functions that preserve second-order differentiability. We demonstrate experimentally that our extensions (individually and combinations thereof) enhance significantly the convergence properties of the algorithm, and outperform previous approaches on a large number of simulated scenarios.