Deploying reinforcement learning (RL) safely in the real world is challenging, as policies trained in simulators must face the inevitable *sim-to-real gap*. Robust safe RL techniques are provably safe, however difficult to scale, while domain randomization is more practical yet prone to unsafe behaviors. We address this gap by proposing SPiDR, short for Sim-to-real via Pessimistic Domain Randomization—a scalable algorithm with provable guarantees for safe sim-to-real transfer. SPiDR uses domain randomization to incorporate the uncertainty about the sim-to-real gap into the safety constraints, making it versatile and highly compatible with existing training pipelines. Through extensive experiments on sim-to-sim benchmarks and two distinct real-world robotic platforms, we demonstrate that SPiDR effectively ensures safety despite the sim-to-real gap while maintaining strong performance.

This work presents a motion retargeting approach for legged robots, aimed at transferring the dynamic and agile movements to robots from source motions. In particular, we guide the imitation learning procedures by transferring motions from source to target, effectively bridging the morphological disparities while ensuring the physical feasibility of the target system. In the first stage, we focus on motion retargeting at the kinematic level by generating kinematically feasible whole-body motions from keypoint trajectories. Following this, we refine the motion at the dynamic level by adjusting it in the temporal domain while adhering to physical constraints. This process facilitates policy training via reinforcement learning, enabling precise and robust motion tracking. We demonstrate that our approach successfully transforms noisy motion sources, such as hand-held camera videos, into robot-specific motions that align with the morphology and physical properties of the target robots. Moreover, we demonstrate terrain-aware motion retargeting to perform BackFlip on top of a box. We successfully deployed these skills to four robots with different dimensions and physical properties in the real world through hardware experiments.

Control policies trained using deep reinforcement learning often generate stiff, high-frequency motions in response to unexpected disturbances. To promote more natural and compliant balance recovery strategies, we propose a simple modification to the typical reinforcement learning training process. Our key insight is that stiff responses to perturbations are due to an agent’s incentive to maximize task rewards at all times, even as perturbations are being applied. As an alternative, we introduce an explicit recovery stage where tracking rewards are given irrespective of the motions generated by the control policy. This allows agents a chance to gradually recover from disturbances before attempting to carry out their main tasks. Through an in-depth analysis, we highlight both the compliant nature of the resulting control policies, as well as the benefits that compliance brings to legged locomotion. In our simulation and hardware experiments, the compliant policy achieves more robust, energy-efficient, and safe interactions with the environment.

This work presents a motion capture-driven locomotion controller for quadrupedal robots that replicates the non-periodic footsteps and subtle body movement of animal motions. We adopt a nonlinear model predictive control (NMPC) formulation that generates optimal base trajectories and stepping locations. By optimizing both footholds and base trajectories, our controller effectively tracks retargeted animal motions with natural body movements and highly irregular strides. We demonstrate our approach with prerecorded animal motion capture data. In simulation and hardware experiments, our motion controller enables quadrupedal robots to robustly reproduce fundamental characteristics of a target animal motion regardless of the significant morphological disparity.

In this paper, we explore the challenge of generating animal-like walking motions for legged robots. To this end, we propose a versatile and robust control pipeline that combines a state-of-the-art model-based controller with a data-driven technique that is commonly used in computer animation. We demonstrate the efficacy of our control framework on a variety of quadrupedal robots in simulation. We show, in particular, that our approach can automatically reproduce key characteristics of animal motions, including speed-specific gaits, unscripted footfall patterns for nonperiodic motions, and natural small variations in overall body movements.

We present a control-aware design optimization method for quadrupedal robots. In particular, we show that it is possible to analytically differentiate typical, inverse dynamics-based whole body controllers with respect to design parameters, and that gradient-based methods can be used to efficiently improve an initial morphological design according to well-established metrics. We apply our design optimization method to various types of quadrupedal robots, including designs that feature closed kinematic chains. The methodology we present enables a principled comparison of different types of optimized legged robot designs. Our experiments, for example, suggest that mechanically-coupled three-link leg designs present notable advantages in terms of performance and efficiency over the common two-link leg designs used in most quadrupedal robots today.