Programming manipulation behaviors can become increasingly difficult with a growing number and complexity of manipulation tasks, particularly in a dynamic and unstructured environment. Recent progress in unsupervised skill discovery algorithms has shown great promise in learning an extensive collection of behaviors without extrinsic supervision. On the other hand, safety is one of the most critical factors for real- world robot applications. As skill discovery methods typically encourage exploratory and dynamic behaviors, it can often be the case that a large portion of learned skills remain too dangerous and unsafe. In this paper, we introduce the novel problem of Safety-Aware Skill Discovery, which aims to learn, in a task-agnostic fashion, a repertoire of reusable skills that are inherently safe to be composed for solving downstream tasks. We present a computationally tractable algorithm that learns a latent-conditioned skill policy that maximizes intrinsic rewards regularized with a safety-critic that can model any user-defined safety constraints. Using the pretrained safe skill repertoire, hierarchical reinforcement learning can solve multiple downstream tasks without the need for explicit consideration of safety during training and testing. We evaluate our algorithm on a collection of force-controlled robotic manipulation tasks in simulation and show promising downstream task performance while satisfying safety constraints.

A robot's dynamic model depends on both the kinematic and mass-inertial parameters of a robot. Robot model identification therefore typically begins with kinematic identification; the mass-inertial parameters are then identified with the identified kinematic parameters used in the dynamic model. In this article we show that poorly identified kinematic parameters can lead to an uncorrectable bias in the dynamic model, leading to errors in the mass-inertial parameters that are many times that of kinematic parameter errors. We instead argue that a unified kinodynamic identification leads to more accurate identification of both the kinematic and mass-inertial parameters. A linearly weighted kinodynamic objective function is proposed, in which the weight can be interpreted from a maximum likelihood perspective as the relative accuracy of the kinematic sensors vis-à-vis the dynamic sensors. Recursive algorithms for computing exact analytic gradients of the kinodynamic objective function are newly derived, leading to robust and fast-converging identification algorithms. Extensive numerical and hardware experiments demonstrate the advantages of our unified kinodynamic identification procedure.

Existing adaptive robot control laws typically require an engineering choice of a constant adaptation gain matrix, which often involves repeated and time-consuming trial and error. Moreover, physical consistency of the estimated inertial parameters or the uniform positive definiteness of the estimated robot mass matrix cannot in general be guaranteed without nonsmooth corrections, e.g., projection to the boundary of the feasible parameter set. In this paper we present a natural adaptive control law that mitigates many of these difficulties, by exploiting the coordinate-invariant differential geometric structure of the space of physically consistent inertial parameters. Our approach provides a more generalizable and physically consistent adaptation law for the robot parameters without significant additional computations compared to existing methods. Simulation results showing markedly improved tracking error convergence over existing adaptive control laws are provided as validation.