In this paper, we describe the design of the Barkour benchmark for measuring robot agility in navigating complex environments. Despite the growing interest in developing agile robot locomotion skills, the field lacks systematic benchmarks to measure the performance of robotic control systems and hardware in agility-focused tasks. This motivated us to propose the Barkour benchmark, an obstacle course designed to quantify agility across various robotic platforms. Inspired by dog agility competitions, the course features diverse obstacles and a time-based scoring mechanism, encouraging researchers to develop controllers that enable robots to move quickly, precisely, and with adaptability. This benchmark is challenging as it demands diverse motion skills and the time-based scoring requires control precision at high speed. Along with the design details presented in the paper, we release our simulated environment setups in MuJoCo-XLA and the CAD model of a custom-designed quadruped robot to facilitate future research to reproduce the Barkour setup (available at sites.google.com/view/barkour). We hope these together will accelerate the pace of robot agility research.

Human-robot communication is key to establishing transparency of robot states, and promoting psychological safety of the people interacting with a robot. Such communication is especially challenging for appearance-constrained robots, as they do not afford to employ commonly-used an-thropomorphic modalities such as facial expressions. We explore how an appearance-constrained quadruped robot can use LEDs and gestures to generate affective expressions in order to non-verbally communicate. Traditionally, affect has been used as a signalling paradigm in Human-Robot Interaction where expressions are designed with the goal of communicating a particular emotion such as happiness. Instead, this work explores a generalizable mapping from informative messages (e.g., "I am listening") to abstract affective expressions. We modulate the expressions by a model of affect, taking inspirations from Affect Control Theory (ACT). To explore designing the expressions, we first conducted pilot semi-structured interviews consulting stakeholders who regularly operate appearance-constrained robots (N = 12) to understand the communication requirements. Then, we designed a set of six affective expressions and evaluated them in a crowdsourced study (N = 450). Findings suggest that the expressions can significantly improve effective communication of a robot’s awareness and intent, and promote psychological safety of people interacting with it.

Developing agile behaviors for legged robots re-mains a challenging problem. While deep reinforcement learning is a promising approach, learning truly agile behaviors typically requires tedious reward shaping and careful curriculum design. We formulate agile locomotion as a multi-stage learning problem in which a mentor guides the agent throughout the training. The mentor is optimized to place a checkpoint to guide the movement of the robot’s center of mass while the student (i.e. the robot) learns to reach these checkpoints. Once the student can solve the task, we teach the student to perform the task without the mentor. We evaluate our proposed learning system with a simulated quadruped robot on a course consisting of randomly generated gaps and hurdles. Our method significantly outperforms a single-stage RL baseline without a mentor, and the quadruped robot can agilely run and jump across gaps and obstacles. Finally, we present a detailed analysis of the learned behaviors’ feasibility and efficiency.

Learning adaptable policies is crucial for robots to operate autonomously in our complex and quickly changing world. In this work, we present a new meta-learning method that allows robots to quickly adapt to changes in dynamics. In contrast to gradient-based meta-learning algorithms that rely on second-order gradient estimation, we introduce a more noise-tolerant Batch Hill-Climbing adaptation operator and combine it with meta-learning based on evolutionary strategies. Our method significantly improves adaptation to changes in dynamics in high noise settings, which are common in robotics applications. We validate our approach on a quadruped robot that learns to walk while subject to changes in dynamics. We observe that our method significantly outperforms prior gradient-based approaches, enabling the robot to adapt its policy to changes based on less than 3 minutes of real data.

Legged locomotion is a challenging task for learning algorithms, especially when the task requires a diverse set of primitive behaviors. To solve these problems, we introduce a hierarchical framework that can automatically learn to decompose complex locomotion tasks. A high-level policy issues commands in the form of a latent vector and also selects for how long the low-level policy will execute the latent command. Concurrently, the low-level policy uses the latent command and only the robot's on-board sensors to control the robot's actuators. Our approach allows the high-level policy to run at a lower frequency than the low-level one. We test our framework on a path-following task for a dynamic quadruped robot and we show that steering behaviors automatically emerge in the latent command space as low-level skills are needed for this task. We then show efficient adaptation of the trained policy to new tasks by transfer of the trained low-level policy. Finally, we validate the policies on a real quadruped robot. To the best of our knowledge, this is the first application of end-to-end hierarchical learning to a real robotic locomotion task.

Tensegrity robots, composed of rigid rods connected by elastic cables, have a number of unique properties that make them appealing for use as planetary exploration rovers. However, control of tensegrity robots remains a difficult problem due to their unusual structures and complex dynamics. In this work, we show how locomotion gaits can be learned automatically using a novel extension of mirror descent guided policy search (MDGPS) applied to periodic locomotion movements, and we demonstrate the effectiveness of our approach on tensegrity robot locomotion. We evaluate our method with real-world and simulated experiments on the SUPERball tensegrity robot, showing that the learned policies generalize to changes in system parameters, unreliable sensor measurements, and variation in environmental conditions, including varied terrains and a range of different gravities. Our experiments demonstrate that our method not only learns fast, power-efficient feedback policies for rolling gaits, but that these policies can succeed with only the limited onboard sensing provided by SUPERball's accelerometers. We compare the learned feedback policies to learned open-loop policies and hand-engineered controllers, and demonstrate that the learned policy enables the first continuous, reliable locomotion gait for the real SUPERball robot. Our code and supplementary material is available from http://rll.berkeley.edu/drl_tensegrity.

Tensegrity robots are a class of compliant robots that have many desirable traits when designing mass efficient systems that must interact with uncertain environments. Various promising control approaches have been proposed for tensegrity systems in simulation. Unfortunately, state estimation methods for tensegrity robots have not yet been thoroughly studied. In this paper, we present the design and evaluation of a state estimator for tensegrity robots. This state estimator will enable existing and future control algorithms to transfer from simulation to hardware. Our approach is based on the unscented Kalman filter (UKF) and combines inertial measurements, ultra wideband time-of-flight ranging measurements, and actuator state information. We evaluate the effectiveness of our method on the SUPERball, a tensegrity based planetary exploration robotic prototype. In particular, we conduct tests for evaluating both the robot's success in estimating global position in relation to fixed ranging base stations during rolling maneuvers as well as local behavior due to small-amplitude deformations induced by cable actuation.

Truly autonomous robots require the capacity to recognise their surroundings by interpreting their sensorimotor stream. We present an online learning algorithm for training a mixture of echo state network experts that can segment a compliant robot's sensorimotor stream. Our method follows a probabilistic approach, using a hidden Markov model to model the switching dynamics between the experts. The algorithm's performance is evaluated on an unsupervised terrain classification problem using a compliant, underactuated, six-strut tensegrity robot. The results show that our model captures the influence of terrain-robot interactions on the robot's complex dynamics and correctly segments the sensorimotor stream. We demonstrate that the activity pattern of the experts can be used to train a highly compliant robot to distinguish between different environments using only noisy internal sensors.

The Spherical Underactuated Planetary Exploration Robot ball (SUPERball) is an ongoing project within NASA Ames Research Center's Intelligent Robotics Group and the Dynamic Tensegrity Robotics Lab (DTRL). The current SUPERball is the first full prototype of this tensegrity robot platform, eventually destined for space exploration missions. This work, building on prior published discussions of individual components, presents the fully-constructed robot. Various design improvements are discussed, as well as testing results of the sensors and actuators that illustrate system performance. Basic low-level motor position controls are implemented and validated against sensor data, which show SUPERball to be uniquely suited for highly dynamic state trajectory tracking. Finally, SUPERball is shown in a simple example of locomotion. This implementation of a basic motion primitive shows SUPERball in untethered control.

Incorporating the body dynamics of compliant robots into their controller architectures can drastically reduce the complexity of locomotion control. An extreme version of this embodied control principle was demonstrated in highly compliant tensegrity robots, for which stable gait generation was achieved by using only optimized linear feedback from the robot's sensors to its actuators. The morphology of quadrupedal robots has previously been used for sensing and for control of a compliant spine, but never for gait generation. In this paper, we successfully apply embodied control to the compliant, quadrupedal Oncilla robot. As initial experiments indicated that mere linear feedback does not suffice, we explore the minimal requirements for robust gait generation in terms of memory and nonlinear complexity. Our results show that a memoryless feedback controller can generate a stable trot by learning the desired nonlinear relation between the input and the output signals. We believe this method can provide a very useful tool for transferring knowledge from open loop to closed loop control on compliant robots.

Soft robots offer many advantages over traditional rigid robots. However, soft robots can be difficult to control with standard control methods. Fortunately, evolutionary algorithms can offer an elegant solution to this problem. Instead of creating controls to handle the intricate dynamics of these robots, we can simply evolve the controls using a simulation to provide an evaluation function. In this article, we show how such a control paradigm can be applied to an emerging field within soft robotics: robots based on tensegrity structures. We take the model of the Spherical Underactuated Planetary Exploration Robot ball (SUPERball), an icosahedron tensegrity robot under production at NASA Ames Research Center, develop a rolling locomotion algorithm, and study the learned behavior using an accurate model of the SUPERball simulated in the NASA Tensegrity Robotics Toolkit. We first present the historical-average fitness-shaping algorithm for coevolutionary algorithms to speed up learning while favoring robustness over optimality. Second, we use a distributed control approach by coevolving open-loop control signals for each controller. Being simple and distributed, open-loop controllers can be readily implemented on SUPERball hardware without the need for sensor information or precise coordination. We analyze signals of different complexities and frequencies. Among the learned policies, we take one of the best and use it to analyze different aspects of the rolling gait, such as lengths, tensions, and energy consumption. We also discuss the correlation between the signals controlling different parts of the tensegrity robot.

NASA Ames Research Center is developing a compliant modular tensegrity robotic platform for planetary exploration. In this paper we present the design and evolution of the platform's main hardware component, an untethered, robust tensegrity strut, with rich sensor feedback and cable actuation. Each strut is a complete robot, and multiple struts can be combined together to form a wide range of complex tensegrity robots. Our current goal for the tensegrity robotic platform is the development of SUPERball, a 6-strut icosahedron underactuated tensegrity robot aimed at dynamic locomotion for planetary exploration rovers and landers, but the aim is for the modular strut to enable a wide range of tensegrity morphologies. SUPERball is a second generation prototype, evolving from the tensegrity robot ReCTeR, which is also a modular, lightweight, highly compliant 6-strut tensegrity robot that was used to validate our physics based NASA Tensegrity Robot Toolkit (NTRT) simulator. Many hardware design parameters of the SUPERball were driven by locomotion results obtained in our validated simulator. These evolutionary explorations helped constrain motor torque and speed parameters, along with strut and string stress. As construction of the hardware has finalized, we have also used the same evolutionary framework to evolve controllers that respect the built hardware parameters.

In the field of Reservoir Computing, scaling the spectral radius of the weight matrix of a random recurrent neural network to below unity is a commonly used method to ensure the Echo State Property. Recently it has been shown that this condition is too weak. To overcome this problem, other - more involved - sufficient conditions for the Echo State Property have been proposed. In this paper we provide a large-scale experimental verification of the Echo State Property for large recurrent neural networks with zero input and zero bias. Our main conclusion is that the spectral radius method remains a valid indicator of the Echo State Property; the probability that the Echo State Property does not hold, drops for larger networks with spectral radius below unity, which are the ones of practical interest.

Embodiment has led to a revolution in robotics by not thinking of the robot body and its controller as two separate units, but taking into account the interaction of the body with its environment. By investigating the effect of the body on the overall control computation, it has been suggested that the body is effectively performing computations, leading to the term morphological computation. Recent work has linked this to the field of reservoir computing, allowing one to endow morphologies with a theory of universal computation. In this work, we study a family of highly dynamic body structures, called tensegrity structures, controlled by one of the simplest kinds of "brains." These structures can be used to model biomechanical systems at different scales. By analyzing this extreme instantiation of compliant structures, we demonstrate the existence of a spectrum of choices of how to implement control in the body-brain composite. We show that tensegrity structures can maintain complex gaits with linear feedback control and that external feedback can intrinsically be integrated in the control loop. The various linear learning rules we consider differ in biological plausibility, and no specific assumptions are made on how to implement the feedback in a physical system.

This paper presents the results of modeling of a mixed non-linear, strongly coupled and multidomain electromechanical system designed to scavenge the energy of ambient vibrations and to generate an electrical supply for an embedded microsystem. The system is operating in three domains: purely mechanical (the resonator), coupled electromechanical (electrostatic transducer associated with the moving mass) and electrical circuit, including switches, diodes and linear electrical components. Although only linear networks can be properly modeled in SystemC-AMS, we propose a technique allowing modeling of electrical networks including non-linear components (diodes) as well as time-varying capacitors. The whole system was modeled using two solvers of SystemC-AMS simultaneously: the one allowing TDF (timed data flow) modeling and the one allowing LIN ELEC (linear electrical) circuit analysis. The modeling results are compared with VHDL-AMS and Matlab Simulink models.