Control policy learning for modular robot locomotion has previously been limited to proprioceptive feedback and flat terrain. This paper develops policies for modular systems with vision traversing more challenging environments. These modular robots can be reconfigured to form many different designs, where each design needs a controller to function. Though one could create a policy for individual designs and environments, such an approach is not scalable given the wide range of potential designs and environments. To address this challenge, we create a visual-motor policy that can generalize to both new designs and environments. The policy itself is modular, in that it is divided into components, each of which corresponds to a type of module (e.g., a leg, wheel, or body). The policy components can be recombined during training to learn to control multiple designs. We develop a deep reinforcement learning algorithm where visual observations are input to a modular policy interacting with multiple environments at once. We apply this algorithm to train robots with combinations of legs and wheels, then demonstrate the policy controlling real robots climbing stairs and curbs.

Modular robots can be rearranged into a new design, perhaps each day, to handle a wide variety of tasks by forming a customized robot for each new task. However, reconfiguring just the mechanism is not sufficient: each design also requires its own unique control policy. One could craft a policy from scratch for each new design, but such an approach is not scalable, especially given the large number of designs that can be generated from even a small set of modules. Instead, we create a modular policy framework where the policy structure is conditioned on the hardware arrangement, and use just one training process to create a policy that controls a wide variety of designs. Our approach leverages the fact that the kinematics of a modular robot can be represented as adesign graph, with nodes as modules and edges as connections between them. Given a robot, its design graph is used to create apolicy graphwith the same structure, where each node contains a deep neural network, and modules of the same type share knowledge via shared parameters (e.g., all legs on a hexapod share the same network parameters). We developed a model-based reinforcement learning algorithm, interleaving model learning and trajectory optimization to train the policy. We show the modular policy generalizes to a large number of designs that were not seen during training without any additional learning. Finally, we demonstrate the policy controlling a variety of designs to locomote with both simulated and real robots.

Modular robots are made up of a set of components which can be configured and reconfigured to form customized robots for a wide range of tasks. Fully utilizing the flexibility of modular robots is challenging, as it requires the identification of optimal modular designs for each given task, often with limited computation and time. Previous works in design automation achieve efficient run-times by utilizing machine learning to create a one-to-one mapping from task to design. However, the problem of robot design is often multimodal, where multiple distinct designs can be similarly or equally good for a task. Alternative design solutions may be needed in the field, for instance, if a module in the optimal design fails and no replacement is available. This paper presents a novel method based on generative adversarial networks (GANs) that learns a one-to-many mapping from task to a distribution of designs. We apply our method to construct locomoting modular robots for terrains with varying obstacle heights and infill. We compare our method against the state-of-the-art, and find that our algorithm results in better solution quality, diversity, and alternatives for when the optimal design fails.

Modular robots hold the promise of versatility in that their components can be re-arranged to adapt the robot design to a task at deployment time. Even for the simplest designs, determining the optimal design is exponentially complex due to the number of permutations of ways the modules can be connected. Further, when selecting the design for a given task, there is an additional computational burden in evaluating the capability of each robot, e.g., whether it can reach certain points in the workspace. This work uses deep reinforcement learning to create a search heuristic that allows us to efficiently search the space of modular serial manipulator designs. We show that our algorithm is more computationally efficient in determining robot designs for given tasks in comparison to the current state-of-the-art.

Snake robots have the potential to locomote through tightly packed spaces, but turning effectively within unmodelled and unsensed environments remains challenging. Inspired by a behavior observed in the tiny nematode worm C. elegans, we propose a novel in-place turning gait for elongated limbless robots. To simplify the control of the robots' many internal degrees-of-freedom, we introduce a biologically-inspired template in which two co-planar traveling waves are superposed to produce an in-plane turning motion, the omega turn. The omega turn gait arises from modulating the wavelengths and amplitudes of the two traveling waves. We experimentally test the omega turn on a snake robot, and show that this turning gait outperforms previous turning gaits: it results in a larger angular displacement and a smaller area swept by the body over a gait cycle, allowing the robot to turn in highly confined spaces.

Modular robots can be rapidly reconfigured into customized articulated legged morphologies capable of mobile manipulation and inspection. However, current gait generation methods do not keep pace with the speed of physical reconfiguration. This work focuses on quickly creating gaits for modular legged robots. We build on a recent method that uses trajectory optimization to design quasi-static gaits given only robot geometry and foot contact patterns. We develop methods to automatically generate contact patterns for new gaits and transitions between them. We show the utility of these methods applied to robots with many limbs, such that limbs can be fluidly reassigned to locomotion, manipulation, or inspection tasks, or to adapt gaits to hardware failures online. We demonstrate gait and transition generation with our modular hexapod and dodecapod robots. The robots switch between gaits that use all limbs for locomotion and those that leave some limbs free to pick up objects or position a camera.