In this paper, we focus on the problem of directing the gaze of a vision system mounted to the body of a high-degree-of-freedom (DOF) legged robot for active perception deployments. In particular, we consider the case where the vision system is rigidly attached to the robot's body (i.e., without any additional DOF between the vision system and robot body) and show how the supernumerary DOFs of the robot can be leveraged to allow independent locomotion and gaze control. Specifically, we augment a workspace central pattern generator (CPG) with omnidirectional capabilities by coupling it with a body pose control mechanism. We leverage the smoothing nature of the CPG framework to allow online adaptation of relevant locomotion parameters, and obtain a stable mid-level controller that translates desired gaze orientation and body velocity directly into joint angles. We validate our approach on an 18-DOF hexapod robot, in a series of indoor and outdoor trials, where the robot inspects an environmental feature or follows a pre-planned path relative to a visually-tracked landmark, demonstrating simultaneous locomotion and directed vision.

State-of-the-art distributed algorithms for reinforcement learning rely on multiple independent agents, which simultaneously learn in parallel environments1while asynchronously updating a common, shared policy. Moreover, decentralized control architectures (e.g., central pattern generators) can coordinate spatially distributed portions of an articulated robot to achieve system-level objectives. In this paper, we investigate the relationship between distributed learning and decentralized control by learning decentralized control policies for the locomotion of articulated robots in challenging environments. To this end, we present an approach that leverages the structure of the asynchronous advantage actor-critic (A3C) algorithm to provide a natural means of learning decentralized control policies on a single articulated robot. Our primary contribution shows individual agents in the A3C algorithm can be defined by independently controlled portions of the robot's body, thus enabling distributed learning on a single robot for efficient hardware implementation. We present results of closed-loop locomotion in unstructured terrains on a snake and a hexapod robot, using decentralized controllers learned offline and online, respectively, as a natural means to cover the different key applications of our approach. For the snake robot, we are optimizing the forward progression in unstructured environments, but for the hexapod robot, the goal is to maintain a stabilized body pose. Our results show that the proposed approach can be adapted to many different types of articulated robots by controlling some of their independent parts in a distributed manner, and the decentralized policy can be trained with high sample efficiency.

Decentralized control architectures, such as those conventionally defined by central pattern generators, independently coordinate spatially distributed portions of articulated bodies to achieve system-level objectives. State of the art distributed algorithms for reinforcement learning employ a different but conceptually related idea; independent agents simultaneously coordinating their own behaviors in parallel environments while asynchronously updating the policy of a system-or, rather, meta-level agent. This work, to the best of the authors' knowledge, is the first to explicitly explore the potential relationship between the underlying concepts in homogeneous decentralized control for articulated locomotion and distributed learning. We present an approach that leverages the structure of the asynchronous advantage actor-critic (A3C) algorithm to provide a natural framework for learning decentralized control policies on a single platform. Our primary contribution shows an individual agent in the A3C algorithm can be defined by an independently controlled portion of the robot's body, thus enabling distributed learning on a single platform for efficient hardware implementation. To this end, we show how the system is trained offline using hardware experiments implementing an autonomous decentralized compliant control framework. Our experimental results show that the trained agent outperforms the compliant control baseline by more than 40% in terms of steady progression through a series of randomized, highly cluttered evaluation environments.