The ability to accomplish manipulation and locomotion tasks in the presence of significant time-varying external loads is a remarkable skill of humans that has yet to be replicated convincingly by humanoid robots. Such an ability will be a key requirement in the environments we envision deploying our robots: dull, dirty, and dangerous. External loads constitute a large model bias, which is typically unaccounted for. In this work, we enable our humanoid robot to engage in loco-manipulation tasks in the presence of significant model bias due to external loads. We propose an online estimation and control framework involving the combination of a physically consistent extended Kalman filter for inertial parameter estimation coupled to a whole-body controller. We showcase our results both in simulation and in hardware, where weights are mounted on Nadia’s wrist links as a proxy for engaging in tasks where large external loads are applied to the robot.

For achieving significant levels of autonomy, legged robot behaviors require perceptual awareness of both the terrain for traversal, as well as structures and objects in their surroundings for planning, obstacle avoidance, and high-level decision making. In this work, we present a perception engine for legged robots that extracts the necessary information for developing semantic, contextual, and metric awareness of their surroundings. Our custom sensor configuration consists of (1) an active depth sensor, (2) two monocular cameras looking sideways, (3) a passive stereo sensor observing the terrain, (4) a forward facing active depth camera, and (5) a rotating 3D LIDAR with a large vertical field-of-view (FOV). The mutual overlap in the sensors' FOVs allows us to redundantly detect and track objects of both dynamic and static types. We fuse class masks generated by a semantic segmentation model with LIDAR and depth data to accurately identify and track individual instances of dynamically moving objects. In parallel, active depth and passive stereo streams of the terrain are also fused to map the terrain using the on-board GPU. We evaluate the engine using two different humanoid behaviors, (1) look-and-step and (2) track-and-follow, on the Boston Dynamics Atlas.

Legged robots require fast and accurate representation of their surrounding terrain to achieve behaviors such as running, push recovery, continuous walking, backflips, while also utilizing on-board computational resources efficiently. The desired tasks can be achieved efficiently by representing the environment using planar regions. However, existing methods for planar region extraction are either too slow or require significant compute time on the Central Processing Unit (CPU). In this work we exploit key properties of depth images and Graphical Processing Unit (GPU) to estimate planar regions around the robot at very high frame rates of 150-200 Hz. The proposed algorithm uses a set of fully customizable and interchangeable set of kernel layers on the GPU to process the depth map in parallel and generate a locally connected graph structure, which is later separated into planar components using a basic depth-first search. We test the proposed algorithm on the Atlas robot while performing different walking behaviors on oriented cinder blocks, as well as in simulation with simulated sensor and robot. The algorithm is open-sourced for research on legged robots and other fields.