This work presents a motion retargeting approach for legged robots, aimed at transferring the dynamic and agile movements to robots from source motions. In particular, we guide the imitation learning procedures by transferring motions from source to target, effectively bridging the morphological disparities while ensuring the physical feasibility of the target system. In the first stage, we focus on motion retargeting at the kinematic level by generating kinematically feasible whole-body motions from keypoint trajectories. Following this, we refine the motion at the dynamic level by adjusting it in the temporal domain while adhering to physical constraints. This process facilitates policy training via reinforcement learning, enabling precise and robust motion tracking. We demonstrate that our approach successfully transforms noisy motion sources, such as hand-held camera videos, into robot-specific motions that align with the morphology and physical properties of the target robots. Moreover, we demonstrate terrain-aware motion retargeting to perform BackFlip on top of a box. We successfully deployed these skills to four robots with different dimensions and physical properties in the real world through hardware experiments.

For multi-limbed robots, motion planning with posture and force constraints tends to be a difficult optimization problem due to nonlinearities, which also present extended solve times. We propose a multi-stage optimization framework with data-driven inter-stage coupling constraints to address the nonlinearity. Both clustering and evolutionary approaches to find the McCormick envelope relaxations are used to find the problem-specific parameters. The learned constraints are then used in the prior stages, which provides advanced knowledge of the following stages. This leads to improved solve times and interpretability of the results. The planner is validated through multiple walking and climbing tasks on a 10 kg hexapod robot.

In this paper, we introduce the Surface Material Dataset for Robotics Applications (SMDRA), a collection of RGB color image, depth data, and pixel-wise friction coefficient data of 10 different materials for computer vision research specifically with robotics applications in mind that require physical contact between the robot and its environment such as robotic manipulators or walking robots. These selected surface materials are both easily accessible around our daily lives and cover a wide range of friction coefficients. Our dataset is unique in that while there is an abundance of RGB-D data due to the popularization of imaging sensors, additional pixel-wise aligned data of a different modality are not readily available. The depth data is collected by an active stereo camera which has shown promise on a variety of different robotic applications, especially in outdoor environments. In addition, this dataset is greatly expanded with the addition of friction coefficient data. Similarly to humans, this additional information can be helpful in ensuing proper decision making in tasks ranging from grasping orientation and strength to path determination in an unstructured environment. A newly developed friction measuring device was used to obtain this data. We verify that existing Convolutional Neural Network (CNN) architectures, the Fully Convolutional Network (FCN) and U-Net, can be trained on the SMDRA. This result demonstrates that the SMDRA can be utilized to train a neural network model for segmentation and these different modes are not just additional information, but valuable modes that researchers can incorporate and exploit when applying computer vision algorithms on robotic platforms.

This paper presents a complete motion planning approach for quadruped locomotion across an unknown terrain using a framework based on mixed-integer convex optimization and visual feedback. Vision data is used to find convex polygons in the surrounding environment, which acts as potentially feasible foothold regions. Then, a goal position is initially provided, which the best feasible destination planner uses to solve for an actual feasible goal position based on the extracted polygons. Next, a footstep planner uses the feasible goal position to plan a fixed number of footsteps, which may or may not result in the robot reaching the position. The center of mass (COM) trajectory planner using quadratic programming is extended to solve for a trajectory in 3D space while maintaining convexity, which reduces the computation time, allowing the robot to plan and execute motions online. The suggested method is implemented as a policy rather than a path planner, but its performance as a path planner is also shown. The approach is verified on both simulation and on a physical robot, ALPHRED, walking on various unknown terrains.

Major rule changes for the RoboCup Humanoid League in 2015 pose significant vision and locomotion challenges for disambiguating similarly colored objects and navigating soft terrain. These significant changes highlight the need for applying general purpose humanoid robotics approaches that can handle abrupt environment modifications, and we utilize the general purpose THOR (Tactical Hazardous Operations Robot) series of robot from the recent DARPA Robotics Challenge (DRC). Specific techniques for vision, kicking and autonomy complement software developed for robust deployments in the DRC. In this paper, we present these soccer playing techniques, which were validated in the Humanoid AdultSize league in Hefei. These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.