Imitation learning by watching humans offers a promising path to learning general-purpose robot skills with intuitive task specifications. While prior approaches for video-conditioned imitation learning directly extract skill embeddings from unstructured human videos and follow demonstrations step-by-step, such paradigm usually falls short in generalizing to unseen long-horizon tasks with a single human prompt video due to the significant embodiment and environment gap. To this end, our key insight is to infer local intentions from videos in order to retrieve robot skill memories from prior experience, and conversely select the feasible video clip to follow based on robot observations. Motivated by this, we introduce AdaMimic, a hierarchical imitation learning method that learns the bidirectional mapping of cross-domain sensorimotor skills and derives skill-based policy conditioned on adaptable latent plans. To enable generalization to unseen tasks given cross-domain human videos, AdaMimic leverages task-agnostic play data for interaction-aware skill embedding extraction and video-robot trajectory pairs for semantic and temporal human-to-robot skill alignment. In addition, our method exploits a skill adapter for robot-to-human alignment to adaptively align the robot with the skill intentions. We systematically evaluate AdaMimic on both simulated and real-world kitchen domains, demonstrating AdaMimic’s superiority over prior imitation learning methods in generalizing to novel long-horizon tasks with a single human prompt video.

For a general-purpose robot, it is desirable to imitate human demonstration videos that can effectively solve long-horizon tasks and perform novel ones. Recent advances in skill-based imitation learning have shown that extracting skill embedding from raw human videos is a promising paradigm to enable robots to cope with long-horizon tasks. However, generalization to unseen tasks in a different domain with a human prompt video poses a significant challenge due to the big embodiment and environment difference. To this end, we present Hierarchical Human-to-Robot Imitation Learning (H2RIL) that learns the mapping of cross-domain sensorimotor skills and utilizes it to generalize to unseen tasks given a human video in a different environment. To allow for generalizing zero-shot across environments and embodiments, H2RIL leverages task-agnostic play data for low-level policy training and paired human-robot data for both semantic and temporal skill embedding alignment. Extensive experiments in a simulated kitchen environment demonstrate that H2RIL significantly outperforms other prior baselines and is capable of generalizing to composable new tasks and adapting to Out-of-Distribution (OOD) tasks.

Critics are applied to estimating policy gradients in the actor-critic framework, an essential part of Reinforcement Learning methods. An appropriate critic is supposed to balance variance from sample returns and bias introduced by parameterized value functions. A typical critic of balancing variance and bias, the generalized advantage estimator (GAE), combines sample returns and value functions with a fixed weight parameter. However, such a parameter is hard to fit, which results in no promised stability for GAE. In this paper, indicators of variance and bias are proposed to get adaptive weight parameters, with which adaptive advantage estimators are obtained. Empirical results on both 2D and 3D simulated robotic locomotion tasks show that the adaptive advantage estimators achieve similar or superior performance compared to GAE.