Observing that the key for robotic action planning is to understand the target-object motion when its associated part is manipulated by the end effector, we propose to generate the 3D object-part scene flow and extract its transformations to solve the action trajectories for diverse embodiments. The advantage of our approach is that it derives the robot action explicitly from object motion prediction, yielding a more robust policy by understanding the object motions. Also, beyond policies trained on embodiment-centric data, our method is embodiment-agnostic, generalizable across diverse embodiments, and being able to learn from human demonstrations. Our method comprises three components: an object-part predictor to locate the part for the end effector to manipulate, an RGBD video generator to predict future RGBD videos, and a trajectory planner to extract embodiment-agnostic transformation sequences and solve the trajectory for diverse embodiments. Trained on videos even without trajectory data, our method still outperforms existing works significantly by 27.7% and 26.2% on the prevailing virtual environments MetaWorld and Franka-Kitchen, respectively. Furthermore, we conducted real-world experiments, showing that our policy, trained only with human demonstration, can be deployed to various embodiments.

Skill determination aims to evaluatehow wella participant performs a specific action. The task is rather challenging, due to the diversity of action types and the scarcity of samples. Many existing works train a skill determination model on limited samples of each action type separately. However, they neglect the skill similarities shared by different action types that can be exploited to enhance the skill determination process. How to exploit useful assessment skills from source actions to a related target action remains a challenge, and existing works have not ever found an effective way to accomplish this. In this work, we propose to achieve skill transfer for action assessment by anAdaptive Stage-aware AssessmentSkillTransfer framework (AdaST) that transfers assessment skills from source actions to different stages of a target action adaptively. A source action search scheme is proposed to select relevant source actions for each target action. Furthermore, to encourage transferring effective and non-redundant assessment skills, a consistency loss and an orthogonality loss are introduced to ensure that the transferred assessment skills do not degrade the accurate determination and it provides complementary information. Extensive experiments on three public datasets demonstrate the effectiveness of the proposed method.

Robotic bin packing is very challenging, especially when considering practical needs such as object variety and packing compactness. This paper presents SDF-Pack, a new approach based on signed distance field (SDF) to model the geometric condition of objects in a container and compute the object placement locations and packing orders for achieving a more compact bin packing. Our method adopts a truncated SDF representation to localize the computation, and based on it, we formulate the SDF -minimization heuristic to find optimized placements to compactly pack objects with the existing ones. To further improve space utilization, if the packing sequence is controllable, our method can suggest which object to be packed next. Experimental results on a large variety of everyday objects show that our method can consistently achieve higher packing compactness over 1,000 packing cases, enabling us to pack more objects into the container, compared with the existing heuristics under various packing settings. The code is publicly available at: https://github.com/kwpoon/SDF-Pack.