Vision-based grasping, though widely employed for industrial and household applications, still struggles with object stacking scenarios. Current methods face three major challenges: limited inter-object relationship understanding; poor grasping adaptation across different viewpoints; and error propagation. To address the above challenges, we propose D2TriPO-DETR, a dual-decoder transformer with three outputs, of which are object detection, manipulation relationship, and grasp detection. Specifically, a distributed attention perception module and a rotation attention invariance module are designed to address limited interobject relationship understanding and poor grasping adaptation across different viewpoints. These two modules are respectively integrated into the two parallel decoders to output the triple results simultaneously, partly eliminating task-level error propagation. Experimental results on the visual manipulation relationship dataset indicate that D2TriPO-DETR outperforms existing state-of-the-art methods across all metrics, e.g., +6.1% object detection recall, +6.7% manipulation relationship image accuracy, and +1.5% grasp detection accuracy. Extensive real-world experiments and quantitative results validate D2TriPO-DETR’s effectiveness.

Deformable linear object (DLO) manipulation is needed in many fields. Previous research on deformable linear object (DLO) manipulation has primarily involved parallel jaw gripper manipulation with fixed grasping positions. However, the potential for dexterous manipulation of DLOs using an anthropomorphic hand is under-explored. We present DexDLO, a model-free framework that learns dexterous dynamic manipulation policies for deformable linear objects with a fixed-base dexterous hand in an end-to-end way. By abstracting several common DLO manipulation tasks into goal-conditioned tasks, DexDLO can perform tasks such as DLO grabbing, DLO pulling, DLO end-tip position controlling, etc. Using the Mujoco physics simulator, we demonstrate that our framework can efficiently and effectively learn five different DLO manipulation tasks with the same framework parameters. We further provide a thorough analysis of learned policies, reward functions, and reduced observations for a comprehensive understanding of the framework.

Developing physically assistive robots capable of dressing assistance has the potential to significantly improve the lives of the elderly and disabled population. However, most robotics dressing strategies considered a single robot only, which greatly limited the performance of the dressing assistance. In fact, healthcare professionals perform the task bimanually. Inspired by them, we propose a bimanual cooperative scheme for robotic dressing assistance. In the scheme, an interactive robot joins hands with the human thus supporting/guiding the human in the dressing process, while the dressing robot performs the dressing task. We identify a key feature: elbow angle that affects the dressing action and propose an optimal strategy for the interactive robot using the feature. A dressing coordinate based on the posture of the arm is defined to better encode the dressing policy. We validate the interactive dressing scheme with extensive experiments and also an ablation study. The experiment video is available onhttps://sites.google.com/view/bimanualassitdressing/home

Deforming a cable to a desired (reachable) shape is a trivial task for a human to do without even knowing the internal dynamics of the cable. This paper proposes a framework for cable shapes manipulation with multiple robot manipulators. The shape is parameterized by a Fourier series. A local deformation model of the cable is estimated on-line with the shape parameters. Using the deformation model, a velocity control law is applied on the robot to deform the cable into the desired shape. Experiments on a dual-arm manipulator are conducted to validate the framework.