Recent robotic task planning frameworks have integrated large multimodal models (LMMs) such as GPT-4o. To address grounding issues of such models, it has been suggested to split the pipeline into perceptional state grounding and subsequent state-based planning. As we show in this work, the state grounding ability of LMM-based approaches is still limited by weaknesses in granular, structured, domain-specific scene understanding. To address this shortcoming, we develop a more structured state grounding framework that features a domain-conditioned scene graph as its scene representation. We show that such representation is actionable in nature as it is directly mappable to a symbolic state in planning languages such as the Planning Domain Definition Language (PDDL). We provide an instantiation of our state grounding framework where the domain-conditioned scene graph generation is implemented with a lightweight vision-language approach that classifies domain-specific predicates on top of domain-relevant object detections. Evaluated across three domains, our approach achieves significantly higher state grounding accuracy and task planning success rates compared to LMM-based approaches. https://github.com/Vision-Kek/DC-SGG

This paper introduces an optimisation-based trajectory deformation and tracking algorithm for tethered differential-driven mobile robots. The motivation of this work is to generate self-entanglement-free (SEF) commands for a tethered differential-driven robot to track a path. Whilst existing path planners have been capable of generating SEF paths for tethered differential-driven robots lacking an omni-directional tether retracting mechanism, no trajectory planner can handle the unavoidable movement errors that cause robot pose deviate from the pre-defined path. The trajectory deformation and tracking is challenging because the admissible heading direction of the robot is highly constrained by the SEF constraint. As a result, even with an SEF path, the robot still encounters self-entanglement issues during execution.This paper fills this gap by formulating the trajectory deforming and tracking (TDT) problem of a tethered robot into a multi-objective optimisation framework. Explicit consideration of the constraint of the relative angle between the tether stretching direction and the robot’s heading direction to be admissible during its movement is provided in this framework. The proposed algorithm repeatedly deforms the pre-defined path for easier tracking, whilst generating a suitable velocity profile for robot execution. Compared to directly applying the commonly used untethered trajectory deformation and tracking algorithm into tethered cases, the proposed algorithm demonstrates improved performance in terms of minimising the risk of self-entanglement and maximising robot safety. These are validated in both simulated and real scenarios. An open-sourcesourcing implementation has also been provided for the benefit of the robotics community.

A novel mechanism to derive self-entanglement-free path for tethered differential-driven robots is proposed in this work. The problem is tailored to the applications of tethered robots without an omni-directional tether re-tractor which is often encountered when an omni-directional tether retracting mechanism is incapable of being jointly equipped with other geometrically complex devices (e.g. a manipulator), for instance the disaster recovery, spatial exploration, etc. Without a special consideration on the spatial relation between the pose of the mobile base and the tether, self-entanglement appears when the robot moves, resulting in unsafe motion of the robot and potential damage to the tether. In this paper, the self-entanglement-free constraint is modelled as the admissible orientation of the tether anchoring on the robot with respect to the robot's heading orientation. A searching-based path planning algorithm is then proposed to generate a near optimal path solution with guaranteed null of tether self-entanglement. The effectiveness of the proposed algorithm is compared with the motions without considering self-entanglement-free constraint, illustrated in challenging planning cases, and validated in realworld scenes. An open-source implementation has also been provided for the benefit of the robotics community.