Recognizing failures during task execution and implementing recovery procedures is challenging in robotics. Traditional approaches rely on the availability of extensive data or a tight set of constraints, while more recent approaches leverage large language models (LLMs) to verify task steps and replan accordingly. However, these methods often operate offline, necessitating scene resets and incurring in high costs. This paper introduces Recover, a neuro-symbolic framework for online failure identification and recovery. By integrating ontologies, logical rules, and LLM-based planners, Recover exploits symbolic information to enhance the ability of LLMs to generate recovery plans and also to decrease the associated costs. In order to demonstrate the capabilities of our method in a simulated kitchen environment, we introduce OntoThor, an ontology describing the AI2Thor simulator setting. Empirical evaluation shows that OntoThor’s logical rules accurately detect all failures in the analyzed tasks, and that Recover considerably outperforms, for both failure detection and recovery, a baseline method reliant solely on LLMs. Supplementary material, including the OntoThor ontology, is available at: https://recover-ontothor.github.io.

Abstract Trustworthy human–robot interaction (HRI) during activities of daily living (ADL) presents an interesting and challenging domain for assistive robots, particularly since methods for estimating the trust level of a human participant towards the assistive robot are still in their infancy. Trust is a multifaced concept which is affected by the interactions between the robot and the human, and depends, among other factors, on the history of the robot’s functionality, the task and the environmental state. In this paper, we are concerned with the challenge of trust transfer, i.e. whether experiences from interactions on a previous collaborative task can be taken into consideration in the trust level inference for a new collaborative task. This has the potential of avoiding re-computing trust levels from scratch for every new situation. The key challenge here is to automatically evaluate the similarity between the original and the novel situation, then adapt the robot’s behaviour to the novel situation using previous experience with various objects and tasks. To achieve this, we measure the semantic similarity between concepts in knowledge graphs (KGs) and adapt the robot’s actions towards a specific user based on personalised interaction histories. These actions are grounded and then verified before execution using a geometric motion planner to generate feasible trajectories in novel situations. This framework has been experimentally tested in human–robot handover tasks in different kitchen scene contexts. We conclude that trust-related knowledge positively influences and improves collaboration in both performance and time aspects.

The execution of robotic manipulation tasks needs to be robust in front of failures or changes in the environment, and for this purpose, Behavior Trees (BT) are a good alternative to Finite State Machines, because the ability of BTs to be edited during run time and the fact that one can design reactive systems with BTs, makes the BT executor a robust execution manager. However, the good monitoring of the system state is required in order to react to errors at either geometric or symbolic level requiring, respectively, replanning at motion or at task level. This paper make a proposal in this line and, moreover, makes task planning adaptive to the actual situations encountered by knowledge-based reasoning procedures to automatically generate the Planning Domain Definition Language (PDDL) files that define the task.

Robots should be able to exercise reasoning in both symbolic and geometric levels in order to plan a manipulation task. The execution of such tasks needs to be robust enough to cope with real environments. In an attempt to address this pertinent industry need, the paper proposes the use of behavior trees for effective robotic manipulation in dynamic environments. This paper presents a method to automatically generate a behavior tree and showcases its ability to enable the robot to reason at different levels and adapt to an uncertain and changing environment. This allows for a complex task to be robustly executed, pioneering the advancement towards fully functional service robots.