While Language Models have shown promising performance, they still struggle with limitations regarding reasoning and are very token-sensitive. In contrast, knowledge-based systems, such as ontologies, allow for provable logically valid reasoning and provide explicit justifications regarding newly inferred knowledge. However, those justifications can be hard to understand for non-expert users given their formal syntax and their length. We investigated if language models could be considered as reliable tools for verbalizing such explanations, thus increasing explainability over reasoning output. This paper presents a reference evaluation of a set of embeddable language models on a task of translation from rule-based ontology formatted inferences and justifications into natural language sentences. We show that the order of justifications significantly decreases performance, whereas adding the inference rule as additional context significantly improves performance, leading to more reliable results.

When collaborating with humans during a shared task, a robot must be able to estimate the shared goal and monitor the tasks completed by its partners to adapt its behavior. Our contribution is a lightweight, hierarchical task recognition system that enables the robot to estimate shared goals and monitor human tasks. This recognition system is integrated into a robotic architecture to take advantage of the semantic knowledge flow available and builds upon our previous work on action recognition. We demonstrate the mechanisms of our recognition system and how we improved it to handle missing information using a kitchen scenario. This also enables us to showcase its usability from the perspective of other agents, using Theory of Mind.

In collaborative tasks, expectations for achieving shared goals arise at all hierarchical plan levels, including plans, tasks, subtasks, and actions. However, these expectations also generate uncertainties for individuals executing the joint plan. If left unresolved, these uncertainties can impede successful task completion. Uncertainties may relate to the agents’ motivation to initiate, continue, or complete their plan (motivational uncertainty), the best way to execute their shared plan (instrumental uncertainty), and their knowledge of other agents and the environment (common ground uncertainty). These expectations can be either normative or descriptive, but only normative expectations trigger reactions from agents to resolve the aforementioned types of uncertainties. Thus, this paper introduces a theoretical model that enables a robot to consider all agents’ expectations and take actions that reduce the uncertainties associated with their shared plan. By doing so, we aim to enhance the likelihood of success in joint plans between robots and humans. To demonstrate the effectiveness of our theoretical commitment model, we have implemented a proof of concept for a client service use case in a food shop.

Being able to refer to an object, a person, or a place in a non-ambiguous manner is a need when one has to achieve collaborative activities with a partner. This is the so-called Referring Expression Generation (REG) problem. While widely used for Human-Robot Interaction, state of the art approaches restrict its use to the current environment. We propose a novel extension to the REG which takes full advantage of the Human-Robot shared knowledge about past actions as additional information to generate Referring Expressions. We show that our approach is usable with a domain-independent ontology as a knowledge base and that it can also use a semantic representation of past activity to generate RE. We illustrate our method through simulated situations and discuss its efficiency and pertinence.

Assessing robotic architecture for Human-Robot Interaction can be challenging due to the number of features a robot has to endow to perform an acceptable interaction. While everyday-inspired tasks are interesting as reflecting a realistic use of such robots, they often contain a lot of unknown and uncontrolled conditions and specific robot behavior can be hard to test. In this paper, we propose a new psychology-inspired task, gathering perspective-taking, planning, knowledge representation with theory of mind, manipulation, and communication. Along with a precise description of the task allowing its replication, we present a cognitive robot architecture able to perform it in its nominal cases. We finally suggest some challenges and evaluations for the Human-Robot Interaction research community, all derived from this easy-to-replicate task.

In Human-Robot Interaction (HRI), ensuring nonambiguous communication between the robot and the human is a key point for carrying out fluently a collaborative task. With this work, we propose a method which allows the robot to generate the optimal set of assertions that are necessary in order to produce an unambiguous reference. In this paper, we present a novel approach to the Referring Expression Generation (REG) problem and its integration into a robotic system. Our method is a domain-independent approach based on an ontology as a knowledge base. We show how this generation can be performed on an ontology which is not dedicated to this task. We then validate our method through simulated situations, compare it with state of the art approach and on a real robotic system.

In this paper we present Ontologenius, a semantic knowledge storage and reasoning framework for autonomous robots. More than a classic ontology software to query a knowledge base and a first-order internal logic as it can be done for web-semantics, we propose with Ontologenius features adapted to a robotic use including human-robot interaction. We introduce the ability to modify the knowledge base during execution, whether through dialogue or geometric reasoning, and keep these changes even after the robot is powered off. Since Ontologenius was developed to be used by a robot which interacts with humans, we have endowed the system with ability to perform attributes and properties generalization and with the possibility to model and estimate the semantic memory of a human partner and to implement theory of mind processes. This paper presents the architecture and the main features of Ontologenius as well as examples of its use in robotics applications.