We introduce an open-source ROS 2 architecture for situated social robots, along with a simulator that allows mixed-reality development and interactions. The architecture is a hybrid symbolic/subsymbolic system that integrates explicit ontology semantics for perception, reasoning, and execution, with LLMs. It features multimodal social perception by leveraging the open source ROS4HRI framework; LLMs (both edge- and cloud-based) to facilitate natural language interaction between the user and system; KnowledgeCore, an open-source knowledge base, to reason about facts in the world; and an intent-based controller to supervise the execution of parallel/sequential tasks and skills. We demonstrate our system architecture with a social robot running the mixed-reality system.

In the field of Human-Robot Interaction (HRI), a key challenge lies in enabling humans to comprehend the decisions and behaviours of robots. One promising approach involves leveraging Theory of Mind (ToM) frameworks, wherein a robot estimates the mental model that a user holds about its functioning and compares this with the representation of its internal mental model. This comparison allows the robot to identify potential mismatches and generate communicative actions to bridge such gaps. Effective communication requires the robot to maintain unique mental models for each user and personalise explanations based on past interactions. To address this, we propose an architecture grounded in Large Language Models (LLMs) that operationalises this theoretical framework. We demonstrate the feasibility of this approach through qualitative examples, showcasing responses provided by a robot patrolling a geriatric hospital.

This tutorial aims at providing an up-to-date picture of the state-of-art regarding using the Robot Operating System (ROS) to build robots with socio-cognitive capabilities. The tutorial will briefly introduce the ROS4HRI framework, and show how it can be used to build a complete social robot architecture, from human perception to expressive social interaction. We will illustrate the full software integration required to implement an autonomous social robot using a combination of open-source ROS-based social perception modules, a semantic knowledge base, a Large Language Model (LLM), and multi-modal expressiveness. Participants will be able to follow along using a simple social interaction simulator, as well as their own webcams. The organisers will also provide a new PAL Robotics TIAGo Pro stand-alone head to demonstrate the same system running on actual hardware.

This paper presents the TIAGo Head, a new tabletop social robot from PAL Robotics, focusing on its capabilities as an HRI platform. We detail the robots’ hardware, highlighting its sensors/actuators and on-board computing; and its software architecture, including social perception, expressive face, a knowledge base, and integration with large language models (LLMs) for natural conversations. We also describe a use-case in a receptionist scenario where TIAGo Head dynamically interacts with travelers by displaying news and conversing.

In the field of Human-Robot Interaction (HRI), a fundamental challenge is to facilitate human understanding of robots. The emerging domain of eXplainable HRI (XHRI) investigates methods to generate explanations and evaluate their impact on human-robot interactions. Previous works have highlighted the need to personalise the level of detail of these explanations to enhance usability and comprehension. Our paper presents a framework designed to update and retrieve user knowledge-memory models, allowing for adapting the explanations’ level of detail while referencing previously acquired concepts. Three architectures based on our proposed framework that use Large Language Models (LLMs) are evaluated in two distinct scenarios: a hospital patrolling robot and a kitchen assistant robot. Experimental results demonstrate that a two-stage architecture, which first generates an explanation and then personalises it, is the framework architecture that effectively reduces the level of detail only when there is related user knowledge.

In social robots, visual attention expresses awareness of the scenario components and dynamics. As in humans, their attention should be driven by a combination of different attention mechanisms. In this article, we introduce multi-modal saliency maps, i.e., spatial representations of saliency that dynamically integrate multiple attention sources depending on the context. We provide the mathematical formulation of the model and an open source software implementation. Finally, we present an initial exploration of its potential in social interaction scenarios with humans and evaluate its implementation.

While Automatic Speech Recognition (ASR) systems excel in controlled environments, challenges arise in robot-specific setups due to unique microphone requirements and added noise sources. In this paper, we create a dataset of initiating conversations with brief exchanges in 5 European languages, and we systematically evaluate current state-of-art ASR systems (Vosk, OpenWhisper, Google Speech and NVidia Riva). Besides standard metrics, we also look at two critical downstream tasks for human-robot verbal interaction: intent recognition rate and entity extraction, using the open-source Rasa chatbot. Overall, we found that open-source solutions as Vosk performs competitively with closed-source solutions while running on the edge, on a low compute budget (CPU only).

Many research works currently focus on algorithms designed to generate explanations and then evaluate their effect on user trust and understanding of robots. Even though some projects attempt to design understandable interfaces, they usually serve as extra features for solutions that already exist. In this paper, we suggest a user-centric approach to design explainable robot systems from the very beginning. In particular, we provide a participatory design approach that places emphasis on the cooperative design of an understandable and intuitive interface between the user and the robot system. We suggest turning the attention to the robot’s functionality and autonomous behaviours development after this interface has been established. We exemplify how to apply the proposed framework in a geriatric unit at an intermediate care centre.

With the increase in life expectancy and staff shortage, there is an urgency to understanding the needs of older adults and exploring emerging fields such as social robotics to tackle the challenges of ageing. The paper highlights the importance of providing cognitive support, physical support, reducing loneliness and increasing social engagement among older adults as well as reducing caregiver burden, and suggests that socially assistive robots (SAR) can assist older adults and their carers with such needs. However, the paper also points out that there are several challenges associated with designing and deploying SAR systems, and involving end-users in the design process is necessary to improve user acceptability and adoption. The paper describes the approaches used by PAL Robotics to facilitate real-world deployment of its ARI and TIAGo social robots, and provides examples of how these robots have been used to tackle different healthcare needs.

Abstract Empathy plays a fundamental role in building relationships. To foster close relationships and lasting engagements between humans and robots, empathy models can provide direct clues into how it can be done. In this study, we focus on capturing in a quantitative way indicators of early empathy realization between a human and a robot using a process that encompasses affective attachment, trust, expectations and reflecting on the other’s perspective within a set of collaborative strategies. We hypothesize that an active collaboration strategy is conducive to a more meaningful and purposeful engagement of realizing empathy between a human and a robot compared to a passive one. With a deliberate design, the interaction with the robot was presented as a maze game where a human and a robot must collaborate in order to reach the goal using two strategies: one maintaining control individually taking turns (passive strategy) and the other one where both must agree on their next move based on reflection and argumentation. Quantitative and qualitative analysis of the pilot study confirmed that a general sense of closeness with the robot was perceived when applying the active strategy. Regarding the specific indicators of empathy realization: (1) affective attachment , affective emulation was equally present throughout the experiment in both conditions, and thus, no conclusion could be reached; (2) trust , quantitative analysis partially supported the hypothesis that an active collaborative strategy will promote teamwork attitudes, where the human is open to the robot’s suggestions and to act as a teammate; and (3) regulating expectation , quantitative analysis confirmed that a collaborative strategy promoted a discovery process that regulates the subject’s expectation toward the robot. Overall, we can conclude that an active collaborative strategy impacts favorably the process of realizing empathy compared to a passive one. The results are compelling to move the design of this experiment forward into more comprehensive studies, ultimately leading to a path where we can clearly study engagements that reduce abandonment and disillusionment with the process of realizing empathy as the core design for active collaborative strategies.

Current trends envisage an evolution of collaboration, engagement, and relationship between humans and devices, intelligent agents and robots in our everyday life. Some of the key elements under study are affective states, motivation, trust, care, and empathy. This paper introduces an empathy test-bed that serves as a case study for an existing empathy model. The model describes the steps that need to occur in the process to provoke meaning in empathy, as well as the variables and elements that contextualise those steps. Based on this approach we have developed a fun collaborative scenario where a user and a social robot work together to solve a maze. A set of exploratory trials are carried out to gather insights on how users perceive the proposed test-bed around attachment and trust, which are basic elements for the realisation of empathy.

Social robots have the potential to provide support in a number of practical domains, such as learning and behaviour change. This potential is particularly relevant for children, who have proven receptive to interactions with social robots. To reach learning and therapeutic goals, a number of issues need to be investigated, notably the design of an effective child-robot interaction (cHRI) to ensure the child remains engaged in the relationship and that educational goals are met. Typically, current cHRI research experiments focus on a single type of interaction activity (e.g. a game). However, these can suffer from a lack of adaptation to the child, or from an increasingly repetitive nature of the activity and interaction. In this paper, we motivate and propose a practicable solution to this issue: an adaptive robot able to switch between multiple activities within single interactions. We describe a system that embodies this idea, and present a case study in which diabetic children collaboratively learn with the robot about various aspects of managing their condition. We demonstrate the ability of our system to induce a varied interaction and show the potential of this approach both as an educational tool and as a research method for long-term cHRI.

We report first results on children adaptive behavior towards a dance tutoring robot. We can observe that children behavior rapidly evolves through few sessions in order to accommodate with the robotic tutor rhythm and instructions.

The ALIZ-E1 project's goal is to design a robot companion able to maintain affective interactions with young users over a period of time. One of these interactions consists in teaching a dance to hospitalized children according to their capabilities. We propose a methodology for adapting both, the movements used in the dance based on the user's cognitive and physical capabilities through a set of metrics, and the robot's interaction based on the user's personality traits.

We present insights gleaned from a series of child-robot interaction experiments carried out in a hospital paediatric department. Our aim here is to share good practice in experimental design and lessons learned about the implementation of systems for social HRI with child users towards application in "the wild", rather than in tightly controlled and constrained laboratory environments: a trade-off between the structures imposed by experimental design and the desire for removal of such constraints that inhibit interaction depth, and hence engagement, requires a careful balance.

While key for human-robot interaction, natural language interpretation is a notoriously difficult task, especially because the interaction context is at the same time essential for dialogue understanding, difficult to build for machines, and depends on each speaker point of view. However, robots as embodied artifacts, can perceive their environment and interactors, and hence compute symbolic models from various perspectives. This allows in turn to build symbolic contexts for dialogues. In this paper, we introduce DIALOGS, a component for natural language interpretation that relies on these structured symbolic models of the world to ground verbal interaction.

In daily human interactions spatial reasoning occupies an important place. With this ability we can build relations between objects and people, and we can predict the capabilities and the knowledge of the people around us. An interactive robot is also expected to have these abilities in order to establish an efficient and natural interaction. In this paper we present a situation assessment reasoner, based on spatial reasoning and perspective taking, which generates on-line relations between objects and agents in the environment. Being fully integrated to a complete architecture, this reasoner sends the generated symbolic knowledge to a fact data base which is built on the basis on an ontology and which is accessible to the entire system. This work is also part of a broader effort to develop a complete decisional framework for human-robot interactive task achievement.

Humans constantly generate and solve ambiguities while interacting with each other in their every day activities. Hence, having a robot that is able to solve ambiguous situations is essential if we aim at achieving a fluent and acceptable human-robot interaction. We propose a strategy that combines three mechanisms to clarify ambiguous situations generated by the human partner. We implemented our approach and successfully performed validation tests in several different situations both, in simulation and with the HRP-2 robot.

This paper presents an embeddable knowledge processing framework, along with a common-sense ontology, designed for robotics. We believe that a direct and explicit integration of cognition is a compulsory step to enable human-robots interaction in semantic-rich human environments like our houses. The OpenRobots Ontology (ORO) kernel allows to turn previously acquired symbols into concepts linked to each other. It enables in turn reasoning and the implementation of other advanced cognitive functions like events, categorization, memory management and reasoning on parallel cognitive models. We validate this framework on several cognitive scenarii that have been implemented on three different robotic architectures.

In human-robot interaction, a robot must be prepared to handle possible ambiguities generated by a human partner. In this work we propose a set of strategies that allow a robot to identify the referent when the human partner refers to an object giving incomplete information, i.e. an ambiguous description. Moreover, we propose the use of an ontology to store and reason on the robot's knowledge to ease clarification, and therefore, improve interaction. We validate our work through both simulation and two real robotic platforms performing two tasks: a daily-life situation and a game.