In this paper, we critically examine the current "humanoid hype" in robotics, questioning its alignment with responsible robotics principles. While technical challenges drive internal fascination, the pervasive public image of humanoids demands deeper HRI engagement. We explore how responsible robotics concepts, such as privacy, dignity, and trust, are uniquely challenged or overlooked in the pursuit of anthropomorphic robot forms. By dissecting this hype, and mapping the main findings of the recently-published Roadmap for Responsible Robotics to the humanoids field, we aim to move beyond technical form-factor obsessions to understand the true societal implications and identify potential blind spots for the HRI community.

We introduce an open-source ROS 2 architecture for situated social robots, along with a simulator that allows mixed-reality development and interactions. The architecture is a hybrid symbolic/subsymbolic system that integrates explicit ontology semantics for perception, reasoning, and execution, with LLMs. It features multimodal social perception by leveraging the open source ROS4HRI framework; LLMs (both edge- and cloud-based) to facilitate natural language interaction between the user and system; KnowledgeCore, an open-source knowledge base, to reason about facts in the world; and an intent-based controller to supervise the execution of parallel/sequential tasks and skills. We demonstrate our system architecture with a social robot running the mixed-reality system.

This paper presents a new approach for representing spatially-grounded semantics in interactive robots. The method combines spatial and symbolic data to improve robot interactions in human-occupied environments. A key feature is a voxel-based data structure optimized for dynamic and sparse information, along with a global lookup table to manage and track spatially-grounded entities and their relationships. The implementation, which is integrated into a ROS 2-based framework, allows for seamless querying through semantic web APIs such as SPARQL. Initial tests demonstrate the efficiency of this system in supporting advanced scenarios in human-robot interaction. All the repositories developed as part of this contribution can be found at github.com/RepresentationMaps.

In the field of Human-Robot Interaction (HRI), a key challenge lies in enabling humans to comprehend the decisions and behaviours of robots. One promising approach involves leveraging Theory of Mind (ToM) frameworks, wherein a robot estimates the mental model that a user holds about its functioning and compares this with the representation of its internal mental model. This comparison allows the robot to identify potential mismatches and generate communicative actions to bridge such gaps. Effective communication requires the robot to maintain unique mental models for each user and personalise explanations based on past interactions. To address this, we propose an architecture grounded in Large Language Models (LLMs) that operationalises this theoretical framework. We demonstrate the feasibility of this approach through qualitative examples, showcasing responses provided by a robot patrolling a geriatric hospital.

This tutorial aims at providing an up-to-date picture of the state-of-art regarding using the Robot Operating System (ROS) to build robots with socio-cognitive capabilities. The tutorial will briefly introduce the ROS4HRI framework, and show how it can be used to build a complete social robot architecture, from human perception to expressive social interaction. We will illustrate the full software integration required to implement an autonomous social robot using a combination of open-source ROS-based social perception modules, a semantic knowledge base, a Large Language Model (LLM), and multi-modal expressiveness. Participants will be able to follow along using a simple social interaction simulator, as well as their own webcams. The organisers will also provide a new PAL Robotics TIAGo Pro stand-alone head to demonstrate the same system running on actual hardware.

Robot swarms have the potential to help groups of people with social tasks, given their ability to scale to large numbers of robots and users. Developing multi-human-swarm interaction is therefore crucial to support multiple people interacting with the swarm simultaneously - which is an area that is scarcely researched, unlike single-human, single-robot or single-human, multi-robot interaction. Moreover, most robots are still confined to laboratory settings. In this paper, we present our work with MOSAIX, a swarm of robot Tiles, that facilitated ideation at a science museum. 63 robots were used as a swarm of smart sticky notes, collecting input from the public and aggregating it based on themes, providing an evolving visualization tool that engaged visitors and fostered their participation. Our contribution lies in creating a large-scale (63 robots and 294 attendees) public event, with a completely decentralized swarm system in real-life settings. We also discuss learnings we obtained that might help future researchers create multi-human-swarm interaction with the public.

This paper presents the TIAGo Head, a new tabletop social robot from PAL Robotics, focusing on its capabilities as an HRI platform. We detail the robots’ hardware, highlighting its sensors/actuators and on-board computing; and its software architecture, including social perception, expressive face, a knowledge base, and integration with large language models (LLMs) for natural conversations. We also describe a use-case in a receptionist scenario where TIAGo Head dynamically interacts with travelers by displaying news and conversing.

In the field of Human-Robot Interaction (HRI), a fundamental challenge is to facilitate human understanding of robots. The emerging domain of eXplainable HRI (XHRI) investigates methods to generate explanations and evaluate their impact on human-robot interactions. Previous works have highlighted the need to personalise the level of detail of these explanations to enhance usability and comprehension. Our paper presents a framework designed to update and retrieve user knowledge-memory models, allowing for adapting the explanations’ level of detail while referencing previously acquired concepts. Three architectures based on our proposed framework that use Large Language Models (LLMs) are evaluated in two distinct scenarios: a hospital patrolling robot and a kitchen assistant robot. Experimental results demonstrate that a two-stage architecture, which first generates an explanation and then personalises it, is the framework architecture that effectively reduces the level of detail only when there is related user knowledge.

Accurate recognition of human emotions is a crucial challenge in affective computing and human-robot interaction (HRI). Emotional states play a vital role in shaping behaviors, decisions, and social interactions. However, emotional expressions can be influenced by contextual factors, leading to misinterpretations if context is not considered. Multimodal fusion, combining modalities like facial expressions, speech, and physiological signals, has shown promise in improving affect recognition. This paper proposes a transformer-based multimodal fusion approach that leverages facial thermal data, facial action units, and textual context information for context-aware emotion recognition. We explore modality-specific encoders to learn tailored representations, which are then fused and processed by a shared transformer encoder to capture temporal dependencies and interactions. The proposed method is evaluated on a dataset collected from participants engaged in a tangible tabletop Pacman game designed to induce various affective states. Our results demonstrate improvements from incorporating contextual information and multimodal fusion, achieving 89% F1 score with our full model compared to 65% for action units alone and 30% for thermal data alone.

Human-robot interactions increasingly require adaptive instruction delivery, yet robots struggle to calibrate instruction detail levels without explicit user input. We present a system that automatically modulates instruction granularity using real-time affect detection through multi-modal fusion of thermal imaging, facial expressions, and contextual information. Our transformer-based architecture integrates these signals to enable decisions about instruction delivery based on detected user states. In a between-subjects study (N=40), participants completed assembly tasks under either manual adjustment or automatic adaptation conditions. Results showed significantly fewer manual adjustments in the adaptive condition (0.7 vs 2.0 per session), with comparable user satisfaction across conditions. This work shows the effectiveness of affect-driven adaptive instruction in human-robot interaction, contributing to more responsive robotic interfaces while providing guidelines for balancing automation with user control.

In social robots, visual attention expresses awareness of the scenario components and dynamics. As in humans, their attention should be driven by a combination of different attention mechanisms. In this article, we introduce multi-modal saliency maps, i.e., spatial representations of saliency that dynamically integrate multiple attention sources depending on the context. We provide the mathematical formulation of the model and an open source software implementation. Finally, we present an initial exploration of its potential in social interaction scenarios with humans and evaluate its implementation.

While Automatic Speech Recognition (ASR) systems excel in controlled environments, challenges arise in robot-specific setups due to unique microphone requirements and added noise sources. In this paper, we create a dataset of initiating conversations with brief exchanges in 5 European languages, and we systematically evaluate current state-of-art ASR systems (Vosk, OpenWhisper, Google Speech and NVidia Riva). Besides standard metrics, we also look at two critical downstream tasks for human-robot verbal interaction: intent recognition rate and entity extraction, using the open-source Rasa chatbot. Overall, we found that open-source solutions as Vosk performs competitively with closed-source solutions while running on the edge, on a low compute budget (CPU only).

Multi-modal social perception usually involves several independent software modules, detecting for instance faces, voices, body skeletons. Those features need then to be matched to each other, to create a complete model of a person. While the problem is simple in one-to-one interactions, multi-party interactions require to optimize a probabilistic graph in order to find the most likely persons--features associations, while ensuring practical properties like stability over time. This paper presents an open-source algorithm that searches over all possible partitions of the relationship graph to identify the best partition. We playfully call this algorithm Mr. Potato, after the eponymous children' game.

Many research works currently focus on algorithms designed to generate explanations and then evaluate their effect on user trust and understanding of robots. Even though some projects attempt to design understandable interfaces, they usually serve as extra features for solutions that already exist. In this paper, we suggest a user-centric approach to design explainable robot systems from the very beginning. In particular, we provide a participatory design approach that places emphasis on the cooperative design of an understandable and intuitive interface between the user and the robot system. We suggest turning the attention to the robot’s functionality and autonomous behaviours development after this interface has been established. We exemplify how to apply the proposed framework in a geriatric unit at an intermediate care centre.

This work deals with a practical everyday problem: stable object placement on flat surfaces starting from unknown initial poses. Common object-placing approaches require either complete scene specifications or extrinsic sensor measurements, e.g., cameras, that occasionally suffer from occlusions. We propose a novel approach for stable object placing that combines tactile feedback and proprioceptive sensing. We devise a neural architecture called PlaceNet that estimates a rotation matrix, resulting in a corrective gripper movement that aligns the object with the placing surface for the subsequent object manipulation. We compare models with different sensing modalities, such as force-torque, an external motion capture system, and two classical baseline models in real-world object placing tasks with different objects. The experimental evaluation of our placing policies with a set of unseen everyday objects reveals significant generalization of our proposed pipeline, suggesting that tactile sensing plays a vital role in the intrinsic understanding of robotic dexterous object manipulation. Code, models, and supplementary videos are available on https://sites.google.com/view/placing-by-touching.

With the increase in life expectancy and staff shortage, there is an urgency to understanding the needs of older adults and exploring emerging fields such as social robotics to tackle the challenges of ageing. The paper highlights the importance of providing cognitive support, physical support, reducing loneliness and increasing social engagement among older adults as well as reducing caregiver burden, and suggests that socially assistive robots (SAR) can assist older adults and their carers with such needs. However, the paper also points out that there are several challenges associated with designing and deploying SAR systems, and involving end-users in the design process is necessary to improve user acceptability and adoption. The paper describes the approaches used by PAL Robotics to facilitate real-world deployment of its ARI and TIAGo social robots, and provides examples of how these robots have been used to tackle different healthcare needs.

We present the Social Group Interactions (SoGrIn) dataset; a dataset which captures non-verbal signals of groups as they complete socially collaborative and formation-provoking tasks. The dataset comprises precise proxemics (captured motion) and facial features (facial landmarks, gaze direction, facial action units) encompassing a total duration of 60 minutes involving 30 individuals, divided into six groups. Also included are basic demographic information and responses to the Big 5 personality questionnaire. The Social Group Interactions dataset is publicly available at https://doi.org/10.5281/zenodo.7778123.

This paper introduces a Behaviour Tree based design of long-term social robot behaviour in the context of SHAPES project, using ROS-compatible libraries, specifically two types of behaviours: a robot idle behaviour where the human approaches and begins the interaction, and a second behaviour where the robot actively navigates and searchers for a specific user to deliver a reminder. The behaviours will be tested on-site as part of SHAPES pilots and adjusted based on feedback and needs and is focused on long-term robot acceptance.

We present a software tool, fully integrated with ROS, that enables robots to perceive people full body in 3D. The system works either with a simple RGB camera, or a RGB-D camera for better 3D absolute position estimation. The system is based on Google Mediapipe, and runs at > 8Hz on CPU. The consistency of the human kinematic model is ensured by relying on a URDF-defined kinematic model, that could be adjusted to each person's anthropometric characteristics.

To achieve seamless interactions, robots have to be capable of reliably detecting affective states in real time. One of the possible states that humans go through while interacting with robots is frustration. Detecting frustration from RGB images can be challenging in some real-world situations; thus, we investigate in this work whether thermal imaging can be used to create a model that is capable of detecting frustration induced by cognitive load and failure. To train our model, we collected a data set from 18 participants experiencing both types of frustration induced by a robot. The model was tested using features from several modalities: thermal, RGB, Electrodermal Activity (EDA), and all three combined. When data from both frustration cases were combined and used as training input, the model reached an accuracy of 89% with just RGB features, 87% using only thermal features, 84% using EDA, and 86% when using all modalities. Furthermore, the highest accuracy for the thermal data was reached using three facial regions of interest: nose, forehead and lower lip.

MOSAIX is a new robot swarm platform built to be used in social settings. Consisting of up to 100 individual robot Tiles, MOSAIX is a social swarm system, aimed at helping humans in social tasks such as opinion-mixing and brainstorming. MOSAIX also has the potential to be used as a platform to study human-swarm interaction and swarm expressivity. MOSAIX is intended to be used outside laboratory settings and has already been used to collect 154 opinions about climate change in a local shopping mall, used by participants to create collaborative art and used as an educational tool for schoolchildren. We also discuss future applications, such as MOSAIX acting as smart post-it notes for ideation activities.

Decision-making among groups of humans can benefit from open discussion and inclusion of a diversity of opinions, promoting deliberative democracy. In this work, we test whether a swarm of robots can help facilitate decision-making by visually representing the diversity of opinions. We used a swarm of robots we built, called MOSAIX, that consists of 4-inch touchscreens-on-wheels robots called Tiles. The robots acted as physical avatars for opinions, helping them travel and mix together. We recruited 46 participants split into groups of 7 and 8 to test whether the robot movement had an impact on the decision-making process versus using the robots stationary in the participants' hands akin to smartphones. Furthermore, we wanted to test whether the participants felt comfortable expressing their opinion through the robots. Results show the participants indeed felt comfortable using the robots, and user engagement increased with the movement of the robots. The difference between the participants' first and last opinions also increased with the movement of the robots. We believe that robot swarms have not been used before to facilitate decision-making among a group of people. Therefore, our contribution is in testing the possibility of how and whether using a moving robot swarm helps humans reach a decision.

We present a holistic grasping controller, combining free-space position control and in-contact force-control for reliable grasping given uncertain object pose estimates. Employing tactile fingertip sensors, undesired object displacement during grasping is minimized by pausing the finger closing motion for individual joints on first contact until force-closure is established. While holding an object, the controller is compliant with external forces to avoid high internal object forces and prevent object damage. Gravity as an external force is explicitly considered and compensated for, thus preventing gravity-induced object drift. We evaluate the controller in two experiments on the TIAGo robot and its parallel-jaw gripper proving the effectiveness of the approach for robust grasping and minimizing object displacement. In a series of ablation studies, we demonstrate the utility of the individual controller components.

When we approach a group, there is an exchange of a multitude of verbal or non-verbal social signals to indicate that we are looking to interact. We continue to share these signals throughout the interaction to portray our thoughts and motivations. We define an interaction by the signals we send; sending different signals evokes a different response. Giving social robots the knowledge of group social interaction, they will have the ability to more effectively participate in these interactions in the real world. In this paper, we present the results from an online data collection study looking at social group dynamics. We collected a dataset of social behaviours in a group using a socially interactive game played online by 88 participants. We also introduce a novel visual social engagement metric, which is derived from two social signals: proxemics (distance between interaction participants) and mutual gaze. We propose a mathematical formula of both mutual gaze as the product of the mutual distances to the optical axis, and the visual social engagement as mutual gaze divided by distance between participants. Additionally, we investigate the influence of personality traits on the resulting interaction patterns. Using the metric, we create unique interaction profiles which suggest that participants have an interaction ‘style’. No clear correlation between personality and interaction patterns was found.

We explore the effectiveness of a dynamically processed incremental referring description system using under-specified ambiguous descriptions that are then built upon using linguistic repair statements, which we refer to as a dynamic system. We build a dynamically processed incremental referring description generation system that is able to provide contextual navigational statements to describe an object in a potential real-world situation of nuclear waste sorting and maintenance. In a study of 31 participants, we test the dynamic system in a case where a user is remote operating a robot to sort nuclear waste, with the robot assisting them in identifying the correct barrels to be removed. We compare these against a static non-ambiguous description given in the same scenario. As well as looking at efficiency with time and distance measurements, we also look at user preference. Results show that our dynamic system was a much more efficient method—taking only 62% of the time on average—for finding the correct barrel. Participants also favoured our dynamic system.

Game engines are increasingly used as simulation platforms by the autonomous vehicle community to develop vehicle control systems and test environments. A key requirement for simulation-based development and verification is determinism, since a deterministic process will always produce the same output given the same initial conditions and event history. Thus, in a deterministic simulation environment, tests are rendered repeatable and yield simulation results that are trustworthy and straightforward to debug. However, game engines are seldom deterministic. This paper reviews and identifies the potential causes and effects of non-deterministic behaviours in game engines. A case study using CARLA, an open-source autonomous driving simulation environment powered by Unreal Engine, is presented to highlight its inherent shortcomings in providing sufficient precision in experimental results. Different configurations and utilisations of the software and hardware are explored to determine an operational domain where the simulation precision is sufficiently high i.e. variance between repeated executions becomes negligible for development and testing work. Finally, a method of a general nature is proposed, that can be used to find the domains of permissible variance in game engine simulations for any given system configuration.

Integrating real-time, complex social signal processing into robotic systems – especially in real-world, multi-party interaction situations – is a challenge faced by many in the Human-Robot Interaction (HRI) community. The difficulty is compounded by the lack of any standard model for human representation that would facilitate the development and interoperability of social perception components and pipelines. We introduce in this paper a set of conventions and standard interfaces for HRI scenarios, designed to be used with the Robot Operating System (ROS). It directly aims at promoting interoperability and re-usability of core functionality between the many HRI-related software tools, from skeleton tracking, to face recognition, to natural language processing. Importantly, these interfaces are designed to be relevant to a broad range of HRI applications, from high-level crowd simulation, to group-level social interaction modelling, to detailed modelling of human kinematics. We demonstrate these interfaces by providing a reference pipeline implementation, packaged to be easily downloaded and evaluated by the community.

This study investigated whether virtual reality could be used as platform for conducting human-robot interaction user studies. It was investigated whether user studies performed in virtual reality elicited realistic responses from participants. To answer this question, a real world study was replicated as closely as possible in virtual reality, where a robot tour guide asked participants to keep a secret. The experiment consisted of a virtual museum tour where the robot acted as the tour guide while displaying either social or non-social behaviour. The measurements taken in this study were the objective measurement whether the participants kept the robot's secret or not. Questionnaires were taken to investigate participants' perception of the robot and its feelings, as well as their experienced level of presence and their tendency to become immersed in the virtual environment. Results show that the participants responded differently in the virtual reality study when compared to the original real world study, where the secret was kept more often for the non-social robot, but less often for the social robot. In both the original and replicated study a strong, positive correlation was found between participants' perception of the robot as a social being and their tendency to keep the robot's secret. These inconclusive findings, some changes that were required for the virtual environment compared to the original study, and different participant demographics indicate that more work is needed to determine whether virtual reality can be used as a tool to conduct human-robot interaction experiments.

We present in this paper our work towards a new dynamic method of generating spatial referring expressions. While people are generally ambiguous in their description of locations, previous methods of artificial generation mostly considered non-ambiguous descriptions. However, to increase the naturalness of interaction and share workload in the communication, robots should be able to generate language in a more dynamic way. Our method initially produces ambiguous spatial referring expressions followed by dynamically generating repair statements. We built a classifier using data from 18 participants as they described locations to each other. We perform a preliminary analysis on this method using two further pilot studies.

In this paper we present the results of an experimental study investigating the application of human persuasive strategies to a social robot. We demonstrate that robot displays of goodwill and similarity to the participant significantly increased robot persuasiveness, as measured objectively by participant behaviour. However, such strategies had no impact on subjective measures concerning perception of the robot, and perception of the robot did not correlate with participant behaviour. We hypothesise that this is due to difficulty in accurately measuring perception of a robot using subjective measures. We suggest our results are particularly relevant for the design and development of socially assistive robots.

Reasoning about spatial and geometric relations between objects in a tabletop human-robot interaction is a challenge due to the perception not being always consistent: objects placed on a table seem to be slightly in the air; they overlap; they disappear due to occlusions. Yet, interpreting and anchoring perceptual data in a physically consistent estimation of the scene is a crucial ability for humans, and thus robots in HRI context. In this paper we present a simulation-based physics reasoner integrated in a lightweight situation-assessment framework called Underworlds, that allows the robot to stabilize objects and build at run-time a consistent estimation of the scene, even for entirely hidden objects, while inferring the actions performed by its human partner.

We conducted a study with 25 children to investigate the effectiveness of a robot measuring and encouraging production of spatial concepts in a second language compared to a human experimenter. Productive vocabulary is often not measured in second language learning, due to the difficulty of both learning and assessing productive learning gains. We hypothesized that a robot peer may help assessing productive vocabulary. Previous studies on foreign language learning have found that robots can help to reduce language anxiety, leading to improved results. In our study we found that a robot is able to reach a similar performance to the experimenter in getting children to produce, despite the person's advantages in social ability, and discuss the extent to which a robot may be suitable for this task.

We introduce UNDERWORLDS, a novel lightweight framework for cascading spatio-temporal situation assessment in robotics. UNDERWORLDS allows programmers to represent the robot's environment as real-time distributed data structures, containing both scene graphs (for representation of 3D geometries) and timelines (for representation of temporal events). UNDERWORLDS supports cascading representations: the environment is viewed as a set of worlds that can each have different spatial and temporal granularities, and may inherit from each other. UNDERWORLDS also provides a set of high-level client libraries and tools to introspect and manipulate the environment models. This article presents the design and architecture of this open-source tool, and explores some applications, along with examples of use.

This paper provides a qualitative review of different object recognition techniques relevant for near-proximity Human-Robot Interaction. These techniques are divided into three categories:2D correspondence, 3D correspondence and non-vision based methods. For each technique an implementation is chosen that is representative of the existing technology to provide a broad review to assist in selecting an appropriate method for tabletop object recognition manipulation. For each of these techniques we give their strengths and weaknesses based on defined criteria. We then discuss and provide recommendations for each of them.

An increasing number of human-robot interaction (HRI) studies are now taking place in applied settings with children. These interactions often hinge on verbal interaction to effectively achieve their goals. Great advances have been made in adult speech recognition and it is often assumed that these advances will carry over to the HRI domain and to interactions with children. In this paper, we evaluate a number of automatic speech recognition (ASR) engines under a variety of conditions, inspired by real-world social HRI conditions. Using the data collected we demonstrate that there is still much work to be done in ASR for child speech, with interactions relying solely on this modality still out of reach. However, we also make recommendations for child-robot interaction design in order to maximise the capability that does currently exist.

In this article, we present Cellulo, a novel robotic platform that investigates the intersection of three ideas for robotics in education: designing the robots to be versatile and generic tools; blending robots into the classroom by designing them to be pervasive objects and by creating tight interactions with (already pervasive) paper; and finally considering the practical constraints of real classrooms at every stage of the design. Our platform results from these considerations and builds on a unique combination of technologies: groups of handheld haptic-enabled robots, tablets and activity sheets printed on regular paper. The robots feature holonomic motion, haptic feedback capability and high accuracy localization through a microdot pattern overlaid on top of the activity sheets, while remaining affordable (robots cost about EUR 125 at the prototype stage) and classroom-friendly. We present the platform and report on our first interaction studies, involving about 230 children.

Human–Robot Interaction challenges Artificial Intelligence in many regards: dynamic, partially unknown environments that were not originally designed for robots; a broad variety of situations with rich semantics to understand and interpret; physical interactions with humans that requires fine, low-latency yet socially acceptable control strategies; natural and multi-modal communication which mandates common-sense knowledge and the representation of possibly divergent mental models. This article is an attempt to characterise these challenges and to exhibit a set of key decisional issues that need to be addressed for a cognitive robot to successfully share space and tasks with a human. We identify first the needed individual and collaborative cognitive skills: geometric reasoning and situation assessment based on perspective-taking and affordance analysis; acquisition and representation of knowledge models for multiple agents (humans and robots, with their specificities); situated, natural and multi-modal dialogue; human-aware task planning; human–robot joint task achievement. The article discusses each of these abilities, presents working implementations, and shows how they combine in a coherent and original deliberative architecture for human–robot interaction. Supported by experimental results, we eventually show how explicit knowledge management, both symbolic and geometric, proves to be instrumental to richer and more natural human–robot interactions by pushing for pervasive, human-level semantics within the robot's deliberative system.

Human-Robot Interaction (HRI) research requires the integration and cooperation of multiple disciplines, technical and social, in order to make progress. In many cases using different motivations, each of these disciplines bring with them different assumptions and methodologies. We assess recent trends in the field of HRI by examining publications in the HRI conference over the past three years (over 100 full papers), and characterise them according to 14 categories. We focus primarily on aspects of methodology. From this, a series of practical recommendations based on rigorous guidelines from other research fields that have not yet become common practice in HRI are proposed. Furthermore, we explore the primary implications of the observed recent trends for the field more generally, in terms of both methodology and research directions. We propose that the interdisciplinary nature of HRI must be maintained, but that a common methodological approach provides a much needed frame of reference to facilitate rigorous future progress.

Social HRI requires robots able to use appropriate, adaptive and contingent behaviours to form and maintain engaging social interactions with people. Cognitive Architectures emphasise a generality of mechanism and application, making them an ideal basis for such technical developments. Following the successful first workshop on Cognitive Architectures for HRI at the 2014 HRI conference, this second edition of the workshop focusses specifically on applications to social interaction. The full-day workshop is centred on participant contributions, and structured around a set of questions to provide a common basis of comparison between different assumptions, approaches, mechanisms, and architectures. These contributions will be used to support extensive and structured discussions, with the aim of facilitating the development and application of cognitive architectures to social HRI systems. By attending, we envisage that participants will gain insight into how the consideration of cognitive architectures complements the development of autonomous social robots.

The CoWriter activity involves a child in a rich and complex interaction where he has to teach handwriting to a robot. The robot must convince the child it needs his help and it actually learns from his lessons. To keep the child engaged, the robot must learn at the right rate, not too fast otherwise the kid will have no opportunity for improving his skills and not too slow otherwise he may loose trust in his ability to improve the robot' skills. We tested this approach in real pedagogic/therapeutic contexts with children in difficulty over repeated long sessions (40-60 min). Through 3 different case studies, we explored and refined experimental designs and algorithms in order for the robot to adapt to the troubles of each child and to promote their motivation and self-confidence. We report positive observations, suggesting commitment of children to help the robot, and their comprehension that they were good enough to be teachers, overcoming their initial low confidence with handwriting.

Measuring “how much the human is in the interaction” - the level of engagement - is instrumental in building effective interactive robots. Engagement, however, is a complex, multi-faceted cognitive mechanism that is only indirectly observable. This article formalizes with-me-ness as one of such indirect measures. With-me-ness, a concept borrowed from the field of Computer-Supported Collaborative Learning, measures in a well-defined way to what extent the human is with the robot over the course of an interactive task. As such, it is a meaningful precursor of engagement. We expose in this paper the full methodology, from real-time estimation of the human's focus of attention (relying on a novel, open-source, vision-based head pose estimator), to on-line computation of with-me-ness. We report as well on the experimental validation of this approach, using a naturalistic setup involving children during a complex robot-teaching task.

Subjective appreciation and performance evaluation of a robot by users are two important dimensions for Human-Robot Interaction, especially as increasing numbers of people become involved with robots. As roboticists we have to carefully design robots to make the interaction as smooth and enjoyable as possible for the users, while maintaining good performance in the task assigned to the robot. In this paper, we examine the impact of providing a robot with learning capabilities on how users report the quality of the interaction in relation to objective performance. We show that humans tend to prefer interacting with a learning robot and will rate its capabilities higher even if the actual performance in the task was lower. We suggest that adding learning to a robot could reduce the apparent load felt by a user for a new task and improve the user's evaluation of the system, thus facilitating the integration of such robots into existing work flows.

Building over algorithms previously developed for digital pens, this article introduces a novel 2D localization technique for mobile robots, based on simple printed patterns. This method combines high absolute accuracy (below 0.3mm), unlimited scalability, low computational requirements (the presented open-source implementation runs at above 45Hz on a low-cost microcontroller) and low cost (below €30 per device at prototype stage). The article first presents the underlying algorithms and localization pipeline. It then describes our reference hardware and software implementations, and finally evaluates the performance of this technique for mobile robots.

Research in education has long established how children mutually influence and support each other's learning trajectories, eventually leading to the development and widespread use of learning methods based on peer activities. In order to explore children's learning behavior in the presence of a robotic facilitator during a collaborative writing activity, we investigated how they assess their peers in two specific group learning situations: peer-tutoring and peer-learning. Our scenario comprises of a pair of children performing a collaborative activity involving the act of writing a word/letter on a tactile tablet. In the peer-tutoring condition, one child acts as the teacher and the other as the learner, while in the peer-learning condition, both children are learners without the attribution of any specific role. Our experiment includes 40 children in total (between 6 and 8 years old) over the two conditions, each time in the presence of a robot facilitator. Our results suggest that the peer-tutoring situation leads to significantly more corrective feedback being provided, as well as the children more disposed to self-disclosure to the robot.

This article presents a novel robotic partner which children can teach handwriting. The system relies on the learning by teaching paradigm to build an interaction, so as to stimulate meta-cognition, empathy and increased self-esteem in the child user. We hypothesise that use of a humanoid robot in such a system could not just engage an unmotivated student, but could also present the opportunity for children to experience physically-induced benefits encountered during human-led handwriting interventions, such as motor mimicry. By leveraging simulated handwriting on a synchronised tablet display, a NAO humanoid robot with limited fine motor capabilities has been configured as a suitably embodied handwriting partner. Statistical shape models derived from principal component analysis of a dataset of adult-written letter trajectories allow the robot to draw purposefully deformed letters. By incorporating feedback from user demonstrations, the system is then able to learn the optimal parameters for the appropriate shape models. Preliminary in situ studies have been conducted with primary school classes to obtain insight into children's use of the novel system. Children aged 6-8 successfully engaged with the robot and improved its writing to a level which they were satisfied with. The validation of the interaction represents a significant step towards an innovative use for robotics which addresses a widespread and socially meaningful challenge in education.

Mutual modelling, the reciprocal ability to establish a mental model of the other, plays a fundamental role in human interactions. This complex cognitive skill is however difficult to fully apprehend as it encompasses multiple neuronal, psychological and social mechanisms that are generally not easily turned into computational models suitable for robots. This article presents several perspectives on mutual modelling from a range of disciplines, and reflects on how these perspectives can be beneficial to the advancement of social cognition in robotics. We gather here both basic tools (concepts, formalisms, models) and exemplary experimental settings and methods that are of relevance to robotics. This contribution is expected to consolidate the corpus of knowledge readily available to human-robot interaction research, and to foster interest for this fundamentally cross-disciplinary field.

Presented is PYROBOTS, a new open-source software toolkit for the executive control of complex robotic systems. Borrowing ideas from previous tools like URBI [1], it proposes a lightweight Python framework to develop (preemptive) concurrent and event-based executive controllers. Designed in a bottom-up fashion, out of the need for a practical, unobstrusive tool suitable for rapid prototyping of complex interaction scenarios, PYROBOTS also exposes several simple yet convenient abstractions for physical resources management and pose representation. While middleware-agnostic, it features specific integration with the ROS middleware. Experimental deployments and stress-tests on several robotic platforms (including PR2 and Nao) in dynamic human environments are reported.

In order to explore the impact of integrating a robot as a facilitator in a collaborative activity, we examined interpersonal distancing of children both with a human adult and a robot facilitator. Our scenario involves two children performing a collaborative learning activity, which included the writing of a word/letter on a tactile tablet. Based on the learning-by-teaching paradigm, one of the children acted as a teacher when the other acted as a learner. Our study involved 40 children between 6 and 8 years old, in two conditions (robot or human facilitator). The results suggest first that the child acting as a teacher feel more responsible when the facilitator is a robot, compared to a human; they show then that the interaction between a (teacher) child and a robot facilitator can be characterized as being a reciprocity-based interaction, whereas a human presence fosters a compensation-based interaction.

We present the design approach and evaluation of our prototype called "Ranger". Ranger is a robotic toy box that aims to motivate young children to tidy up their room. We evaluated Ranger in 14 families with 31 children (2-10 years) using the Wizard-of-Oz technique. This case study explores two different robot behaviors (proactive vs. reactive) and their impact on children's interaction with the robot and the tidying behavior. The analysis of the video recorded scenarios shows that the proactive robot tended to encourage more playful and explorative behavior in children, whereas the reactive robot triggered more tidying behavior. Our findings hold implications for the design of interactive robots for children, and may also serve as an example of evaluating an early version of a prototype in a real-world setting.

No abstract available.

Over the last four years, we have been slowly ramping up explicit knowledge representation and manipulation in the deliberative and executive layers of our robots. Ranging from situation assessment to symbolic task planning, from verbal interaction to event-driven execution control, we have built up a knowledge-oriented architecture which is now used on a daily basis on our robots. This article presents our design choices, the articulations between the diverse deliberative components of the robot, and the strengths and weaknesses of this approach. We show that explicit knowledge management is not only a convenient tool from the software engineering point of view, but also pushes for a different, more semantic way to address the decision-making issue in autonomous robots.

Over the last two years, the Modular OpenRobots Simulation Engine (MORSE) project1 went from a simple extension plugged on the Blender's Game Engine to a full-fledged simulation environment for academic robotics. Driven by the requirements of several of its developers, tools dedicated to Human-Robot interaction simulation have taken a prominent place in the project. This late breaking report discusses some of the recent additions in this domain, including the immersive experience provided by the integration of the Kinect device as input controller. We also give an overview of the experiences we plan to complete in the coming months.

No abstract available.

We have designed and implemented new spatio-temporal reasoning skills for a cognitive robot, which explicitly reasons about human beliefs on object positions. It enables the robot to build symbolic models reflecting each agent's perspective on the world. Using these models, the robot has a better understanding of what humans say and do, and is able to reason on what human should know to achieve a given goal. These new capabilities are also demonstrated experimentally.

This paper presents MORSE, a new open-source robotics simulator. MORSE provides several features of interest to robotics projects: it relies on a component-based architecture to simulate sensors, actuators and robots; it is flexible, able to specify simulations at variable levels of abstraction according to the systems being tested; it is capable of representing a large variety of heterogeneous robots and full 3D environments (aerial, ground, maritime); and it is designed to allow simulations of multiple robots systems. MORSE uses a “Software-in-the-Loop” philosophy, i.e. it gives the possibility to evaluate the algorithms embedded in the software architecture of the robot within which they are to be integrated. Still, MORSE is independent of any robot architecture or communication framework (middleware). MORSE is built on top of Blender, using its powerful features and extending its functionality through Python scripts. Simulations are executed on Blender's Game Engine mode, which provides a realistic graphical display of the simulated environments and allows exploiting the reputed Bullet physics engine. This paper presents the conception principles of the simulator and some use-case illustrations.

If robots are to cooperate with humans in an increasingly human-like manner, then significant progress must be made in their abilities to observe and learn to perform novel goal directed actions in a flexible and adaptive manner. The current research addresses this challenge. In CHRIS.I [1], we developed a platform-independent perceptual system that learns from observation to recognize human actions in a way which abstracted from the specifics of the robotic platform, learning actions including “put X on Y” and “take X”. In the current research, we extend this system from action perception to execution, consistent with current developmental research in human understanding of goal directed action and teleological reasoning. We demonstrate the platform independence with experiments on three different robots. In Experiments 1 and 2 we complete our previous study of perception of actions “put” and “take” demonstrating how the system learns to execute these same actions, along with new related actions “cover” and “uncover” based on the composition of action primitives “grasp X” and “release X at Y”. Significantly, these compositional action execution specifications learned on one iCub robot are then executed on another, based on the abstraction layer of motor primitives. Experiment 3 further validates the platform-independence of the system, as a new action that is learned on the iCub in Lyon is then executed on the Jido robot in Toulouse. In Experiment 4 we extended the definition of action perception to include the notion of agency, again inspired by developmental studies of agency attribution, exploiting the Kinect motion capture system for tracking human motion. Finally in Experiment 5 we demonstrate how the combined representation of action in terms of perception and execution provides the basis for imitation. This provides the basis for an open ended cooperation capability where new actions can be learned and integrated into shared plans for cooperation. Part of the novelty of this research is the robots' use of spoken language understanding and visual perception to generate action representations in a platform independent manner based on physical state changes. This provides a flexible capability for goal-directed action imitation.

While key for human-robot interaction, natural language interpretation is a notoriously difficult task, especially because the interaction context is at the same time essential for dialogue understanding, difficult to build for machines, and depends on each speaker point of view. However, robots as embodied artifacts, can perceive their environment and interactors, and hence compute symbolic models from various perspectives. This allows in turn to build symbolic contexts for dialogues. In this paper, we introduce DIALOGS, a component for natural language interpretation that relies on these structured symbolic models of the world to ground verbal interaction.

The topic of reusable software in robotics is now largely addressed. Components based architectures, where components are independent units that can be reused accross applications, have become more popular. As a consequence, a long list of middlewares and integration tools is available in the community, often in the form of open-source projects. However, these projects are generally self contained with little reuse between them. This paper presents a software engineering approach that intends to grant middleware independance to robotic software components so that a clear separation of concerns is achieved between highly reusable algorithmic parts and integration frameworks. Such a decoupling let middle-wares be used interchangeably, while fully benefitting from their specific, individual features. This work has been integrated into a new version of the open-source GenoM component generator tool: GenoM3

One of the long term objectives of robotics and artificial cognitive systems is that robots will increasingly be capable of interacting in a cooperative and adaptive manner with their human counterparts in open-ended tasks that can change in real-time. In such situations, an important aspect of the robot behavior will be the ability to acquire new knowledge of the cooperative tasks by observing humans. At least two significant challenges can be identified in this context. The first challenge concerns development of methods to allow the characterization of human actions such that robotic systems can observe and learn new actions, and more complex behaviors made up of those actions. The second challenge is associated with the immense heterogeneity and diversity of robots and their perceptual and motor systems. The associated question is whether the identified methods for action perception can be generalized across the different perceptual systems inherent to distinct robot platforms. The current research addresses these two challenges. We present results from a cooperative human-robot interaction system that has been specifically developed for portability between different humanoid platforms. Within this architecture, the physical details of the perceptual system (e.g. video camera vs IR video with reflecting markers) are encapsulated at the lowest level. Actions are then automatically characterized in terms of perceptual primitives related to motion, contact and visibility. The resulting system is demonstrated to perform robust object and action learning and recognition on two distinct robotic platforms. Perhaps most interestingly, we demonstrate that knowledge acquired about action recognition with one robot can be directly imported and successfully used on a second distinct robot platform for action recognition. This will have interesting implications for the accumulation of shared knowledge between distinct heterogeneous robotic systems.

This paper presents an embeddable knowledge processing framework, along with a common-sense ontology, designed for robotics. We believe that a direct and explicit integration of cognition is a compulsory step to enable human-robots interaction in semantic-rich human environments like our houses. The OpenRobots Ontology (ORO) kernel allows to turn previously acquired symbols into concepts linked to each other. It enables in turn reasoning and the implementation of other advanced cognitive functions like events, categorization, memory management and reasoning on parallel cognitive models. We validate this framework on several cognitive scenarii that have been implemented on three different robotic architectures.

In human-robot interaction, a robot must be prepared to handle possible ambiguities generated by a human partner. In this work we propose a set of strategies that allow a robot to identify the referent when the human partner refers to an object giving incomplete information, i.e. an ambiguous description. Moreover, we propose the use of an ontology to store and reason on the robot's knowledge to ease clarification, and therefore, improve interaction. We validate our work through both simulation and two real robotic platforms performing two tasks: a daily-life situation and a game.