A key challenge in social robotics is identifying the design features and social mechanisms that sustain long-term engagement with robots. Although mounting evidence suggests that end-user customization is a vital aspect of robot ownership “in the wild”, empirical research on this phenomenon and its psychological outcomes remains sparse. In this mixed methods study, we surveyed 113 robot owners and conducted semi-structured interviews with 13 more, providing a holistic perspective on customization practices among long-term users. Our findings show that customization is highly prevalent among robot owners, with quantitative results indicating that customization indirectly predicts robot attachment through self-extension and psychological ownership. Our interviews furthermore reveal a vibrant culture of customization in online and offline robotics spaces, which is sustained by strong community networks. Together, these results underscore the central role of customization in fostering enduring engagement with companion robots. Through customization, users imbue their robots with personally significant identities, form deeper attachments to them, and reinforce their own individuality as robot owners. Customization also embeds owners within a broader community of robot enthusiasts, promoting social connections, creative practice, and sustained use. Given its prevalence among robot owners, we conclude with recommendations for how robot designers and researchers can leverage customization to set the stage for long-term and personally meaningful human–robot bonds.

This study explores the impact of socio-economic and cultural factors on handwashing behaviour in schools through a robot-assisted intervention. We evaluated hand hygiene compliance across three schools, one in India and two in the UK, representing different cultural and socio-economic backgrounds, using the “WallBo” social robot to guide students (n=77) through proper handwashing techniques. The results revealed that students from lower socio-economic backgrounds demonstrated greater initial learning gains, particularly in the Indian school where the novelty of the robot contributed to heightened engagement. However, these gains were not sustained post-intervention, underscoring the importance of continuous reinforcement strategies in under- resourced settings. In contrast, students from higher-income schools demonstrated more consistent retention, likely due to stronger baseline knowledge and greater familiarity with technology. These findings underscore the importance of contextualising technology-driven interventions using social robots within socio-cultural frameworks to maximise long-term impact. Additionally, this study highlights the potential of social robots as an effective educational tool in diverse school environments, provided that long-term reinforcement strategies are put in place.

International students in UK universities often struggle with interactions in English, particularly on their first days in the country. We have developed a multilingual support robot tailored to their needs. To evaluate the performance of the robot, 60 international students asked the robot, in either English or their native language (Modern Standard Arabic or Mandarin Chinese), for support on topics including campus directions, local tax exemption, financial aid, and official documents. Overall, users preferred to use their native language when interacting with the support robot, and using their native language in the robot interaction also had a positive effect on their perception of the interaction itself.

Recent technological advances have allowed robots to assist in the service sector, and consequently accelerate job and sector transformation. Less attention has been paid to the use of robots in real-world organisations where social benefits, as opposed to profits, are the primary motivator. To explore these opportunities, we have partnered with a working church and visitor attraction in the United Kingdom. We conducted interviews with 15 participants from a range of stakeholder groups within the church to understand worker perspectives of introducing a social robot to the church and analysed the results using reflexive thematic analysis. Findings indicate mixed responses to the use of a robot, with participants highlighting the empathetic responsibility the church has towards people and the potential for unintended consequences. However, information provision and alleviation of menial or mundane tasks were identified as potential use cases. This highlights the need to consider not only the financial aspects of robot introduction, but also how social and intangible values shape what roles a robot should take on within an organisation.

The social robotics community is increasingly embracing human-centered techniques to design robots that align with users’ needs, preferences, and lived experiences. However, given the known challenges of incorporating laypeople into a design process, little empirical work has tested whether these techniques generate robotic concepts that are accepted and understood by a wider audience. In this mixed-methods online study, we examined how laypeople perceived and evaluated robots that were created through human-centered design. Fifty-two participants assessed a set of laypeople-created healthcare, education, entertainment, and telepresence robot designs according to how successfully each design signaled its intended use case. Our findings demonstrate that layperson-designed robots efficiently communicated their use context. Thus, low-level creative prototyping with end-users can be an effective way of eliciting strong initial design concepts as part of the human-centered design process, though this is influenced by the robot’s application domain. Furthermore, we showcase that simplistic robotic designs are sufficient to cue diverse affordances, highlighting the importance of matching a robot’s appearance to its intended use case. Our findings contribute to the study of human-centered design within social robotics, assessing the tools and activities end-users need to be able to meaningfully contribute to robotic design.

The software submission presents a fully working artificial cognition model, which controls a NAO social robot. The model was specifically designed to control a socio-affective companion robot for use in a medical setting. It was deployed using embedded hardware: a Raspberry Pi 4B and a Jetson Nano Board, and an external RGB-D camera. Based on the ROS operating system, this software package includes components for social signal processing, behaviour selection, affective behaviour rendering, and a web-based user interface. The robot's behaviours are selected by a planning system, which generates the robot's behaviours based on the state of the interaction, the progress of the medical procedure, and the user's affective state. The system has been tested in simulated environments and is currently being used in two clinics to perform a usability test and will subsequently be used to carry out a series of clinical trials

To facilitate easy, seamless, and dynamic interactions between humans and social robots, it is important that the robot’s physical appearance gives clear cues to its affordances. However, little is known about what underlying concepts and assumptions shape laypeople’s perception and understanding of different robotic embodiments. To explore what robot-inexperienced users expect robots in different application domains to look like, we drew on Research through Design principles to pilot a tangible design kit with which laypeople could prototype robot designs. 27 participants with no background in robotics were asked to design robots for four different application domains. These participants were then further interrogated about their design choices in structured qualitative interviews. The resulting designs primarily ranged from mechanical to humanoid in appearance, though several animal-like entertainment robots were also created. The inductive thematic analysis of participant interviews revealed complex opinions on anthropomorphism in robotic designs and stressed social robots’ potential for customization and accessibility. Our findings provide qualitative insight into the beliefs that underlie laypeople’s understanding of robotic design in different contexts. Furthermore, we piloted a design toolkit that allows laypeople at any level of creative ability to design and critically reflect on robotic concepts and embodiments.

Facial expressions are used to facilitate daily interactions including conversations, in every culture. However, little is known about which specific facial expressions convey conversational messages, which limits the social signalling capabilities of conversational virtual agents. We address this critical knowledge gap by modeling the facial expressions of key conversational messages directly from cultural perception, and transferring them to a conversational virtual agent. Using a novel data-driven psychology-based method, we modelled facial expressions of 'thinking,' 'interested,' 'bored' and 'confused' directly from the cultural perception of 40 participants across two distinct cultures (Western European, East Asian). We then transferred these facial expression models to a popular conversational virtual agent and validated them with a new group of cultural participants. Results showed that, in both cultures, the majority of our culturally derived conversational facial expression models successfully transferred to the agent. A further cross-cultural analysis of the conversational facial expressions showed systematic similarities (e.g., eye brow raising in 'interested') and differences (e.g., Westerners use horizontal mouth stretch and East Asians use vertical mouth opening to show confusion) that could facilitate or hinder cross-cultural communication. Our results demonstrate the power of using a culturally sensitive perception-based psychological approach to develop psychologically impactful facial expressions for conversational virtual agents. We anticipate that our facial expression models will enhance virtual agent social signalling capabilities and their global marketability.

No abstract available.

This work analyses the relationship between the way robots gesture and the way those gestures are perceived by human users. In particular, this work shows how modifying the amplitude and speed of a gesture affect the Godspeed scores given to those gestures, by means of an experiment involving 45 stimuli and 30 observers. The results suggest that shaping gestures aimed at manifesting the inner state of the robot (e.g., cheering or showing disappointment) tends to change the perception of Animacy (the dimension that accounts for how driven by endogenous factors the robot is perceived to be), while shaping gestures aimed at achieving an interaction effect (e.g., engaging and disengaging) tends to change the perception of Anthropomorphism, Likeability and Perceived Safety (the dimensions that account for the social aspects of the perception).

The principles governing written communication have been well studied, and well incorporated in interactive computer systems. However, the role of spoken language and in human-computer interaction, while an increasingly popular modality, still needs to be explored further [3]. Evidence suggests that this technology must further evolve in order to support more "natural" conversations [2], and that the use of speech interfaces is correlated with a high cognitive demand and attention [4]. In the context of spoken dialogue systems, a continuum has long been identified between "systeminitiative" interactions, where the system is in complete control of the overall interaction and the user answers a series of prescribed questions, and "user-initiative" interactions, where the user is free to say anything and the system must respond [5]. However, much of the work in this area predates the recent explosive growth of conversational interfaces.

This work investigates the role of the attraction paradigm - the tendency to associate similarity and attraction in interpersonal relations - in Human-Robot Interaction. The experiment presented here involved 30 human observers who watched and rated 45 robotic gestures in terms of Big-Five personality traits and Godspeed scores. The results show that, for 24 of the 30 observers, there was a statistically significant correlation between the Godspeed scores and the perceived similarity between the robot's personality and their own. However, the association was positive for 15 subjects - meaning that for these there is a similarity-attraction effect - and negative for the other 9 - meaning that for these there is a complementarity-attraction effect. Furthermore, the strength of the effect depends on the particular trait under examination.

This work explores the role of personality as a mediation variable between the observable behaviour of a robot - gestures of different energy and spatial extension in the experiments of this work - and the subjective experience of its users as measure by the Godspeed questionnaire. The results show that, at least for some traits, the Big Five personality traits that the users attribute to a robot are predictive of the Godspeed scores, i.e., of the quality of the interaction the users have with the robot. In other words, robots that are attributed different personality traits tend to be perceived differently in relation to the quality of the interaction.

This work investigates the relationship between the perception that people develop about a robot and the understandability of the gestures the latter displays. The experiments have involved 30 human observers that have rated 45 robotic gestures in terms of the Godspeed dimensions. At the same time, the observers have assigned a score to 10 possible interpretations (the same interpretations for all gestures). The results show that there is a statistically significant correlation between the understandability of the gestures - measured through an information theoretic approach - and all Godspeed scores. However, the correlation is positive in some cases (Anthropomorphism, Animacy and Perceived Intelligence), but negative in others (Perceived Safety and Likeability). In other words, higher understandability is not necessarily associated with more positive perceptions.

This article examines the educational efficacy of a learning environment in which children diagnosed with Autism Spectrum Conditions (ASC) engage in social interactions with an artificially intelligent (AI) virtual agent and where a human practitioner acts in support of the interactions. A multi-site intervention study in schools across the UK was conducted with 29 children with ASC and learning difficulties, aged 4--14 years old. For reasons related to data completeness and amount of exposure to the AI environment, data for 15 children was included in the analysis. The analysis revealed a significant increase in the proportion of social responses made by ASC children to human practitioners. The number of initiations made to human practitioners and to the virtual agent by the ASC children also increased numerically over the course of the sessions. However, due to large individual differences within the ASC group, this did not reach significance. Although no evidence of transfer to the real-world post-test was shown, anecdotal evidence of classroom transfer was reported. The work presented in this article offers an important contribution to the growing body of research in the context of AI technology design and use for autism intervention in real school contexts. Specifically, the work highlights key methodological challenges and opportunities in this area by leveraging interdisciplinary insights in a way that (i) bridges between educational interventions and intelligent technology design practices, (ii) considers the design of technology as well as the design of its use (context and procedures) on par with one another, and (iii) includes design contributions from different stakeholders, including children with and without ASC diagnosis, educational practitioners, and researchers.

This paper outlines a demonstration of the work carried out in the SoCoRo project investigating how far a neuro-typical population recognises facial expressions on a non-naturalistic robot face that are designed to show approval and disapproval. RFID-tagged objects are presented to an Emys robot head (called Alyx) and Alyx reacts to each with a facial expression. Participants are asked to put the object in a box marked 'Like' or 'Dislike'. This study is being extended to include assessment of participants' Autism Quotient using a validated questionnaire as a step towards using a robot to help train high-functioning adults with an Autism Spectrum Disorder in social signal recognition.

In this demonstration we present a repertoire of social signals generated by the humanoid robot Pepper in the context of the EU-funded project MuMMER. The aim of this research is to provide the robot with the expressive capabilities required to interact with people in real-world public spaces such as shopping malls-and being able to control the non-verbal behaviour of such a robot is key to engaging with humans in an effective way. We propose an approach to modulating the non-verbal social signals of the robot based on systematically varying the amplitude and speed of the joint motions and gathering user evaluations of the resulting gestures. We anticipate that the humans' perception of the robot behaviour will be influenced by these modulations

The papers in this special section focus on the concept of laughter computing. Laughter is considered a significant feature of human-human communication. Laughter is characterized by a complex behavior that includes major modules: auditory, facial expressions, body movements, and postural attitudes, and physiological signals. The goal of this special section is to gather recent achievements in laughter computing in order to trigger new research directions in this area.

We describe how the state representation of a socially aware robot is being extended to handle uncertainty. It incorporates the full range of information provided by the input sensors, including the confidence of all hypotheses. We also show how the Interaction Manager is being updated to make use of the extended representation.

The Workshop on Multimodal, Multi-Party, Real-World Human-Robot Interaction will be held in Istanbul on 16 November 2014, co-located with the 16th International Conference on Multimodal Interaction (ICMI 2014). The workshop objective is to address the challenges that robots face when interacting with humans in real-world scenarios. The workshop brings together researchers from intention and activity recognition, person tracking, robust speech recognition and language processing, multimodal fusion, planning and decision making under uncertainty, and service robot design. The programme consists of two invited talks, three long paper talks, and seven late-breaking abstracts. Information on the workshop and pointers to workshop papers and slides can be found at http://www.macs.hw.ac.uk/~mef3/icmi-2014-workshop-hri/.

In this paper we present results from a user evaluation of a robot bartender system which handles state uncertainty derived from speech input by using belief tracking and generating appropriate clarification questions. We present a combination of state estimation and action selection components in which state uncertainty is tracked and exploited, and compare it to a baseline version that uses standard speech recognition confidence score thresholds instead of belief tracking. The results suggest that users are served fewer incorrect drinks when the uncertainty is retained in the state.

This paper presents results from a user evaluation of a robot bartender system, which supports social engagement and interaction with multiple customers. The system is a Nao-based alternative version of an existing robot bartender developed in the JAMES project [1]. The Nao-based version has given us a local experimentation platform, allowing us to focus on social multi-user interaction rather than the robot technology of object manipulation. We will describe the design of the Nao-based system and discuss the differences with the original JAMES system. In a recent evaluation of the JAMES system with real users, a trained and a hand-coded version of the action selection policy were compared [2]. Here we present results from a similar comparative user evaluation on the Nao-based system, which confirm the conclusions of the previous experiment and provide further evidence in favour of the trained action selection mechanism. Task success was found to be almost 20% higher with the trained policy, with interaction times being about 10% shorter. Participants also rated the trained system as significantly more natural, more understanding, and better at providing appropriate attention.

We describe a variety of machine-learning techniques that are being applied to social multiuser human--robot interaction using a robot bartender in our scenario. We first present a data-driven approach to social state recognition based on supervised learning . We then describe an approach to social skills execution—that is, action selection for generating socially appropriate robot behavior—which is based on reinforcement learning , using a data-driven simulation of multiple users to train execution policies for social skills. Next, we describe how these components for social state recognition and skills execution have been integrated into an end-to-end robot bartender system, and we discuss the results of a user evaluation. Finally, we present an alternative unsupervised learning framework that combines social state recognition and social skills execution based on hierarchical Dirichlet processes and an infinite POMDP interaction manager. The models make use of data from both human--human interactions collected in a number of German bars and human--robot interactions recorded in the evaluation of an initial version of the system.

A robot agent existing in the physical world must be able to understand the social states of the human users it interacts with in order to respond appropriately. We compared two implemented methods for estimating the engagement state of customers for a robot bartender based on low-level sensor data: a rule-based version derived from the analysis of human behaviour in real bars, and a trained version using supervised learning on a labelled multimodal corpus. We first compared the two implementations using cross-validation on real sensor data and found that nearly all classifier types significantly outperformed the rule-based classifier. We also carried out feature selection to see which sensor features were the most informative for the classification task, and found that the position of the head and hands were relevant, but that the torso orientation was not. Finally, we performed a user study comparing the ability of the two classifiers to detect the intended user engagement of actual customers of the robot bartender; this study found that the trained classifier was faster at detecting initial intended user engagement, but that the rule-based classifier was more stable.

We address the question of whether service robots that interact with humans in public spaces must express socially appropriate behaviour. To do so, we implemented a robot bartender which is able to take drink orders from humans and serve drinks to them. By using a high-level automated planner, we explore two different robot interaction styles: in the task only setting, the robot simply fulfils its goal of asking customers for drink orders and serving them drinks; in the socially intelligent setting, the robot additionally acts in a manner socially appropriate to the bartender scenario, based on the behaviour of humans observed in natural bar interactions. The results of a user study show that the interactions with the socially intelligent robot were somewhat more efficient, but the two implemented behaviour settings had only a small influence on the subjective ratings. However, there were objective factors that influenced participant ratings: the overall duration of the interaction had a positive influence on the ratings, while the number of system order requests had a negative influence. We also found a cultural difference: German participants gave the system higher pre-test ratings than participants who interacted in English, although the post-test scores were similar.

We introduce a humanoid robot bartender that is capable of dealing with multiple customers in a dynamic, multi-party social setting. The robot system incorporates state-of-the-art components for computer vision, linguistic processing, state management, high-level reasoning, and robot control. In a user evaluation, 31 participants interacted with the bartender in a range of social situations. Most customers successfully obtained a drink from the bartender in all scenarios, and the factors that had the greatest impact on subjective satisfaction were task success and dialogue efficiency.

The development of social communication skills in children relies on multimodal aspects of communication such as gaze, facial expression, and gesture. We introduce a multimodal learning environment for social skills which uses computer vision to estimate the children's gaze direction, processes gestures from a large multi-touch screen, estimates in real time the affective state of the users, and generates interactive narratives with embodied virtual characters. We also describe how the structure underlying this system is currently being extended into a general framework for the development of interactive multimodal systems.

This book gives a short but comprehensive overview of the field of spoken dialogue systems, outlining the issues involved in building and evaluating this type of system and making liberal use of techniques and examples from a wide range of implemented systems.It provides an excellent review of the research, with particularly relevant discussions of error handling and system evaluation, and is suitable both as an introduction to this research area and as a survey of current state-of-the-art techniques.The book is structured into seven chapters.Chapter 1 provides an introduction to the research area and briefly introduces the topics covered in the book.The end of the chapter consists of a list of links to tools and components that can be used for dialogue system development, which-although currently useful-seems likely to go out of date quickly.Chapter 2 addresses the task of dialogue management, beginning by describing simple graph-and frame-based methods for dialogue control, and continuing with a discussion of VoiceXML.The chapter ends with an extended discussion of recent work in statistical approaches to dialogue control and modeling.It is unfortunate that the discussion of other methods such as the information state approach and plan-based models is postponed to Chapter 4, but otherwise this chapter provides a good summary of both classic and recent approaches.Chapter 3 discusses error handling, which is divided into three processes: error detection, error prediction (i.e., the online prediction of errors based on dialogue features), and error recovery.After surveying a range of previous approaches to these three subtasks, the authors go on to discuss several more recent, data-driven approaches.Error handling is both a vital component of any spoken dialogue system designed for realworld use and an active area of current research, so this compact summary of techniques and issues is welcome.Chapter 4 contains case studies illustrating a range of dialogue control strategies and models.It begins with a description of the information state approach, and then moves on to discuss plan-based approaches as exemplified in the TRAINS and TRIPS projects.This is followed by a discussion of two systems that employ software agents for dialogue management: the Queen's Communicator and the AthosMail system.Finally, two systems which make use of statistical models are presented: the Microsoft Bayesian receptionist, which models conversation as decision making under uncertainty, and the DIHANA system, which employs corpus-based dialogue management.The case studies in this chapter provide detailed examples of a range of techniques, along with

Generating referring expressions is a task that has received a great deal of attention in the natural-language generation community, with an increasing amount of recent effort targeted at the generation of multimodal referring expressions. However, most implemented systems tend to assume very little shared knowledge between the speaker and the hearer, and therefore must generate fully-elaborated linguistic references. Some systems do include a representation of the physical context or the dialogue context; however, other sources of contextual information are not normally used. Also, the generated references normally consist only of language and, possibly, deictic pointing gestures.

We describe a human-robot dialogue system that allows a human to collaborate with a robot agent on assembling construction toys. The human and the robot are fully equal peers in the interaction, rather than simply partners. Joint action is supported at all stages of the interaction: the participants agree on a construction task, jointly decide how to proceed to proceed with the task, and also implement the selected plans jointly. The symmetry provides novel challenges for a dialogue system, and also makes it possible for findings from human-human joint-action dialogues to be easily implemented and tested.