In this study, we developed SCAPED, a platform for running mass experiments with voice-based conversational agents. The main purpose of the toolkit is to study particular aspects of dialogues with an agent. The platform can be integrated into crowdsourcing platforms, such as Prolific as a source for participants and only requires a web browser. It can therefore be used in large-scale experiments. We present a proof-of-concept study which addresses the question of laughter accompanying apologies in a dialogues about the assessments of artworks.

We explore how laughter functions in in-the-wild human–Alexa interactions recorded in domestic settings. To do so, we analysed all instances of laughter in a corpus containing audio recordings from six households where an Alexa device had been newly acquired. The participants had no or very limited prior experience with voice assistants. Their interactions with Alexa were recorded over the first seven to ten weeks of use. Unlike previous HRI studies that primarily focus on dyadic, task-based exchanges, our analysis reveals that laughter in these real-world settings often emerges in multiparty interactions and serves a range of social functions beyond direct responses to the device. These observations highlight not only the need for ecologically grounded models of laughter in human–robot interaction, but also the value of linguistic and interactional analysis in uncovering the nuanced communicative roles laughter plays in everyday technology use. Such an approach allows us to identify how laughter signals both matches and mismatches in communication by marking alignment, managing breakdowns, and negotiating social meaning in interactions that often involve more than just the user and the device.

Gaze and laughter play a crucial role in managing miscommunication and coordinating social interactions. We hypothesise that models of laughter and gaze coordination in human dialogue extend to virtual entities. This paper describes methodology of the future experiment which involves a socially interactive agent (SIA) that incorporates previous theoretical findings.

Gaze cues, which initiate an action or behaviour, are necessary for a responsive and intuitive interaction. Using gaze to signal intentions or request an action during conversation is conventional. We propose a new approach to estimate gaze using a neural network architecture, while considering the dynamic patterns of real world gaze behaviour in natural interaction. The main goal is to provide foundation for robot/avatar to communicate with humans using natural multimodal-dialogue. Currently, robotic gaze systems are reactive in nature but our Gaze-Estimation framework can perform unified gaze detection, gaze-object prediction and object-landmark heatmap in a single scene, which paves the way for a more proactive approach. We generated 2.4M gaze predictions of various types of gaze in a more natural setting (GHIGaze). The predicted and categorised gaze data can be used to automate contextualized robotic gaze-tracking behaviour in interaction. We evaluate the performance on a manually-annotated data set and a publicly available gaze-follow dataset. Compared to previously reported methods our model performs better with the closest angular error to that of a human annotator. As future work, we propose an implementable gaze architecture for a social robot from Furhat robotics1.1https://furhatrobotics.com/

Laughter and gaze have an important role in managing and coordi-nating social interactions. In the current work, using a multimodal corpus of dyadic taste-testing interactions, we explore whether laughs performing different pragmatic functions are accompanied by different gaze patterns towards the interlocutor, both from the point of view of the laughing participant and from her partner. We also investigate the role of gaze in laughter coordination between interactants. Our results show that laughs performing different pragmatic functions are related to different gaze patterns, both for the laugher and her partner, and that gaze is an important cue exploited by interactants when reciprocating laughter or laughing simultaneously. We discuss our data in relation to the literature about laughter and gaze functions in interaction, linking them to dialogic context. Our results stress the importance of laughter and gaze for modeling of multimodal meaning construction and coordination in interaction, and are therefore relevant for researchers designing human-like embodied conversational agents.

This paper introduces an approach for annotating eye gaze considering both its social and the referential functions in multi-modal human-human dialogue. Detecting and interpreting the temporal patterns of gaze behavior cues is natural for humans and also mostly an unconscious process. However, these cues are difficult for conversational agents such as robots or avatars to process or generate. The key factor is to recognize these variants and carry out a successful conversation, as misinterpretation can lead to total failure of the given interaction. This paper introduces an annotation scheme for eye-gaze in human-human dyadic interactions that is intended to facilitate the learning of eye-gaze patterns in multi-modal natural dialogue.