While research in multi-party spoken conversation with intelligent embodied agents has made significant progress in sub-tasks like speaker identification and non-verbal cues, there’s a gap in fully autonomous applications users can directly interact with. This lack translates to the absence of a standard methodology for evaluating multi-party conversational speech agents that considers both task-based system performance and user experience.

Angus Addlesee, Weronika Sieińska, Nancie Gunson, Daniel Hernandez Garcia, Christian Dondrup, Oliver Lemon. Proceedings of the 24th Meeting of the Special Interest Group on Discourse and Dialogue. 2023.

Possible applications for Social Robots in health-care settings, that could have a tremendous social impact in helping alleviating staff workload, are those of a patient-facing role such as robot receptionist, providing assistance to patients and visitors. Examples of functions that such robots would need to be able to execute are greeting visitors, reception check-in/out of patients, answering common questions they may have, showing them where to sit, helping them locate missing objects, providing directions to facilities, guiding them to different locations, etc. In this paper we describe current progress towards developing a multimodal conversational AI system integrated in a Social Conversational Robot (an ARI robot) that will act as a receptionist in a hospital waiting room. We present the developed architecture of the system and report on an initial experimental validation study carried out in laboratory conditions with the ARI robot.

Nancie Gunson, Daniel Hernandez Garcia, Weronika Sieińska, Angus Addlesee, Christian Dondrup, Oliver Lemon, Jose L. Part, Yanchao Yu. Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue. 2022.

We will demonstrate a prototype multimodal conversational AI system that will act as a receptionist in a hospital waiting room, combining visually-grounded dialogue with social conversation. The system supports visual object conversation in the waiting room (e.g. looking for available seats or personal belongings), task-based dialogues regarding navigation and check-in procedures in the hospital, as well as access to the latest news, and a quiz game about coronavirus. The prototype system therefore demonstrates how to weave together a wide range of natural, daily conversations with end users that vary in complexity; from complex visual dialogue to chitchat and quiz games, to task-oriented domain-specific conversations. We are currently able to demonstrate the system via a web-based interface. It will soon be deployed on the ARI robot in a hospital waiting room.

We present and evaluate a deployed conversational AI system that acts as a host of a working public building on a university campus. The system combines open-domain social chat with task-based conversation regarding navigation in the building, live resource updates (e.g. available computers), and events in the building. We investigated the impact of open-domain social chat on task completion and user preferences by comparing the combined system with a task-only version. We find that there is no significant difference in task completion or several aspects of user preference between the two systems, but that users would be significantly happier to talk to the task-only system in the future. This suggests that the "walk-up" public setting and workplace nature of the environment creates a markedly different use case to the in-home, and more individual and private "companion/assistant" setting which is commonly assumed for systems like Alexa. We discuss the implications for the design of conversational systems in other public settings.

We will demonstrate a deployed conversational AI system that acts as a host of a smartbuilding on a university campus.The system combines open-domain social conversation with task-based conversation regarding navigation in the building, live resource updates (e.g.available computers) and events in the building.We are able to demonstrate the system on several platforms: Google Home devices, Android phones, and a Furhat robot.

This paper describes an experiment to investigate the usability of voiceprints for customer authentication in automated telephone banking. The usability of voiceprint authentication using digits (random strings and telephone numbers) and sentences (branded and unbranded) are compared in a controlled experiment with 204 telephone banking customers. Results indicate high levels of usability and customer acceptance for voiceprint authentication in telephone banking. Customers find voiceprint authentication based on digits more usable than that based on sentences, and a majority of participants would prefer to use digits.

Abstract The SPELL (Spoken Electronic Language Learning) system is a self‐access computer‐assisted language learning (CALL) package that integrates speaker‐independent continuous speech recognition technology with virtual worlds and embodied virtual agents to create an environment in which learners can converse in the target language within meaningful contextualized scenarios. In this paper we provide an overview of the functionality, architecture, and implementation of the SPELL system. We also describe four phases of usability evaluation conducted with the system and summarize the main results of these user assessments. Finally, we discuss the most significant lessons learned in the development and evaluation of the system. The paper focuses on the technological aspects of the system and its evaluation for usability and robustness, rather than its pedagogical methodology. Copyright © 2008 John Wiley & Sons, Ltd.