As we introduce social robots into family homes, there is a heightened need to understand both parental and child perceptions of privacy, taking into account family perspectives. Haru4Kids (H4K) is an app-based robot simulator designed to cohabitate with children in their home. In a multi-site investigation, H4K cohabitated with 11 American, 11 Spanish, and 10 Ukrainian families. Interviews preceding and following the cohabitation period with at least one parent and all participating children ages 5–12 allowed us to gauge general expectations and experiences during cohabitation and to estimate their comfort with sharing different kinds of information with the platform. In general, children perceived Haru as a technological device that was social, yet non-human. All stakeholders were more comfortable with general information collection as opposed to information sharing with third parties. Lastly, though culture is known to affect expectations and perceptions of privacy and robots, our study shows cohabitating with H4K led to a general convergence across sites in users’ perceptions of Haru and attitudes towards preserving their privacy with robots.

Social robots for children have focused mainly on conventional education domains such as teaching language, science, and math, while applications focusing on the enhancement of cultural competency are quite scarce. In this paper, we present a prototype of a robot-mediation framework for cross-cultural communication. This framework paves the way for a social robot to act as a mediator between groups of schoolchildren from different countries. First, we conducted a participatory design activity by an interdisciplinary team, resulting in the extraction of the design, robot’s roles, and technical requirements. Based on these requirements, we built the robot-mediation system prototype. We conducted a pilot study using the system with groups of high school children in Japan and Australia and our results show the potential of the system to drive children’s interest in communicating, sharing, and discussing cultural themes with their remote peers through the social robot.

Haru4Kids (H4K) is a system that emulates the physical, social, family-oriented robot Haru, designed with the goal to cohabitate with children in their home for extended periods of time. Seven families kept H4K for two net weeks in their homes. Throughout this period of cohabitation, we collected user logs comprised of the children users ’ head angles, the rotation angles of the platform, and the actions taken by H4K as well as captured images which were afterwards hand-annotated to estimate user engagement. We report the trends of these external metrics that we collected during every session of interaction. We also developed an annotation tool and report the Engagement Level Metric we chose to estimate child engagement throughout interactions “in-the-wild.” Overall, our platform offers a feasible system that can engage with children while also allowing us to monitor their engagement and behaviour throughout each interaction.

Child-robot interaction (CRI) has been mostly studied in labs and classroom settings. In this work, we share a CRI language processing study carried out in children’s homes. Any automated system deployed “in-the-wild” faces practical problems, but when the target users are children, these problems get even more sensitive and challenging. In this work we analyse how each language processing layer performs with children at home with no researcher present. We carried out an experiment with 7 families [N=14 children, 6-13 years old] cohabiting with a simulated robot for 2 weeks in their own homes. Our goal in this study is to evaluate the performance of voice recognition, language understanding and dialogue management when children interact with a robot at home. Our results indicate that dialogue management capabilities are becoming the key element in the language processing pipeline; they also denote that the dialogue engine should include mixed-initiative capabilities and show the relative usage of different common built-in intents.

Conversation can play an essential role in forging bonds between humans and social robots, but participants need to feel like they are being listened to, remembered, and cared about in order to effectively build rapport. In this paper, we propose a novel strategy for conducting small talk with a social robot. Our approach is known as the Tiers of Friendship. It is centered around three core design elements: 1) Persuasive content and character is provided through topic modules created by professional creative writers to ensure engaging conversational content and a compelling personality for the social robot. 2) Conversational memory is achieved by allowing topic modules to specify required information that can be learned through conversation or recalled from previous interactions and organizing topic modules into a hierarchy that enforces information requirements between topics. 3) Dynamicity in conversation is promoted through topic navigation that supports fluid transitions to topics of human interest and employs elements of random ordering to create fresh conversation experiences. In this paper, we show how the Tiers of Friendship can be used to generate conversation content for a social robot that encourages the development of rapport. We describe a working implementation of a small talk system for a social robot based on the Tiers of Friendship that combines off-the-shelf ASR and NLU components and custom robot behavior components implemented via behavior trees on ROS. Finally, in order to evaluate our approach's effectiveness, we conduct an elicitation survey that evaluates conversations in terms of perceived engagement, personality traits, and rapport expectation and discuss the implications for social robotics.

This is a large extension to a previous paper presented in LREC 2006 [6]. It describes the motivation, collection and format of the MIMUS corpus, as well as an in-depth and issue-focused analysis of the data. MIMUS [8] is the result of multimodal WoZ experiments conducted at the University of Seville as part of the TALK project. The main objective of the MIMUS corpus was to gather information about different users and their performance, preferences and usage of a multimodal multilingual natural dialogue system in the Smart Home scenario in Spanish. The focus group is composed by wheel-chair-bound users, because of their special motivation to use this kind of technology, along with their specific needs. Throughout this article, the WoZ platform, experiments, methodology, annotation schemes and tools, and all relevant data will be discussed, as well as the results of the in-depth analysis of these data. The corpus compresses a set of three related experiments. Due to the limited scope of this article, only some results related to the first two experiments (1A and 1B) will be discussed. This article will focus on subject's preferences, multimodal behavioural patterns and willingness to use this kind of technology.

One of the most widely pursued goals in dialogue system development is the improvement of usability, which is mainly achieved by providing users with both friendly and manageable interfaces. The MIMUS dialogue system supports multimodal interactions, allowing the user to interact not only verbally but also graphically. In addition to this, MIMUS (MultIModal, University of Seville) is a multilingual system that enables the user to communicate (dynamically) in Spanish and English. The present paper describes the MIMUS architecture, the components that entail direct relationship with multilinguality, and the way in which languages can be dynamically switched within a single dialogue. Index Terms : dialogue systems, multimodal, multilingual, dynamic language change

This is a new hybrid fusion strategy based primarily on the implementation of two former and differentiated approaches to multimodal fusion [11] in multimodal dialogue systems. Both approaches, their predecessors and their respective advantages and disadvantages will be described in order to illustrate how the new strategy merges them into a more solid and coherent solution. The first strategy was largely based on Johnston's approach [5] and implies the inclusion of multimodal grammar entries and temporal constraints. The second approach implied the fusion of information coming from different channels at dialogue level. The new hybrid strategy hereby described requires the inclusion of multimodal grammar entries and temporal constraints plus the additional information at dialogue level utilized in the second strategy. Within this new approach therefore, the fusion process will be initiated at grammar level and will be culminated at dialogue level.

This paper describes the architecture of MIMUS, a multimodal, multilingual interaction system which allows users to control the devices in their houses, interacting by voice and clicks. MIMUS design relies on Wizard of Oz experiments and is targeted at disabled users in the in-home scenario. MIMUS consists of a set of OAA collaborative agents with a clear distinction between knowledge resources, dialogue management and domain specific functionalities. The MIMUS system includes contributions to the scientific community in multimodal fusion, knowledge management and presentation strategies.