As we introduce social robots into family homes, there is a heightened need to understand both parental and child perceptions of privacy, taking into account family perspectives. Haru4Kids (H4K) is an app-based robot simulator designed to cohabitate with children in their home. In a multi-site investigation, H4K cohabitated with 11 American, 11 Spanish, and 10 Ukrainian families. Interviews preceding and following the cohabitation period with at least one parent and all participating children ages 5–12 allowed us to gauge general expectations and experiences during cohabitation and to estimate their comfort with sharing different kinds of information with the platform. In general, children perceived Haru as a technological device that was social, yet non-human. All stakeholders were more comfortable with general information collection as opposed to information sharing with third parties. Lastly, though culture is known to affect expectations and perceptions of privacy and robots, our study shows cohabitating with H4K led to a general convergence across sites in users’ perceptions of Haru and attitudes towards preserving their privacy with robots.

Hospitalized pediatric cancer patients often experience anxiety. Social robots have been proposed as intelligent monitors, embodied mediators, and embodied companions to watch over and support children with such distress. To do so, we propose to augment social robots with biosensors. ElectroDermal Activity recordings of ±22-hour were collected from 8 in-hospital pediatric cancer patients and 6 survivors outside the hospital, together with their diaries and an anxiety questionnaire. To optimally exploit the limited data gathered, external datasets were used to build classification models, with the best performing model achieving a cross-validated F1-score of 0.59 (SD=0.12) on the test set. The models delivered promising out-of-distribution predictions of high-arousal on the pediatric recordings. The limited number of labels available did not allow validation of all high-arousal segments, confirming the challenge of gathering reliable ground truth labels in pediatric hospitals. We suggest data collection methods to foster the further development of augmented social robots.

Social robots are beginning to be utilized as part of the collective networks supporting pediatric treatment, however there are few studies on children's perceptions of these agents in hospitals from a privacy and safety perspective. Through a mixed-method and value-sensitive design approach, we introduced hypothetical vignettes and engaged in discussion with 15 youth who are either receiving cancer treatments or are in remission (ages 6-25), 11 of their parents, and 5 out of 8 of their clinical staff to learn how stakeholders in pediatric oncology discuss privacy concerns regarding child-robot interactions. Our thematic analysis imparts how stakeholders perceive robots as social, non-authoritative extensions of the hospital's care network. From this privacy-sensitive perspective, this study revealed that for maximizing a robot's social utility within care systems while critically engaging with the comfort and privacy preferences of stakeholders, robots should take on the role of 1) mediators of social interaction among various stakeholders, 2) companions for children and 3) informational tools for clinicians when consent is given by the family. We emphasize how assistive technologies in pediatrics should continue to be co-designed within communities for identifying appropriate roles and returning agency to stakeholders as they navigate the blurry boundaries of privacy in healthcare.

Haru4Kids (H4K) is a system that emulates the physical, social, family-oriented robot Haru, designed with the goal to cohabitate with children in their home for extended periods of time. Seven families kept H4K for two net weeks in their homes. Throughout this period of cohabitation, we collected user logs comprised of the children users ’ head angles, the rotation angles of the platform, and the actions taken by H4K as well as captured images which were afterwards hand-annotated to estimate user engagement. We report the trends of these external metrics that we collected during every session of interaction. We also developed an annotation tool and report the Engagement Level Metric we chose to estimate child engagement throughout interactions “in-the-wild.” Overall, our platform offers a feasible system that can engage with children while also allowing us to monitor their engagement and behaviour throughout each interaction.

Child-robot interaction (CRI) has been mostly studied in labs and classroom settings. In this work, we share a CRI language processing study carried out in children’s homes. Any automated system deployed “in-the-wild” faces practical problems, but when the target users are children, these problems get even more sensitive and challenging. In this work we analyse how each language processing layer performs with children at home with no researcher present. We carried out an experiment with 7 families [N=14 children, 6-13 years old] cohabiting with a simulated robot for 2 weeks in their own homes. Our goal in this study is to evaluate the performance of voice recognition, language understanding and dialogue management when children interact with a robot at home. Our results indicate that dialogue management capabilities are becoming the key element in the language processing pipeline; they also denote that the dialogue engine should include mixed-initiative capabilities and show the relative usage of different common built-in intents.

This is a large extension to a previous paper presented in LREC 2006 [6]. It describes the motivation, collection and format of the MIMUS corpus, as well as an in-depth and issue-focused analysis of the data. MIMUS [8] is the result of multimodal WoZ experiments conducted at the University of Seville as part of the TALK project. The main objective of the MIMUS corpus was to gather information about different users and their performance, preferences and usage of a multimodal multilingual natural dialogue system in the Smart Home scenario in Spanish. The focus group is composed by wheel-chair-bound users, because of their special motivation to use this kind of technology, along with their specific needs. Throughout this article, the WoZ platform, experiments, methodology, annotation schemes and tools, and all relevant data will be discussed, as well as the results of the in-depth analysis of these data. The corpus compresses a set of three related experiments. Due to the limited scope of this article, only some results related to the first two experiments (1A and 1B) will be discussed. This article will focus on subject's preferences, multimodal behavioural patterns and willingness to use this kind of technology.

One of the most widely pursued goals in dialogue system development is the improvement of usability, which is mainly achieved by providing users with both friendly and manageable interfaces. The MIMUS dialogue system supports multimodal interactions, allowing the user to interact not only verbally but also graphically. In addition to this, MIMUS (MultIModal, University of Seville) is a multilingual system that enables the user to communicate (dynamically) in Spanish and English. The present paper describes the MIMUS architecture, the components that entail direct relationship with multilinguality, and the way in which languages can be dynamically switched within a single dialogue. Index Terms : dialogue systems, multimodal, multilingual, dynamic language change

This is a new hybrid fusion strategy based primarily on the implementation of two former and differentiated approaches to multimodal fusion [11] in multimodal dialogue systems. Both approaches, their predecessors and their respective advantages and disadvantages will be described in order to illustrate how the new strategy merges them into a more solid and coherent solution. The first strategy was largely based on Johnston's approach [5] and implies the inclusion of multimodal grammar entries and temporal constraints. The second approach implied the fusion of information coming from different channels at dialogue level. The new hybrid strategy hereby described requires the inclusion of multimodal grammar entries and temporal constraints plus the additional information at dialogue level utilized in the second strategy. Within this new approach therefore, the fusion process will be initiated at grammar level and will be culminated at dialogue level.

This paper describes the architecture of MIMUS, a multimodal, multilingual interaction system which allows users to control the devices in their houses, interacting by voice and clicks. MIMUS design relies on Wizard of Oz experiments and is targeted at disabled users in the in-home scenario. MIMUS consists of a set of OAA collaborative agents with a clear distinction between knowledge resources, dialogue management and domain specific functionalities. The MIMUS system includes contributions to the scientific community in multimodal fusion, knowledge management and presentation strategies.