Selma Sabanovic

dblp:83/290 · DBLP profile ↗
← Back
106ranked-venue papers
9as first author
51since 2021 · last 2026
0000-0002-2553-854XORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Human-computer interaction and ubiquitous computing · 103 · 9 first-author · 50 since 2021Artificial intelligence and machine learning · 69 · 1 first-author · 32 since 2021Applied, interdisciplinary, general and emerging computing · 30 · 14 since 2021Systems, architecture and hardware · 2 · 1 since 2021
YearPublicationVenuePosition
2026 Rethinking Pedagogical Relationships in the Presence of Social Robots
abstract
As social robots enter educational settings, how they are positioned as new social actors within existing teacher-student pedagogical relationships becomes an important yet underexplored question. They may be shaped by existing relationships, while also having the potential to reconfigure them. In this study, we explore the relational positioning of educational social robots through a drawing-based design activity with teachers. By analyzing the pedagogical configurations imagined by teachers, we reveal how teachers assign roles, responsibilities, and degrees of authority to robots, in relation to themselves and students. Building on these findings, we discuss the design implications for future educational social robots, highlighting the importance of considering how robots are added into and reshape pedagogical relationships.
Zhennan Yi, Paulina Zguda, Randy Gomez, Selma Sabanovic
DIS5
2026 Robots that Evolve with Us: Modular Co-Design for Personalization, Adaptability, and Sustainability
abstract
Many current robot designs prioritize efficiency and one-size-fits-all solutions, oftentimes overlooking personalization, adaptability, and sustainability. To explore alternatives, we conducted two co-design workshops with 23 participants, who engaged with a modular robot co-design framework. Using components we provided as building blocks, participants combined, removed, and invented modules to envision how modular robots could accompany them from childhood through adulthood and into older adulthood. The participants’ designs illustrate how modularity (a) enables personalization through open-ended configuration, (b) adaptability across shifting life-stage needs, and (c) sustainability through repair, reuse, and continuity. We therefore derive design principles that establish modularity as a foundation for lifespan-oriented human–robot interaction. This work reframes modular robotics as a flexible and expressive co-design approach, supporting robots that evolve with people, rather than static products optimized for single moments or contexts of use.
Qing Xiao 0002, Zitao Zhang, Eli Blevis, Selma Sabanovic
CHI5
2026 Tracking Together: A Robot-and-App-Based Speech Analysis System to Support Shared Meaning-Making Among Dementia Care Partners
abstract
Tracking for people living with dementia and their care partners is primarily focused on quantified dementia symptoms presented to care partners. However, what people living with dementia want to track, what other aspects of dementia care partners wish to know, and how tracking fits within the care relationship remain to be identified. We performed an exploratory study in which eight people living with dementia and nine care partners provided iterative design feedback on a system concept: one that captures conversational data from a robot and visualizes it through a speech-tracking mobile application. Through reflexive thematic analysis, we found that people living with dementia wanted to use the system to maintain autonomy, especially by talking about their symptoms with the robot and using tracked information as a memory aid. Care partners valued numerical insights into the cognitive progress of people living with dementia only when accompanied by clear calls to action that supported them in their caregiver roles. Simultaneously, in their relational roles as spouses or children, care partners valued tracking memories and discussion points to understand their loved ones better. Our results suggest that providing related but distinct information tailored to each user’s needs can support both in their care relationship.
Long-Jing Hsu, Alex Foster, Andrew Murphy, Rohith Perumandla, Jennifer Schwabe, Cedomir Stanojevic, Casey C. Bennett, Selma Sabanovic
CHI9
2026 Social Robots as Active Safeguards for Children's Welfare: Community Stakeholder Insights and Design Recommendations
abstract
Integrating robots into children's spaces demands careful attention to the likelihood that children may disclose information indicating that their welfare is at risk. To consider the active involvement of social robots in safeguarding children, we interviewed 22 community stakeholders who work with children professionally in education, social services, healthcare or legal services in the UK and the USA. All stakeholders saw value in having robots create playful, safe spaces that can help alleviate the anxiety and reduce the emotional burden of the safeguarding process. However, they worried about a robot's ability to handle disclosures, interpret context and not overburden welfare services. We present hypothetical roles that robots could adopt within the safeguarding pipeline and map them to design considerations that emphasise the need for relational understanding, structured thresholds, and transparent data collection. Together, these considerations provide directions for ethical, child-centred robot technologies in safeguarding youth.
Nida Itrat Abbasi, Leigh Levinson, Selma Sabanovic, Hatice Gunes
HRI3
2026 Generative Encounters with Robin: Design through Adaptation and Appropriation of a Social Robot in Four Eldercare Facilities
abstract
Iterative engagement conceptualizes robot design as a continuous process that extends beyond initial deployment, using real-life feedback from "generative encounters" with residents and staff to guide ongoing technology improvements. This study examines iterative development of "Robin the Robot"—a socially assistive robot deployed across multiple eldercare facilities—to investigate how contextual factors influence robot use, appropriation, and collaborative adaptation by developers and users. Using ethnographic methods, we conducted interviews with robot developers and care facility staff, along with on-site observations of interactions between older adults, care staff, and the robot. Our analysis reveals four key contextual factors that affect user perceptions and robot use: resident types within each facility, staff mental models of the robot, communication practices between staff and developers, and users' awareness of remote teleoperators. Our findings contribute new insights into how contextual factors shape eldercare robot adaptation and appropriation across diverse care institutions, providing practical guidance for designing socially assistive robots that can be effectively integrated into varied care environments.
Selma Sabanovic
HRI2
2026 Kept Alive, Bricked, Revived: Community Articulation Work and Value Renegotiation beyond a Robot's Commercial Failure
abstract
This paper examines the community-driven sustenance of Moxie, a social robot that faced discontinuation when its parent company failed to secure funding. Through interviews and investigative digital ethnography, we study how users performed extensive articulation work to transition from corporate support to an open-source platform. Our findings reveal that invisible labor and value negotiation were central to Moxie's continued operation, simultaneously opening access for some users while excluding others. These processes also fundamentally reshaped the robot's desirability and meaning within the user community. This work demonstrates how socio-technical infrastructures, articulation work, and value renegotiation can sustain robots beyond their commercial lifecycles, while revealing the uneven distribution of both labor and access in community-driven technology repair and maintenance.
Waki Kamino, Bengisu Cagiltay, Bilge Mutlu, Malte F. Jung, Selma Sabanovic
HRI5
2026 Let's Talk about (Our and Robots') Death: Mortality as a Core Principle in Human-Robot Interaction
abstract
As robots achieve product longevity and increasingly enter intimate spaces of human life, from healthcare facilities to homes, they inevitably encounter contexts involving death, dying, grief, and loss. Nevertheless, mortality as a fundamental relational dimension has received limited systematic attention within Human-Robot Interaction (HRI) research. Drawing from empirical vignettes across multiple research projects, from robots at deathbeds and participants who passed away, to owners planning for robots they will leave behind, we demonstrate how death already surfaces in HRI, even when not designed for it. Through analysis of these encounters, we develop a framework for mortality-aware HRI that identifies key dimensions for designing and researching robots that acknowledge human and robotic finitude. We argue that as robots achieve long-term integration into people's lives, creating intertwined lifespans of humans and machines, the field must move beyond treating mortality as an edge case and instead recognize it as a core principle shaping how people relate to and make meaning with robots.
Waki Kamino, Long-Jing Hsu, Selma Sabanovic, Malte F. Jung
HRI3
2026 What Is a Robot? Understanding Baseball's "Robot Umpire" through the Lens of Fluid Technology
abstract
The question “what is a robot?” has long been contested as automated embodied systems encompass many forms. We examine this fundamental question in Human-Robot Interaction through the case of Major League Baseball’s “robot umpire,” officially known as the Automated Ball-Strike System (ABS). Drawing on the concept of “fluid technology,” we analyze how the robot umpire is not a fixed technological artifact but a fluid sociotechnical assemblage whose definition and function are continuously negotiated. Through ethnographic fieldwork and interviews with stakeholders across the baseball ecosystem, we demonstrate that the robot umpire’s physical boundaries, operational parameters, and authorship remain contested and evolving, shaped by ongoing interactions between technology developers, league officials, umpires, players, and fans. Our findings reveal that treating robots as fluid technologies—rather than as discrete objects—opens new possibilities for understanding human-robot relationships. We contribute both theoretical insights regarding the ontological flexibility of “robots” and methodological approaches for studying and designing robots as sociotechnical assemblages.
Waki Kamino, Andrea W. Wen-Yi, Guy Hoffman, Selma Sabanovic, Malte F. Jung
HRI4
2026 BehaviorKit: A Multi-modal Real-Time Behavior Analysis Library for Robots
abstract
We present BehaviorKit, an open-source plug-and-play module that enables the addition of real-time behavior analysis using deep learning to any robot with internet access. The library bundles together gaze tracking, real-time transcription, textual sentiment analysis, facial emotion (valence and arousal) estimation, and face and pose landmark localization. The library is designed to be run on a GPU-powered computer, either on the robot or externally, to process the incoming visual and auditory inputs that the robot receives. The library connects via WebSocket to the robot, which receives the processed outputs from all of the models. The WebSocket client can easily connect to a ROS-powered robot, or a custom client can be written to adapt to any robot; alternatively, the library can also be run between two laptop computers. A small dataset is provided to test the framework. By packaging together and optimizing many commonly used models, we hope to enable easier access to high-performing behavior models for the HRI community. The code is available at https://github.com/IUB-RHouse/BehaviorKit.
William Valentine, Selma Sabanovic, David Crandall, Weslie Khoo
HRI2
2026 Adding More Value Than Work: Practical Guidelines for Integrating Robots into Intercultural Competence Learning
abstract
While social robots have demonstrated effectiveness in supporting students' intercultural competence development, it is unclear how they can effectively be adopted for integrated use in K-12 schools. We conducted two phases of design workshops with teachers, where they co-designed robot-mediated intercultural activities while considering student needs and school integration concerns. Using thematic analysis, we identify appropriate scenarios and roles for classroom robots, explore how robots could complement rather than replace teachers, and consider how to address ethical and compliance considerations. Our findings provide practical design guidelines for the HRI community to develop social robots that can effectively support intercultural education in K-12 schools.
Zhennan Yi, Sophia Sakakibara Capello, Randy Gomez, Selma Sabanovic
HRI4
2026 Family Privacy Perceptions of Robots in the Home: A Multi-Site Convergence across USA, Spain, and Ukraine
abstract
As we introduce social robots into family homes, there is a heightened need to understand both parental and child perceptions of privacy, taking into account family perspectives. Haru4Kids (H4K) is an app-based robot simulator designed to cohabitate with children in their home. In a multi-site investigation, H4K cohabitated with 11 American, 11 Spanish, and 10 Ukrainian families. Interviews preceding and following the cohabitation period with at least one parent and all participating children ages 5–12 allowed us to gauge general expectations and experiences during cohabitation and to estimate their comfort with sharing different kinds of information with the platform. In general, children perceived Haru as a technological device that was social, yet non-human. All stakeholders were more comfortable with general information collection as opposed to information sharing with third parties. Lastly, though culture is known to affect expectations and perceptions of privacy and robots, our study shows cohabitating with H4K led to a general convergence across sites in users’ perceptions of Haru and attitudes towards preserving their privacy with robots.
Leigh Levinson, Gonzalo A. García, Levko Ivanchuk, Guillermo Pérez 0001, Gloria Alvarez-Benito, J. Gabriel Amores, Manuel Castro-Malet, Randy Gomez, Selma Sabanovic
ACM Trans. Hum. Robot Interact.9
2025 Research as Care: A Reflection on Incorporating the Ethics of Care in Design Research with People Living with Dementia
Long-Jing Hsu, Janice K. Bays, Manasi Swaminathan, Weslie Khoo, Hiroki Sato 0002, Kyrie Jig Amon, Sathvika Dobbala, Min Min Thant, Alex Foster, Katherine M. Tsui, Philip B. Stafford, David Crandall, Selma Sabanovic
Conference on Designing Interactive Systems13
2025 Haru in the Kitchen: Investigating Family Members' Perceptions Toward a Social Robot Mediator of Food Experiences
Aswati Panicker, Chia-Fang Chung, Selma Sabanovic
Conference on Designing Interactive Systems3
2025 Towards a Roboticist's Practical Guide to Working with Children
abstract
Working with children in human-robot interaction (HRI) research presents novel challenges that require thoughtful preparation and reflection.However, given that HRI is still an emerging field of research, there is limited guidance for early-career HRI researchers focusing on child-robot interaction.We present a practical guide to working with children and robots, tailored for early-career researchers whome we refer to throughout as 'roboticists.'We pose several questions to encourage reflection and consideration for working with children, and other vulnerable populations, when designing and testing robots.The goal of this guide is to provide a resource for early-career roboticists when designing human-robot interaction studies that include children.We structure our guide to reflect on three research phases-pre-study, study, and post-studyand address critical questions regarding ethics, logistics, and inclusive communication.With this WiP report, we hope to receive feedback from the IDC community and establish collaborations to improve the proposed guide.In future work, we plan to collect testimonials from roboticists to diversify and expand the practical insights into a more comprehensive practical guide.
Leigh Levinson, Bengisu Cagiltay, Selma Sabanovic, Bilge Mutlu
IDC3
2025 Designing Playful and Ethical Child-AI Systems
abstract
The increasing presence of Artificial Intelligence (AI) systems geared towards children necessitates those who design and develop these technologies to understand how to address the emerging ethical questions in their development and use while maintaining a playful, child-friendly approach.Even more importantly, it is crucial to understand how we can address various tensions that have emerged among ethical principles.In this half-day workshop, keynote talks, poster presentations and interactive, "playful by design" will promote hands-on and rights-based design experiences for researchers and practitioners to ideate the benefits and challenges of designing playful and ethical child-AI systems.
Leigh Levinson, Elmira Yadollahi, Bengisu Cagiltay, Shyamli Suneesh, Vicky Charisi, Angela Colvert, Kruakae Pothong, Selma Sabanovic
IDC8
2025 Designing with Dynamics: Reflections on Co-design Workshops Between People Living with Dementia and Their Care Partners
Long-Jing Hsu, Alex Foster, Selma Sabanovic, Chia-Fang Chung
CHI3
2025 Bittersweet Snapshots of Life: Designing to Address Complex Emotions in a Reminiscence Interaction between Older Adults and a Robot
Long-Jing Hsu, Manasi Swaminathan, Weslie Khoo, Kyrie Jig Amon, Hiroki Sato 0002, Sathvika Dobbala, Katherine M. Tsui, David Crandall, Selma Sabanovic
CHI9
2025 "A Robot's Life is Over When People Give Up": Socio-Technical Infrastructure for Sustaining Consumer Robots
abstract
Sustenance of long-term human-robot relationships relies on a socio-technical infrastructure of care and repair, in which both technical maintenance and social practices work together to sustain robots as relational objects. This infrastructure is, however, often invisible and taken for granted while working well. It is in moments of breakdown and repair that the necessary infrastructure that supports long-term human-robot relationships is exposed. In this paper, we present ethnographic fieldwork and stakeholder interviews with eight users and eight producers and repair staff of successful consumer robots in Japan, that highlight the often-overlooked moments of technological breakdown and the meaning-making that surrounds them. This allows us to shed light on the evolving components of the socio-technical infrastructure-such as technical processes, cultural norms, shared language, emotional labor, and interpersonal relationships-that are essential for sustaining the functionality and relevance of robots in people's everyday lives.
Waki Kamino, Selma Sabanovic, Malte F. Jung
HRI2
2025 Million Eyes on the "Robot Umps": The Case for Studying Sports in HRI Through Baseball
abstract
In this position paper, we argue that baseball-and sports more broadly-provide a unique and under-explored opportunity for researchers to study human-robot interaction (HRI) in real-world settings. Using the rise of robot umpires in baseball as a primary example, we examine emerging themes such as power dynamics among players and umpires, labor implications, and technical challenges. We emphasize the affordances and benefits of studying sports within HRI, including the integration of interdisciplinary perspectives, the large-scale deployment of robots, and the examination of their role in deeply rooted cultural practices.
Waki Kamino, Andrea W. Wen-Yi, Dhruv Agarwal 0001, Sil Hamilton, Eun Jeong Kang, Keigo Kusumegi, Pegah Moradi, Daniel Mwesigwa, Yan Tao, I-Ting Tsai, Ethan Yang, Shengqi Zhu 0002, Shu-Jung Han, Chi-Jung Lee, Michael J. Sack, Tianhong Catherine Yu, Weslie Khoo, Andy Elliot Ricci, Yoyo Tsung-Yu Hou, Selma Sabanovic, David Crandall, Karen Levy, Malte F. Jung
HRI22
2025 Privacy-Sensitive Robotics: Perceptions, Measures, and Metrics
abstract
As robots enter our homes and workplaces, they will have more direct access to us, our information, and our daily lives. The more a robot knows about us, the more helpful it can be, at least in theory. Where is the balance between privacy and utility? How might the robot communicate the tradeoffs between privacy and utility in a nuanced but clear manner? How can we ensure that the perception of privacy-protection offered by the robot accurately reflects how much the robot is actually preserving our privacy? How can a robot learn information that it can use to better help us without being intrusive? What behaviors should the robot have (or not have) to not only ensure that it protects our privacy, but also is perceived as protecting our privacy? How can we measure the effectiveness of these behaviors, so that we can track our progress towards more useful, less invasive robot companions? This workshop is the the third in a series at HRI, and will bring together researchers from a wide variety of intellectual communities to look at these questions, identify promising research directions, and set an agenda for how to start making progress. We are particularly interested in expanding the core of researchers interested in privacy-related issues in HRI, and in those “privacy-curious” researchers who want to find out more about he intersection of privacy and their own research.
Janet Kim, Leigh Levinson, Jonathan Ota, Selma Sabanovic, William D. Smart
HRI4
2025 Effective Engineering, Stakeholder Involvement, and Regulatory Plurality Within Privacy-Aware Robotics
abstract
Privacy is an important yet understudied focus in consideration of successful human-robot interaction (HRI). In this paper, we present a thematic analysis of the topics discussed in the Privacy-Aware Robotics Workshop held at the HRI Conference in 2024. The analysis points across the perspectives of User, Engineering, and Society with particular identification of open “interdisciplinary zones” at the intersections of these perspectives. Based on that, we formulate and present three main themes for future research directions: the tension between robot functioning and effectiveness of privacy implementations in real-world contexts; the need to involve and empower target user communities in the co-design of privacy-aware robots, and the consideration of regulatory frameworks that extend across jurisdictions in robot design.
Leigh Levinson, Manuel Dietrich, Alan Sarkisian, Selma Sabanovic, William D. Smart
HRI4
2025 Welcome to aibo's Hometown! Framing Social Robots as Cultural Resources for the Local Economy
abstract
This paper explores the case study of Kota, Japan, as "aibo’s furusato" – or hometown – through ethnographic observation and interviews to investigate how diverse stake-holders, including local government, industry partners, and robot owners, collaboratively construct the town’s identity as a tourist destination centered around the social robot aibo. Our findings reveal how new robotic technologies are incorporated into existing frameworks of local economic production while also creating new opportunities for social interaction, as well as cultural and economic production that connect robotics to traditional products and practices. This work more broadly supports an understanding of robot design that goes beyond the artifacts itself to encompass a broader socio-technical infrastructure that supports robots’ meaningful use.
Waki Kamino, Malte F. Jung, Selma Sabanovic
RO-MAN3
2025 Building Friendships Across Borders: The Role of Social Robot Haru in Children Group Communication and Connection Development
abstract
Forming friendship with peers from diverse backgrounds is key to children’s social emotional development. In this study, we explored the use of social robot, Haru, as mediator for remote communication in children group, to support connection and friendship building. We invited children from different countries aged from 10 to 15 to participate in two interaction sessions with peers from other countries, after which we conducted interviews with children from three countries, focusing on their experiences, and perceptions of the robot’s roles in the process. The findings indicated that social robot Haru effectively served as an icebreaker and entertainer; However, improvements are needed in conversation flow, transitions between different roles, and supporting children’s autonomy in guiding the conversation and the depth of their communication.
Zhennan Yi, Leigh Levinson, Diego Delgado-Chaves, Jose M. Perez-Moleron, Nabil Bougria, Antonia Krummheuer, Matthias Rehm, Anders Kalsgaard Møller, Katrine Kielsholm Ramsgaard, Selma Auala, Heike Winschiers-Theophilus, Edward Nepolo, David Calero, Devis Dal Moro, Daniel Serrano, Magí Dalmau-Moreno, Randy Gomez, Luis Merino, Selma Sabanovic
RO-MAN19
2025 Game of Life With Your Companion Robot: Exploring the Sustainable Future for Long-Term Human-Robot Interaction
abstract
Reducing electronic waste is one of the key topics in Sustainable Interaction Design. However, research regarding the sustainable future of the long-term use of robots is limited. Our study employs a game-based workshop to investigate the factors influencing potential users’ sustainability choices in long-term human-robot interactions. We developed a board game called “Game of Life with Your Companion Robot” to help participants situate themselves in the context of cohabiting with companion robots of their choice. Through five workshops with seventeen participants, we explore (a) the factors mentioned by participants that influence their sustainability choices in long-term human-robot interactions, and (b) the connections between how participants frame their companion robots and their sustainability choices. We use four sustainable criteria to evaluate participants’ choices. Our findings show that different framings of robots can result in different sustainable outcomes.
Zaiqiao Ye, Zitao Zhang, Xinyao Ma, Eli Blevis, Selma Sabanovic
Int. J. Hum. Comput. Interact.5
2025 Haru in the Care Network: Stakeholder Perspectives on Privacy with Social Robots in Pediatrics
abstract
Social robots are beginning to be utilized as part of the collective networks supporting pediatric treatment, however there are few studies on children's perceptions of these agents in hospitals from a privacy and safety perspective. Through a mixed-method and value-sensitive design approach, we introduced hypothetical vignettes and engaged in discussion with 15 youth who are either receiving cancer treatments or are in remission (ages 6-25), 11 of their parents, and 5 out of 8 of their clinical staff to learn how stakeholders in pediatric oncology discuss privacy concerns regarding child-robot interactions. Our thematic analysis imparts how stakeholders perceive robots as social, non-authoritative extensions of the hospital's care network. From this privacy-sensitive perspective, this study revealed that for maximizing a robot's social utility within care systems while critically engaging with the comfort and privacy preferences of stakeholders, robots should take on the role of 1) mediators of social interaction among various stakeholders, 2) companions for children and 3) informational tools for clinicians when consent is given by the family. We emphasize how assistive technologies in pediatrics should continue to be co-designed within communities for identifying appropriate roles and returning agency to stakeholders as they navigate the blurry boundaries of privacy in healthcare.
Leigh Levinson, Gloria Alvarez-Benito, J. Gabriel Amores, Deborah Szapiro, Randy Gomez, Selma Sabanovic
Proc. ACM Hum. Comput. Interact.6
2025 Do You Want Me?: Exploring Differences in Consumer Home Robot Preferences, Perceptions, and Purchase Intent
abstract
While research in human–robot interaction is beginning to focus on the acceptance of domestic robots, there is little research on the potential adoption of these agents. Technology adoption is a complex phenomenon requiring not only positive perceptions of technology but also its value, along with product desire strong enough to lead to desired adoption behavior. Adoption of innovations also occurs in phases, from early adopters to mainstream consumers, then laggards. While the characteristics of technology early adopters generally have been researched extensively, there is no previous work that seeks to validate some of these variables for domestic robots specifically, and which draws from HRI research to further amend them. In this work, we determine how various consumer and robot characteristics affect assessments of home robot liking, privacy concerns, and purchasing intent. We find that five main consumer characteristics are associated with robot early adopters, and that surprisingly, income has a negative correlation with purchasing intent, specifically for the companion robot. We further compare predicted product liking to purchasing intent, showing that although robot acceptance is reasonably high for those in the mainstream, purchasing intent is low. For all market segments, perceptions of high price accounted for about 20–30% of the variance in intended purchasing, intended use 6–7%, belief in performance as advertised 4–6%, and design 3%. While privacy concerns were not influential to purchasing intentions held by early adopters, they were to mainstream users and laggards.
Natasha Randall, Selma Sabanovic
ACM Trans. Hum. Robot Interact.2
2024 Expert Insights on Robots for Safeguarding Children: How (not) and Why (not)?
abstract
To investigate a robot’s role in children’s welfare and safety, we conducted interviews with 8 Subject Matter Experts and Professionals (SMEs) across the disciplines of robotics, child technology, psychology, and psychiatry disciplines. Through qualitative analysis, we synthesize the challenges of safeguarding, compounding limitations, and potential solutions for involving robots in safeguarding as broadly defined in SME interviews. While they agree robots should not be responsible for making judgement calls, the experts also identified the ways robots can be a valuable addition to the safeguarding team. However, more conversations spanning disciplines need to occur to inform policy and legal frameworks that will better establish a robot’s role in intimate spaces in children’s lives. While this line of inquiry is specific to robots in safeguarding, many of the themes reflect the nuances of finding appropriate places for child-robot interactions in the context of children’s welfare.
Leigh Levinson, Nida Itrat Abbasi, Selma Sabanovic, Hatice Gunes
IDC3
2024 "An Emotional Support Animal, Without the Animal": Design Guidelines for a Social Robot to Address Symptoms of Depression
abstract
Socially assistive robots can be used as therapeutic technologies to address depression symptoms. Through three sets of workshops with individuals living with depression and clinicians, we developed design guidelines for a personalized therapeutic robot for adults living with depression. Building on the design of Therabot, workshop participants discussed various aspects of the robot's design, sensors, behaviors, and a robot connected mobile phone app. Similarities among participants and workshops included a preference for a soft textured exterior and natural colors and sounds. There were also differences - clinicians wanted the robot to be able to call for aid, while participants with depression differed in their degree of comfort in sharing data collected by the robot with clinicians.
Sawyer Collins, Kenna Baugus, Zachary Henkel, Casey C. Bennett, Cedomir Stanojevic, Jennifer A. Piatt, Cindy L. Bethel, Selma Sabanovic
HRI8
2024 "Give it Time: " Longitudinal Panels Scaffold Older Adults' Learning and Robot Co-Design
abstract
Participatory robot design projects with older adults often use multiple sessions to encourage design feedback and active participation from users. Prior projects have, however, not analyzed the learning outcomes for older adults across co-design sessions and how they support constructive design feedback and meaningful participation. To bridge this gap, we examined the learning outcomes within a "longitudinal panel." This panel comprised seven co-design sessions with 11 older adults of varying cognitive abilities over six months, aimed at designing a robot to guide a photograph-based conversational activity. Using Nelson and Stolterman's framework of the hierarchy of design-learning, we demonstrate how older adult panelists achieved multiple design-learning outcomes- capacity, confidence, capability, competence, courage, and connection- which allowed them to provide actionable design suggestions. We provide guidelines for conducting longitudinal panels that can enhance user design-learning and participation in robot design.
Long-Jing Hsu, Philip B. Stafford, Weslie Khoo, Manasi Swaminathan, Kyrie Jig Amon, Hiroki Sato 0002, Katherine M. Tsui, David Crandall, Selma Sabanovic
HRI9
2024 Constructing a Social Life with Robots: Shifting Away From Design Patterns Towards Interaction Ritual Chains
abstract
Robot designers commonly conceptualize robot sociality as a collection of features and capabilities. In contrast, sociologists define sociality as continuously constructed through interpersonal interactions. Based on the latter perspective, we trace how robots are incorporated into emerging social interaction ritual chains by robot companies and their staff and by robot owners across diverse contexts: homes, cafes, robot stores, user-organized meetups, and company events for robot users. Our empirical findings from ethnographic field work in Japan relating to three robots -- aibo, RoboHon, and LOVOT -- show how companies create positive interactions between people and robots by incorporating familiar design patterns into robots, modeling successful interactions in person and online, and bringing owners together in events that establish common values of acceptance of social robots as artifacts to live with and nurture. Owners, for their part, develop interaction rituals that include robots in their daily activities, make interpersonal connections, and experience emotionally resonant interactions, around robots in public meetups and events. Through these emerging interaction ritual chains, companies and owners construct the notion of robots as social agents to live with as a meaningful component of their emotional experiences and broader social relationships. Our work suggests that social robot design should consider this broader framing of sociality and create affordances for establishing interaction ritual chains more explicitly.
Waki Kamino, Malte F. Jung, Selma Sabanovic
HRI3
2024 Snitches Get Unplugged: Adolescents' Privacy Concerns about Robots in the Home are Relationally Situated
abstract
Though teens are a population with growing agency and use of smart technologies, their concerns surrounding privacy with AI and robots are under-represented in research. Using focus group discussions and a mixed methods analysis, we present teens' comfort levels with robotic information collection and sharing during three hypothetical scenarios involving a child interacting with the Haru social robot in the home. We find participant concerns align with an access-based definition of privacy which prioritizes being in control of their information and of when the robot behaves autonomously. Responses also indicate that teens conceptualize Haru not just as an intelligent device, but also as a social entity. Their shifts in comfort and discussions reflect an engagement in social relationship management with robots in the home in cases where the robot mediates a user's responsibilities and relationships with others.
Leigh Levinson, Christena Nippert-Eng, Randy Gomez, Selma Sabanovic
HRI4
2024 Ties That Bind: Group Effects in Human-Robot Team Interaction in Japan and the United States
abstract
Past research with participants in the United States showed that, in competitive group tasks, they have more positive attitudes and behaviors toward robots on their team over humans in another team. Here we present a study in which two Japanese students and two robots, placed in a randomly assigned group, compete with another student-and-robot team in a digital game. We explored participants’ moral behavior towards the robots, measured through their assignment of loud noise blasts to human and robot participants, and their perceptions of and attitudes towards the robots. We then compared this data to that which was collected within the United States. Results indicated that participants in Japan favored their ingroup humans and robots over outgroup agents and differentiated ingroup members more than outgroup members, as within the US. Japanese participants also anthropomorphized robots more than US participants and treated them more positively than US participants.
Sawyer Collins, Marlena R. Fraune, Kyrie Jig Amon, Eliot R. Smith, Selma Sabanovic
RO-MAN5
2024 The Ins and Outs of Socially Assistive Robots: Sensors and Behaviors of a Therapeutic Robot for Depression Management
abstract
Using socially assistive robots (SARs) as specialized companions for those living with depression to manage symptoms provides a unique opportunity for exploration of robotic systems as comfort objects. Moreover, the robotic components allow for specialized behavioral responses to particular stimuli, as preferred by the user. We have conducted semi-structured interviews with 10 participants about the zoomorphic robot’s Therabot™ desired behaviors and focus groups with five additional participants regarding the preferred sensors within the Therabot™ system. In this paper, using the data from interviews and focus groups, we explore SAR input and output for depression management. While participants overall expected the robot to respond in much similar ways as a well-trained service animal, they expressed interest in the robot understanding unique information about the environment and the user, such as when the user might need interaction.
Sawyer Collins, Zachary Henkel, Kenna Baugus, Casey C. Bennett, Cedomir Stanojevic, Jennifer A. Piatt, Cindy L. Bethel, Selma Sabanovic
RO-MAN8
2024 Let's Talk About You: Development and Evaluation of an Autonomous Robot to Support Ikigai Reflection in Older Adults
abstract
The sources of a person’s ikigai—their sense of meaning and purpose in life—often change as they age. Reflecting on past and new sources of ikigai may help people renew their sense of meaning as their life circumstances shift. Building on insights from an initial Wizard-of-Oz robot prototype [1], we describe the design of an autonomous robot that uses a semi-structured conversation format to help older adults reflect on what gives their life meaning and purpose. The robot uses both pre-determined (scripted) and Large Language Model (LLM) generated questions to personalize conversations with older adults around themes of social interaction, planning, accomplishments, goal setting, and the recent past. We evaluated the autonomous robot with 19 older adult participants in a lab setting and at two eldercare facilities. Analysis of the older adults’ conversations with the robot and their responses to an evaluative survey allowed us to identify several design considerations for an autonomous robot that can support ikigai reflection. Interweaving simple yet detailed predetermined questions with LLM-generated follow-up questions yielded enjoyable, in-depth conversations with older adults. We also recognized the need for the robot to be able to offer relevant suggestions when participants cannot recall events and people they find meaningful. These findings aim to further refine the design of an interactive robot that can support users in their exploration of life’s purpose.
Long-Jing Hsu, Weslie Khoo, Manasi Swaminathan, Kyrie Jig Amon, Rasika Muralidharan, Hiroki Sato 0002, Min Min Thant, Anna S. Kim, Katherine M. Tsui, David Crandall, Selma Sabanovic
RO-MAN11
2024 The Lifecycle of Social Robots: Obsolescence and Values in Repair
abstract
This paper introduces a "lifecycle perspective" on social robot design and human-robot interaction, and explores the practices of maintenance, repair, and letting go of social robots. Drawing on interviews with robot owners and representatives of robot development and repair companies, we argue that these previously disregarded aspects of everyday use provide a context for negotiating the social value and meaning of interactions with robots. We discuss owner concerns about robot obsolescence, as well as company support for long term human-robot interaction through repair, reuse, and giving owners closure in letting go of robots they can no longer use. Our work expands the purview of HRI study and design beyond the common focus on initial design and adoption and to perceptions and practices that can foster more enduring relationships with social robots, support sustainability in robot design, and address owners’ emotional attachment to robots.
Waki Kamino, Selma Sabanovic, Malte F. Jung
RO-MAN2
2024 Bow Ties & Colorful Eyes: Centering Youth Designs of Social Robots
abstract
Under UNICEF’s Policy guidance on AI for children, child-centered AI should always ‘ensure inclusion of and for children.’ To spotlight youth visions for robots, we led co-design workshops with children between 5-14 years old. Youth designs were expressive, customized, relatable, and approachable. Based on 54 drawings and descriptions of the social robot Haru, we suggest that future child-centered robots should 1) be expressive across verbal and non-verbal channels of communication, 2) allow for customization to give children more agency when interacting with the robot, 3) adapt to children’s style and hobbies to make them feel seen, and 4) aesthetically keep proportions of robot faces consistent and cartoon-like to make robots more approachable.
Leigh Levinson, Randy Gomez, Selma Sabanovic
RO-MAN3
2024 Articulation work for supporting the values of students attending class via telepresence robots
Houda Elmimouni, Jennifer Ann Rode, Selma Sabanovic
Int. J. Hum. Comput. Stud.3
2023 Co-designing Social Robots with People Living with Dementia: Fostering Identity, Connectedness, Security, and Autonomy
abstract
Conventional co-design methods, such as storyboarding and focus groups, are not always appropriate for people living with dementia (PLwD). In pilot robot co-design workshops in a local memory care facility, we noticed PLwD struggled to understand, express themselves, fully participate, and benefit from the experience. After reflecting on challenges with the facility’s director of program development and education, we redesigned the workshops prioritizing elements of the Eden Alternative’s well-being for PLwD: identity, connectedness, security, and autonomy. We delivered these new workshops over five weeks with 12 PLwD participants. Analysis of resulting video recordings and transcripts shows the new activities allowed participants to see themselves as having knowledge relevant to social robot design; to relate to each other, the robot, and the researchers; to feel comfortable; and to actively contribute to and offer valuable insights for robot design. Participants reported feeling meaning, growth, and joy during the workshops.
Long-Jing Hsu, Janice K. Bays, Katherine M. Tsui, Selma Sabanovic
Conference on Designing Interactive Systems4
2023 Dressing up AIBO: An Exploration of User-generated Content of Robot Clothing on Twitter
abstract
Robots do not need clothes to keep them warm or for other purely functional reasons; nevertheless, certain people choose to clothe their home robots. This paper contributes to an emerging interest in understanding the practice of clothing robots by analyzing how people dress up the pet-type companion robot aibo. We study a collection of 2320 Twitter posts with photos containing aibo robots in clothes provided for them by their owners. The collection of tweets spans four years (2018-2021). We report our findings regarding aibo clothing types, fashion trends, clothing choices in different use scenarios, and clothing sources. Our study provides insights into the benefits of clothing robots for users, different modes of robot fashion, and how users’ current practices of robot clothing design and use can inform the future design of companion robots.
Zaiqiao Ye, Wei-Chu Chen, Selma Sabanovic
HAI3
2023 Coffee, Tea, Robots?: The Performative Staging of Service Robots in 'Robot Cafes' in Japan
abstract
We present an ethnographic observational study of six robot cafes in Japan to understand how service robots are performatively staged and presented to the public. We particularly attend to the diverse ways in which the physical setting and ambience of the cafes, the verbal characterization of and staff behaviors toward robots, explicit and implicit instructions on appropriate interactions with robots, and handling of robot malfunctions constitute robots as socially acceptable and useful in daily life. Such scaffolding enables robots to provide material and affective services to cafe visitors, and visitors to explore various interaction possibilities with robots. Our work contributes to the critical study of the ongoing construction of "robot cultures" in Japan, and calls attention to public interactions with robots and the importance of contextual staging beyond individual robot features in human-robot interaction design.
Waki Kamino, Selma Sabanovic
HRI2
2023 A Picture Might Be Worth a Thousand Words, But It's Not Always Enough to Evaluate Robots
abstract
Evaluation of robots commonly occurs using various stimuli, including photos, videos, and live interaction. However, a better understanding of how and why chosen stimuli affect perceptions, and how evaluations using lower fidelity media (e.g. photos) compare to evaluations using higher context stimuli (e.g., videos), is needed. Through a survey of 599 M-Turk participants, we compare robot evaluations based on exposure to three types of media - photos, GIFs, and promotional videos. We analyze nine perception and behavioral intention measures of three home robots with varying levels of anthropomorphism (Olly, Jibo, and Liku): overall liking, liking of appearance, liking of intended use, eeriness, human-likeness, performance expectations, privacy concerns, information seeking intention, and purchase intention. We find that photos consistently differ from ratings of videos for all measures, except for liking of robots' intended use. Use of GIFs led to measurements in line with videos for seven of the nine measurements, due to the importance of movement in perceptual assessments and character judgments (e.g., friendly, creepy). Except for the most human-like robot, neither photos nor GIFs captured human-likeness to a similar degree as videos, due to the importance of speech in assessments. Though GIFs captured informational and overall privacy concerns well, they did not adequately capture physical privacy concerns.
Natasha Randall, Selma Sabanovic
HRI2
2023 Enabling Robotic Pets to Autonomously Adapt Their Own Behaviors to Enhance Therapeutic Effects: A Data-Driven Approach
abstract
Socially-assistive robots (SARs) hold significant potential to transform the management of chronic healthcare conditions (e.g. diabetes, Alzheimer’s, dementia) outside the clinic walls. However doing so entails embedding such autonomous robots into people’s daily lives and home living environments, which are deeply shaped by the cultural and geographic locations within which they are situated. That begs the question whether we can design autonomous interactive behaviors between SARs and humans based on universal machine learning (ML) and deep learning (DL) models of robotic sensor data that would work across such diverse environments? To investigate this, we conducted a long-term user study with 26 participants across two diverse locations (United States and South Korea) with SARs deployed in each user’s home for several weeks. We collected robotic sensor data every second of every day, combined with sophisticated ecological momentary assessment (EMA) sampling techniques, to generate a large-scale dataset of over 270 million data points representing 173 hours of randomly-sampled naturalistic interaction data between the human and SAR. Models built on that data were capable of achieving nearly 84% accuracy for detecting specific interaction modalities (AUC 0.885) when trained/tested on the same location, though suffered significant performance drops when applied to a different location. Further analysis and participant interviews showed that was likely due to differences in home living environments in the US and Korea. The results suggest that our ability to create adaptable behaviors for robotic pets may be dependent on the human-robot interaction (HRI) data available for modeling.
Casey C. Bennett, Selma Sabanovic, Cedomir Stanojevic, Zachary Henkel, Jinjae Lee, Kenna Baugus, Jennifer A. Piatt, Janghoon Yu, Jiyeong Oh, Sawyer Collins, Cindy L. Bethel
RO-MAN2
2023 Living with Haru4Kids: Study on children's activity and engagement in a family-robot cohabitation scenario
abstract
Haru4Kids (H4K) is a system that emulates the physical, social, family-oriented robot Haru, designed with the goal to cohabitate with children in their home for extended periods of time. Seven families kept H4K for two net weeks in their homes. Throughout this period of cohabitation, we collected user logs comprised of the children users ’ head angles, the rotation angles of the platform, and the actions taken by H4K as well as captured images which were afterwards hand-annotated to estimate user engagement. We report the trends of these external metrics that we collected during every session of interaction. We also developed an annotation tool and report the Engagement Level Metric we chose to estimate child engagement throughout interactions “in-the-wild.” Overall, our platform offers a feasible system that can engage with children while also allowing us to monitor their engagement and behaviour throughout each interaction.
Gonzalo A. García, Guillermo Pérez 0001, Leigh Levinson, J. Gabriel Amores, Gloria Alvarez-Benito, Manuel Castro-Malet, Mario Castaño-Ocaña, Marta J. López-González de Quevedo, Ricardo Durán-Viñuelas, Randy Gomez, Selma Sabanovic
RO-MAN11
2023 Finding its Voice: The Influence of Robot Voice on Fit, Social Attributes, and Willingness to Use Among Older Adults in the U.S. and Japan
abstract
Robots may be able to significantly assist older adults through making activity recommendations. Prior research suggests that gender and age of a robot’s voice may affect how people respond to such recommendations, but few studies have explored how a robot’s voice is perceived by older adults, and whether their perceptions differ across cultures. We conducted a survey study with older adult participants (aged 65+) in the U.S. (N=225) and Japan (N=466), asking them to evaluate a humanoid robot speaking with three different voices (male, female, child). After seeing a video of a robot making recommendations, participants rated the fit of the voice to the robot, its sociality (via the Robotic Social Attributes Scale - RoSAS), and their willingness to use the robot in various contexts. We discovered that robot’s social attributes and participants’ culture impacted willingness to use the robot in both countries. Having positive social attributes and lower negative attributes increases willingness to use the robot. The U.S. older adults preferred the adult robot voices, had more positive social attributes, less negative social attributes, and were more likely to accept lifestyle recommendations than Japanese older adults. This study contributes to our understanding of older adults’ perceptions of robot voice and provides design implications for robots that make recommendations to older adults.
Long-Jing Hsu, Weslie Khoo, Natasha Randall, Waki Kamino, Swapna Joshi, Hiroki Sato 0002, David Crandall, Katherine M. Tsui, Selma Sabanovic
RO-MAN9
2023 We All Make Mistakes: Terminal, Non-critical, Recoverable, and Favorable Interaction Failures Between People and a Social Robot
abstract
In this paper, we present an in-depth illustration of interaction failures relatively unexplored in the field of human-robot interaction (HRI). Our qualitative analysis of interactions between a social robot and 12 participants sheds light on different types of erroneous interactions initiated by human and robot actors and their outcomes. Our findings show that a small portion of observed failures had fatal impacts on interactions. In most cases, they had little negative effects on interactions or even led to favorable outcomes, causing laughter and giggling from participants, for example. Overall, our study calls for further examination of the roles of failures and contextual factors that influence the consequences of failures in HRI.
Waki Kamino, Natasha Randall, Tanya Saga, Long-Jing Hsu, Katherine M. Tsui, Selma Sabanovic, Shinichi Nagata
RO-MAN6
2023 Child-Robot Conversation in the Wild Wild Home: A Language Processing User Study
abstract
Child-robot interaction (CRI) has been mostly studied in labs and classroom settings. In this work, we share a CRI language processing study carried out in children’s homes. Any automated system deployed “in-the-wild” faces practical problems, but when the target users are children, these problems get even more sensitive and challenging. In this work we analyse how each language processing layer performs with children at home with no researcher present. We carried out an experiment with 7 families [N=14 children, 6-13 years old] cohabiting with a simulated robot for 2 weeks in their own homes. Our goal in this study is to evaluate the performance of voice recognition, language understanding and dialogue management when children interact with a robot at home. Our results indicate that dialogue management capabilities are becoming the key element in the language processing pipeline; they also denote that the dialogue engine should include mixed-initiative capabilities and show the relative usage of different common built-in intents.
Guillermo Pérez 0001, Gonzalo A. García, Manuel Castro-Malet, Mario Castaño-Ocaña, Marta J. López-González de Quevedo, Ricardo Durán, J. Gabriel Amores, Gloria Alvarez-Benito, Leigh Levinson, Selma Sabanovic, Randy Gomez
RO-MAN10
2023 Realizing a Life Well Lived: The Design of a Home Robot to Assist Older Adults with Self-Reflection and Intentional Living
abstract
Previous work suggests that older adults’ meaning and happiness may be increased simply by having them engage in self-reflection exercises. Therefore, we design four modules to promote self-reflection, delivered by the QT robot. These modules were created with reference to three different time orientations — past, present, and future — and by incorporating aspects of life satisfaction and the PERMA model of well-being. Results show that about half of our older adult participants experienced subjective changes in meaning, happiness, and desire to make positive life changes during each module, with 11 of 15 participants experiencing changes to one of these measures after engaging in all four interactions. We make several suggestions for updating these modules for autonomous and longer-term deployment using the robot.
Natasha Randall, Tanya Saga, Waki Kamino, Katherine M. Tsui, Selma Sabanovic, Shinichi Nagata
RO-MAN5
2022 Hey Haru, Let's Be Friends! Using the Tiers of Friendship to Build Rapport through Small Talk with the Tabletop Robot Haru
abstract
Conversation can play an essential role in forging bonds between humans and social robots, but participants need to feel like they are being listened to, remembered, and cared about in order to effectively build rapport. In this paper, we propose a novel strategy for conducting small talk with a social robot. Our approach is known as the Tiers of Friendship. It is centered around three core design elements: 1) Persuasive content and character is provided through topic modules created by professional creative writers to ensure engaging conversational content and a compelling personality for the social robot. 2) Conversational memory is achieved by allowing topic modules to specify required information that can be learned through conversation or recalled from previous interactions and organizing topic modules into a hierarchy that enforces information requirements between topics. 3) Dynamicity in conversation is promoted through topic navigation that supports fluid transitions to topics of human interest and employs elements of random ordering to create fresh conversation experiences. In this paper, we show how the Tiers of Friendship can be used to generate conversation content for a social robot that encourages the development of rapport. We describe a working implementation of a small talk system for a social robot based on the Tiers of Friendship that combines off-the-shelf ASR and NLU components and custom robot behavior components implemented via behavior trees on ROS. Finally, in order to evaluate our approach's effectiveness, we conduct an elicitation survey that evaluates conversations in terms of perceived engagement, personality traits, and rapport expectation and discuss the implications for social robotics.
Eric Nichols, Sarah Rose Siskind, Levko Ivanchuk, Guillermo Pérez 0001, Waki Kamino, Selma Sabanovic, Randy Gomez
IROS6
2022 ACM Transactions on Human-Robot Interaction: The State of the Journal
abstract
As the first robotics journal offered by ACM, the Transactions on Human-Robot Interaction (ACM THRI) made considerable progress to advance the field of HRI since its inaugural offering in 2018.ACM THRI remains the flagship journal for leading research in the burgeoning area of Human-Robot Interaction (HRI).The interdisciplinary field of HRI aims to understand and advance our knowledge of human-robot systems towards advancing usable and useful robotic technology.ACM THRI features leading thought into how people interact with robots and robotic technologies, how to improve these interactions and make new kinds of interaction possible, and the effects of such interactions on organizations or society.HRI brings together researchers and practitioners with diverse expertise, including but not limited to robotics, artificial intelligence, human factors, psychology, design, haptics, and mechatronics.We officially began our term in January 2017 as Editors-in-Chief of the Journal of Human-Robot Interaction, the predecessor to ACM THRI.We were fortunate to build on the foundation of JHRI provided by its founding editors, Profs.Sara Kiesler and Michael Goodrich.Our leadership of ACM THRI would not be possible without the tremendous effort and dedication of our core editorial team, including our amazing Managing Editor (Prof.James Young), Journal Administrator (Gita Delsing) and Information Director (Dr.Daniel Rea) as well as their predecessors (Prof.David Feil-Seifer, Ms. Jeanie Lyubelsky, and Prof. Monica Anderson).Although it has presented unique challenges and new opportunities, we remain grateful for the opportunity to lead the transition of JHRI to ACM ownership and further extend ACM THRI.Our focus as Editors-in-Chief has been to establish commitment to excellence, sustainability, and inclusion in the scholarship and review processes of ACM THRI, which has proved successful to this point.As stated in our introductory editorial, our primary aims for the Journal have been three fold: (1) increase the sustainability and impact of HRI as a field (both quantitatively and qualitatively), (2) enable timely and productive review feedback, and (3) cultivate new and leading ideas in both robotics and the human-centered sciences.ACM THRI has remained true to this scope and made significant strides towards realizing these values across the practices of the journal.For our review processes, our goal has been to provide useful feedback to our authors that enables them to improve their work and make contributions worthy of intellectual leadership.We aim to return substantive review feedback to authors within 105 days of the date of submission, and 70 days for revisions.The total average for ACM THRI returning decisions for submitted manuscripts is currently 142 days in 2021, 122 days from April 2020 to December 2020, 114.7 days in the first three months of 2020 (submissions just prior to the COVID-19 pandemic), and 99 days in 2019.Our review process includes significant deliberation and discussion among our editorial board of every manuscript and decision, which occurs during our weekly editorial teleconferences.A summary of our editorial consensus with a reasoned and supported argument is provided with
Selma Sabanovic, Odest Chadwicke Jenkins
ACM Trans. Hum. Robot Interact.1
2021 When No One is Watching: Ecological Momentary Assessment to Understand Situated Social Robot Use in Healthcare
abstract
Socially-Assistive Robots (SARs) hold great potential to revolutionize the way we manage chronic illness outside clinical settings, but a current limitation to their broad adoption for this purpose is the lack of "ground truth" around interactions between robots and humans in in-home settings.Such ground truth is a necessity for using robotic sensor data for machine learning models of patient activity patterns or to create AI to customize robotic interactive behavior autonomously.Traditional subjective recall-based data collection methods lack the fine-grained temporal detail to support such AI development, as well as suffering from "recall bias" effects.One potential solution to this challenge is to adapt novel forms of interaction assessment, such as ecological momentary assessment (EMA), to collect patient interaction data in real-time.Here we describe a pilot study utilizing such an EMA system with SARs.We describe the development of the EMA framework, theoretical design issues, and lessons learned.Preliminary machine learning results indicate 75-80% accuracy for detecting specific interaction modalities.We also discuss the potential utility of EMA for exploring cross-cultural differences with in-the-wild robot use, and as a tool to support participatory design research on robotics in healthcare settings.
Casey C. Bennett, Cedomir Stanojevic, Selma Sabanovic, Jennifer A. Piatt
HAI3
2021 "What Does Your Robot Do?" A Tabletop Role-Playing Game to Support Robot Design
abstract
Social robot co-design requires aiding users as they imagine these novel devices within their everyday lives and enabling designers to understand and address users’ experiences. This paper presents the exploratory development and evaluation of a role-playing game aimed at identifying the desired features and uses of a social robot that can assist people diagnosed with depression. Participants (n = 16) played the game as a character with depression, designed a companion robot for that character, and chose reactions to daily challenges. Though participants initially selected robot capabilities based on their own needs, after the game they identified alternative designs that would better address daily challenges faced by individuals with depression. We discuss aspects of the game that allowed participants to understand how various robot characteristics can address the experience of depression and suggest how role-playing games can support users and designers in identifying beneficial features and uses of emerging robotic technologies.
Sawyer Collins, Selma Sabanovic
RO-MAN2
2020 We're in This Together: Social Robots in Group, Organizational, and Community Interactions
abstract
Naturalistic studies of human-robot interaction (HRI) in various domains of everyday life, including healthcare, domestic assistance, education, transportation, and other services, show that natural exchanges between people and robots often take place in the broader context of small groups, organizations, and communities. While initial research in HRI focused largely on evaluating the one-on-one interactions between humans and robots, the field is increasingly turning its attention to interactions involving multiple people and/or multiple robots. This talk is motivated by this 'group turn' in HRI, by theories and studies of group interaction in the social sciences, and by our own initial observations of group human-robot interaction (HRI) in public spaces. I will discuss how we can go beyond a dyadic understanding of HRI to incorporate group, organizational, and community goals and experiences into robot design.
Selma Sabanovic
HAI1
2020 A Three-Site Reproduction of the Joint Simon Effect with the NAO Robot
abstract
The generalizability of empirical research depends on the reproduction of findings across settings and populations. Consequently, generalizations demand resources beyond that which is typically available to any one laboratory. With collective interest in the joint Simon effect (JSE) -- a phenomenon that suggests people work more effectively with humanlike (as opposed to mechanomorphic) robots -- we pursued a multi-institutional research cooperation between robotics researchers, social scientists, and software engineers. To evaluate the robustness of the JSE in dyadic human-robot interactions, we constructed an experimental infrastructure for exact, lab-independent reproduction of robot behavior. Deployment of our infrastructure across three institutions with distinct research orientations (well-resourced versus resource-constrained) provides initial demonstration of the success of our approach and the degree to which it can alleviate technical barriers to HRI reproducibility. Moreover, with the three deployments situated in culturally distinct contexts (Germany, the U.S. Midwest, and the Mexico-U.S. Border), observation of a JSE at each site provides evidence its generalizability across settings and populations.
Megan K. Strait, Florian Lier, Jasmin Bernotat, Sven Wachsmuth, Friederike Eyssel, Robert L. Goldstone, Selma Sabanovic
HRI7
2020 Robust Real-Time Hand Gestural Recognition for Non-Verbal Communication with Tabletop Robot Haru
abstract
In this paper, we present our work in close-distance non-verbal communication with tabletop robot Haru through hand gestural interaction. We implemented a novel hand gestural understanding system by training a machine-learning architecture for real-time hand gesture recognition with the Leap Motion. The proposed system is activated based on the velocity of a user's palm and index finger movement, and subsequently labels the detected movement segments under an early classification scheme. Our system is able to combine multiple gesture labels for recognition of consecutive gestures without clear movement boundaries. System evaluation is conducted on data simulating real human-robot interaction conditions, taking into account relevant performance variables such as movement style, timing and posture. Our results show robustness in hand gesture classification performance under variant conditions. We furthermore examine system behavior under sequential data input, paving the way towards seamless and natural real-time close-distance hand-gestural communication in the future.
Heike Brock, Selma Sabanovic, Keisuke Nakamura, Randy Gomez
RO-MAN2
2019 I Beg to Differ: Soft Conflicts in Collaborative Design Using Design Fictions
abstract
The adoption of design fiction into design research has recently been expanding within the HCI community. Design fiction workshops have fruitfully facilitated users and researchers discussing and creating future technologies by exposing differing viewpoints. Yet, most scholarship focuses on the ostensibly successful outputs of these workshops. It remains unclear exactly what sort of interaction dynamics are instigated by design fiction in collaborative design. How might design fiction affect what we consider in design, and how is this reflected in the ensuing design? To fill this gap, our study examines design fictions across five workshops where diverse participants created futuristic autobiographies, a method to elicit values, and built diegetic prototypes both individually and collaboratively. We detail their design processes and unpack three kinds of soft conflicts that arose between participants and allowed them to bring up and discuss differing values regarding technology in society. Reflecting on our workshops, we discuss their implications on how one might employ design fiction in collaborative design.
Stephen Tsung-Han Sher, Selma Sabanovic, Norman Makoto Su
Conference on Designing Interactive Systems3
2019 Understanding Life Transitions: A Case Study of Support Needs of Low-Income Mothers
abstract
Life transitions are an integral part of the human experience. However, research shows that lack of support during life transitions can result in adverse health outcomes. To better understand the support needs and structures of low-income women during transition to motherhood, we interviewed 10 women and their 14 supporters during the transition. Our findings suggest that support needs and structures of mothers evolve during transition, and that they also vary by socio-economic contexts. In this paper, we detail our study design and findings. Informed by our findings, we posit that all life-transitions are not the same, and that therefore, the optimal support intervention point varies for different life transitions. Currently there are no tools available to identify optimal support intervention points during life transitions. To this end, we also introduce a preliminary framework - the Strength-Stress-Analysis (SSA) framework - to identify optimal support intervention points during life-transitions.
Annu Prabhakar, Erik Stolterman, Selma Sabanovic
CHI3
2019 Expressivity for Sustained Human-Robot Interaction
abstract
Expressivity - the use of multiple, non-verbal, modalities to convey or augment the communication of internal states and intentions - is a core component of human social interactions. Studying expressivity in contexts of artificial agents has led to explicit considerations of how robots can leverage these abilities in sustained social interactions. Research on this covers aspects such as animation, robot design, mechanics, as well as cognitive science and developmental psychology. This workshop provides a forum for scientists from diverse disciplines to come together and advance the state of the art in developing expressive robots. Participants will discuss points of methodological opportunities and limitations, to develop a shared vision for next steps in expressive social robots.
Vicky Charisi, Selma Sabanovic, Serge Thill, Emilia Gómez, Keisuke Nakamura, Randy Gomez
HRI2
2019 Is Human-Robot Interaction More Competitive Between Groups Than Between Individuals?
abstract
As robots, both individually and in groups, become more prevalent in everyday contexts (e.g., schools, workplaces, educational and caregiving institutions), it is possible that they will be perceived as outgroups, or come into competition for resources with humans. Research indicates that some of the psychological effects of intergroup interaction common in humans translate to human-robot interaction (HRI). In this paper, we examine how intergroup competition, like that among humans, translates to HRI. Specifically, we examined how Number of Humans (1, 3) and Number of Robots (1, 3) affect behavioral competition on dilemma tasks and survey ratings of perceived threat, emotion, and motivation (fear, greed, and outperformance). We also examined the effect of perceived group entitativity (i.e., cohesiveness) on competition motivation. Like in social psychological literature, these results indicate that groups of humans (especially entitative groups) showed more greed-based motivation and competition toward robots than individual humans did. However, we did not find evidence that number of robots had an effect on fear-based motivation or competition against them unless the robot groups were perceived as highly entitative. Our data also show the intriguing finding that participants displayed more fear of and competed slightly more against robots that matched their number. Future research should more deeply examine this novel pattern of results compared to one-on-one HRI and typical group dynamics in social psychology.
Marlena R. Fraune, Steven Sherrin, Selma Sabanovic, Eliot R. Smith
HRI3
2019 Robots for Inter-Generational Interactions: Implications for Nonfamilial Community Settings
abstract
Social robots have been designed to engage with older adults and children separately, but their use for intergenerational (IG) interactions, especially in nonfamilial settings, has not been studied. In addition to the challenge of simultaneously meeting the varied needs and preferences of older adults and children, the dynamic nature of these settings makes the use of robots for IG activities difficult. This paper presents a first exploratory study meant to inform the design and use of social robots for IG activities in nonfamilial settings by analyzing interviews and observations conducted at a co-located preschool and assisted living-dementia care center. Interactions occurring with and around robots were analyzed, particularly focusing on whether they fulfill the community's goals of providing children and older adults with engaging opportunities for IG contact. Findings suggest integrating intermittent pauses and breaks in interactions with the robot and unstructured collaborative robot-assisted activities can meet the needs of both generations, and call for greater community involvement in HRI for IG research.
Swapna Joshi, Selma Sabanovic
HRI2
2019 Editorial: Representation Learning in HRI
Amir Aly, Odest Chadwicke Jenkins, Selma Sabanovic
ACM Trans. Hum. Robot Interact.3
2019 Editorial Introduction to ACM THRI Volume 8, Issue 2
abstract
No abstract available.
Odest Chadwicke Jenkins, Selma Sabanovic
ACM Trans. Hum. Robot Interact.2
2019 Editorial - The HRI Spring: Shaping Our Future From a Foundation of Diverse Scholarship
abstract
No abstract available.
Selma Sabanovic, Odest Chadwicke Jenkins
ACM Trans. Hum. Robot Interact.1
2019 Editorial
abstract
No abstract available.
Selma Sabanovic, Odest Chadwicke Jenkins
ACM Trans. Hum. Robot Interact.1
2018 Projecting Life Onto Robots: The Effects of Cultural Factors and Design Type on Multi-Level Evaluations of Robot Anthropomorphism
abstract
Existing research has shown that people often attribute human-like attributes to robots, which is generally known as the “anthropomorphism” phenomenon. We use the notion of “multi-dimensional anthropomorphism,” to perform a more fine-grained analysis of anthropomorphism in relation to robots in terms of several dimensions (e.g., uniquely and typically human, being alive or not, having emotions or not). Additionally, we expand on existing work, which has mostly focused on organism-based robot designs, by including object-based robot designs in our study of robot anthropomorphism. The results of an online survey study with 775 U.S. (393) and Chinese (382) participants show how people's personal characteristics (e.g., nationality) affect their perceptions of the anthropomorphism of robots, and how such perceptions differ between organism- and object-based robot designs. The effect on people's multi-dimensional anthropomorphism perceptions suggests new design implications for robotic technologies.
Haodan Tan, Dakuo Wang, Selma Sabanovic
RO-MAN3
2018 Robots For Us: Organizational and Community Perspectives on the Collaborative Design of Ubiquitous Robots
abstract
Robots are expected to become ubiquitous in the near future, working with people in various environments, including homes, schools, hospitals, and offices. As physically and socially interactive technologies, robots present new opportunities for embodied interaction and active as well as passive sensing in these contexts. They have also been shown to psychologically impact individuals, affect group and organizational dynamics, and modify our concepts and experiences of work, care, and social relationships. Designing robots for increasingly ubiquitous everyday use requires understanding how robots are perceived, and can be adopted and supported in open-ended, natural social circumstances. This, in turn, calls for design and evaluation methodologies that go beyond the dyadic and small group interactions in laboratories that have largely been the focus of research in human-robot interaction. In this talk, I will present alternative perspectives on the design and evaluation of socially interactive robotic technologies in real-world contexts, focusing on several case studies of socially assistive robots in eldercare. I will first discuss how older adults make sense of robots for use in their homes, in relation to the broader social contexts in which they live, as part of collaborative design activities, and in the course of month-long implementations of robots in their homes. These in-home studies bring up various issues relating to the types of data older adults and the clinicians who work with them would like to collect, related privacy concerns, impacts on other people in the home, and how robot designs can support the relationships older adults hope to have with and through robots. Secondly, I will explore the institutional and community-based use and design of robots in different eldercare facilities, including a nursing home, a retirement community, and an intergenerational daycare. These studies bring out how robots fit into and affect the institutional and group dynamics of interaction, and also allow us to explore how robots might be envisioned as technologies that can support not only individual, but community-level goals. Through these case studies of robots, as emerging ubiquitous interactive technologies, I will bring out themes that can inform the design and study of pervasive systems more broadly, including collaborative design, the use of data collected during social interactions with and around technologies, related ethical concerns, and the need for incorporating the aims of groups, institutions, and communities in the design of intelligent interactive technologies.
Selma Sabanovic
UIST1
2018 Understanding the ACM THRI Review Process
abstract
No abstract available.
Odest Chadwicke Jenkins, Selma Sabanovic
ACM Trans. Hum. Robot Interact.2
2018 ACM Transactions on Human-Robot Interaction: A Welcome from the Editors-in-Chief
abstract
No abstract available.
Selma Sabanovic, Odest Chadwicke Jenkins
ACM Trans. Hum. Robot Interact.1
2017 Collaborative Map Making: A Reflexive Method for Understanding Matters of Concern in Design Research
abstract
HCI researchers investigating the politics of technology design have recently focused on how design practice can tackle "Matters of Concern" - complex social issues perceived and experienced in multiple ways. These researchers suggest design research can generate new networks of human and non-human actors to express and act on these issues. Prior studies, however, tend to restrict their networks within traditional boundaries (e.g. existing organizations, local communities) and categories (e.g. human/nonhuman binary) without examining their significance for participants. We suggest collaborative map making as a reflexive method for understanding current Matters of Concern from the perspectives of diverse actors, not just researchers. As case studies of the method's use, we present two studies of domestic computing technologies in the US and South Korea, which show how collaborative map making allows salient networks to expand beyond the individual actors in the home to local and global power issues outside of boundaries (e.g. physical house) and categories (e.g. private/public space) commonly recognized in HCI. Our methodology provides HCI researchers with a way to understand existing Matters of Concern, so they can position themselves to address and act on these issues.
Hee Rin Lee, Selma Sabanovic, Sonya S. Kwak
CHI2
2017 Threatening Flocks and Mindful Snowflakes: How Group Entitativity Affects Perceptions of Robots
abstract
Robots are expected to become present in society in increasing numbers, yet few studies in human-robot interaction (HRI) go beyond one-to-one interaction to examine how characteristics of robot groups will affect HRI. In particular, people may show more negative or aggressive behavior toward entitative (i.e., cohesive) robot groups, like they do toward entitative human groups, compared to diverse groups. Furthermore, because people in collectivist (e.g., Japan) and individualistic (e.g., US) cultures respond to groups and to cues of entitativity differently, entitative robot groups may affect people differently across such cultures. This study examines how robot Entitativity Condition (Single Robots, Diverse Group, Entitative Group) and Country (USA, Japan) affect emotions toward, mind attributions to, and willingness to interact with robots. Results indicate that Entitative robot groups, compared to Single robots, were viewed more negatively. Entitative robots were also more threatening than Diverse robots. Diverse robot groups, compared to Single robots, were viewed as having more mind, and participants were more willing to interact with them. These findings were similar in the USA and Japan. This indicates that robot group entitativity and diversity is critical to keep in mind when designing robots.
Marlena R. Fraune, Yusaku Nishiwaki, Selma Sabanovic, Eliot R. Smith, Michio Okada
HRI3
2017 Steps Toward Participatory Design of Social Robots: Mutual Learning with Older Adults with Depression
abstract
This paper presents the results of research aimed at developing a methodology for the participatory design of social robots, which are meant to be incorporated into various social contexts (e.g. home, work) and establish social relations with people. In contrast to the dominant technologically driven robot development process, we aim to develop a socially robust and responsible approach to robot design using Participatory Design (PD) methods. The PD process builds on participants' self-identified issues and concerns, and develops robot concepts according to participants' interpretations of the capabilities and potential applications of robotic technologies. We present methodological insights from an ongoing PD project aimed at designing socially assistive robots with older adults diagnosed with depression and their therapists, and identify remaining challenges in this project. We particularly focus on supporting mutual learning between researchers and participants and on promoting active participation of older adults as "designers" (rather than consumers) as foundational aspects of PD. We conclude with reflections regarding how this work can contribute to the further development of social robots and relevant PD methodologies.
Hee Rin Lee, Selma Sabanovic, Wan Ling Chang, Shinichi Nagata, Jennifer A. Piatt, Casey C. Bennett, David Hakken
HRI2
2017 Teammates first: Favoring ingroup robots over outgroup humans
abstract
When it's between a robot on your team and a human member of a competing team, who will you favor? Past research indicates that people favor and behave more morally toward ingroup than outgroup members. Conversely, people typically indicate that they have more moral responsibilities toward humans than nonhumans. This study puts participants into two competing teams, each consisting of two humans and two robots, to examine how people behave toward others depending on Group (ingroup, outgroup) and Agent (human, robot) variables. Measures of behavioral aggression used in previous studies (i.e., noise blasts) and reported liking and anthropomorphism evaluations of humans and robots indicated that participants favored the ingroup over the outgroup, and humans over robots. Group had a greater effect than Agent, so participants preferred ingroup robots to outgroup humans.
Marlena R. Fraune, Selma Sabanovic, Eliot R. Smith
RO-MAN2
2017 A communal perspective on shared robots as social catalysts
abstract
Recent years have seen robust advancements in robotic platforms for multiple users, while HRI research is increasingly examining small group interactions. However, there has been little consideration of appropriate methodologies for design or development of human-robot interactions to foster and enhance context-specific shared goals, interactions, and experiences within larger communities. This paper presents a preliminary study using a community-centered approach to collective perceptions about shared social robots in a retirement village. It reveals novel aspects regarding people's sense of community, community roles and purposes of robots. Findings indicate need of a framework for community robotics and further studies using a community perspective to bring rich insight into goal oriented and context specific multi-user experiences and interactions in HRI.
Swapna Joshi, Selma Sabanovic
RO-MAN2
2017 He said, she said, it said: Effects of robot group membership and human authority on people's willingness to follow their instructions
abstract
Research in HRI indicates that people follow a robot's instructions even when they are incorrect. However, when a robot's instructions or requests contradict those of a human (e.g. an authoritative experimenter), people obey the human instead. This might be due to the experimenter's perceived ingroup status, or to their higher presumed authority compared to the robot. This study manipulated experimenter authority (high, low) and robot group membership (ingroup, neutral) to test how they affected responses to conflicting orders from the two agents depending on the request's importance (big, small). While there was no main effect of group membership and authority on most participant behavior, when experimenter authority was low and the robot an ingroup member, participants defied the experimenter's instructions to turn off an ingroup robot at the end of the experiment, following the robot's instructions instead. Further, request importance affected participant behavior. Participants typically followed the robot's low-importance requests (e.g., moving from one chair to another), but not high-importance requests (e.g., how to perform a simulated task of diagnosing and talking to patients).
Catherine E. Sembroski, Marlena R. Fraune, Selma Sabanovic
RO-MAN3
2017 Editorial introduction: impact, sustainability, and inclusion for JHRI
abstract
Dear colleagues in the HRI community, JHRI needs you!
Odest Chadwicke Jenkins, Selma Sabanovic
J. Hum. Robot Interact.2
2017 Introduction to the Special Issue on HRI Education
abstract
We are happy to present this Special Issue on Education in Human-Robot Interaction (HRI) to the community. As HRI has matured as a field, it is also becoming an increasingly popular educational topic and resource at all levels of instruction, from elementary through graduate programs. While several excellent review articles for the field exist, there is no textbook or recognized curriculum in HRI. The interdisciplinary nature of the field presents students and instructors with opportunities for building on diverse perspectives from design, engineering, computer science, and the social sciences and humanities, as well as challenges in presenting and working with material from such a broad array of disciplines. The authors in this special issue discuss their experiences with and strategies for designing HRI curricula and teaching HRI to students of diverse backgrounds and skill sets. We hope this special issue will inspire many more courses, summer schools, and educational outreach activities in HRI. We also hope that it sparks more discussions about diverse approaches to and necessary standards for HRI curricula.
Selma Sabanovic, Carlotta A. Berry, Cindy L. Bethel
J. Hum. Robot Interact.1
2016 Happy Moves, Sad Grooves: Using Theories of Biological Motion and Affect to Design Shape-Changing Interfaces
abstract
The design of shape-changing interfaces to show emotions relies on craft skill with few clear guidelines. Through two experiments, we explore how to design such interfaces using theories of the relation between biological motion and affect. In the first experiment, 19 participants viewed six shape-changing behaviors that varied the velocity, fluidity, direction, and orientation of the movement of an extrusion from a small box in accordance with existing theories of affective motion. Participants were able to recognize four of the six intended basic Ekman emotions (sadness, fear, happiness, surprise) with above-chance probability. The second experiment used 36 shape-changing behaviors that systematically varied speed, regularity of motion, and direction. For each behavior, 23 participants rated valence, arousal, and dominance. Speed, direction, and orientation impacted emotion ratings significantly and in the predicted directions. These results offer an initial basis for the systematic design of emotions in shape-changing interfaces.
Haodan Tan, John Tiab, Selma Sabanovic, Kasper Hornbæk
Conference on Designing Interactive Systems3
2016 The challenge (not) to go wild!: Challenges and Best Practices to Study HRI in Natural Interaction Settings
abstract
Robots are more and more leaving our labs and entering natural interaction settings, such as homes, care facilities, factories, kindergartens, museums, and many more. In the HRI research community, we are aware of all the challenges involved in studying autonomously behaving agents “in the wild”. In order “not to go wild” we want to discuss best practices as well as pitfalls in studying robots in natural interaction settings. Specifically, we want to focus on challenges in the development of systems for everyday usage in peoples' homes and on challenges in evaluating these systems in the wild, especially with regard to long-term interaction. We want to discuss and explore common and new methodological approaches to study HRI outside the lab, and collect recommendations for fellow researchers. In tandem, we want to reflect our visions for the “new era of socially capable robots” and how we as robot developers and researchers imagine them to integrate into the fabrics of our everyday life.
Astrid M. Rosenthal-von der Pütten, Astrid Weiss, Selma Sabanovic
HRI3
2016 That robot is not for me: Addressing stereotypes of aging in assistive robot design
abstract
Assistive robots designed to support independently living older adults are expected to solve diverse physical, social and mental challenges related to aging. This paper presents how older adults-the potential users of assistive robots-interpret robotic technologies in terms of their usefulness and fit in their everyday lives. We interviewed 7 older adults aged 57-72, showing them photos and videos of assistive robots designed with their age group in mind. Most participants saw the robots as potentially useful in general terms, yet also commented that “those robots are not for me.” An analysis of their discussion about the robots identified several aspects of robot design that participants found problematic, particularly negative representations of aging depicted in robot use scenarios. Participants saw issues related to aging as important daily problems, but did not appreciate the manner in which robotics was expected to address those issues. They were particularly sensitive to the dominance of stereotypes defining aging primarily in terms of disability in robotics discourse and design. Based on participant feedback, we suggest a broader understanding of the experience of aging as the basis for future assistive robot design for this population, and conclude by discussing related robot design possibilities.
Hee Rin Lee, Haodan Tan, Selma Sabanovic
RO-MAN3
2016 Feeling green: Empathy affects perceptions of usefulness and intention to use a robotic recycling bin
abstract
This paper investigates whether empathy towards an interactive trash bin, particularly empathy induced by the bin's anthropomorphic appearance and behavior, affects people's evaluations of the artifact and their intention to use it to recycle. We performed an online survey in which 344 participants viewed a video showing a person interacting with a recycling bin in one of three conditions: non-interactive/control, non-anthropomorphic interactive, and anthropomorphic interactive. We measured people's perceptions of the bin's anthropomorphism and animacy, their empathy and anxiety towards and enjoyment of the bin, their perceptions of its sociability and usefulness, and finally their intentions to use the bin. The results suggest that the level of anthropomorphism attributed to the artifact by participants positively affected their empathy towards the recycling bin, which in turn directly influenced their intentions to use the bin.
Haodan Tan, Selma Sabanovic
RO-MAN3
2016 Getting in Touch: How imagined, actual, and physical contact affect evaluations of robots
abstract
Although it is widely accepted that robots will be used in everyday contexts in near future, many people feel anxious and hold negative attitudes toward robots. This negative reaction might be stronger when users come into direct physical contact with them, particularly when touch is required between robots and humans, (e.g., when using robots as assistants to help elderly people at home). Intergroup contact research in social psychology has proposed various forms of contact as a means to reduce negative feelings toward outgroup members. The present study examined how Contact Type (Actual vs. Imagined) and Contact Modality (Look vs. Touch) with a NAO robot would impact attitudes toward NAO compared to a no-contact control condition. Results showed that nearly any type of contact effectively reduced negative emotions compared to the control condition. However, for participants with preexisting negative emotions toward robots, contact sometimes produced more negative attitudes. We discuss these findings and the resulting implications for future research.
Ricarda Wullenkord, Marlena R. Fraune, Friederike Eyssel, Selma Sabanovic
RO-MAN4
2015 Interaction Expands Function: Social Shaping of the Therapeutic Robot PARO in a Nursing Home
abstract
We use the "social shaping of technology and society" framework to qualitatively analyze data collected through observation of human-robot interaction (HRI) between social actors in a nursing home (staff, residents, visitors) and the socially assistive robot PARO. The study took place over the course of three months, during which PARO was placed in a publicly accessibly space where participants could interact with it freely. Social shaping focuses attention on social factors that affect the use and interpretation of technology in particular contexts. We therefore aimed to understand how different social actors make sense of and use PARO in daily interaction. Our results show participant gender, social mediation, and individual sense making led to differential use and interpretation of the robot, which affected the success of human-robot interactions. We also found that exposure to others interacting with PARO affected the nursing staff's perceptions of robots and their potential usefulness in eldercare. This shows that social shaping theory provides a valuable perspective for understanding the implementation of robots in long-term HRI and can inform interaction design in this domain.
Wan Ling Chang, Selma Sabanovic
HRI2
2015 Rabble of Robots Effects: Number and Form of Robots Modulates Attitudes, Emotions, and Stereotypes
abstract
Robots are expected to become present in society in increasing numbers, yet few studies in human-robot interaction (HRI) go beyond one-to-one interaction to examine how emotions, attitudes, and stereotypes expressed toward groups of robots differ from those expressed toward individuals. Research from social psychology indicates that people interact differently with individuals than with groups. We therefore hypothesize that group effects might similarly occur when people face multiple robots. Further, group effects might vary for robots of different types. In this exploratory study, we used videos to expose participants in a between-subjects experiment to robots varying in Number (Single or Group) and Type (anthropomorphic, zoomorphic, or mechanomorphic). We then measured participants' general attitudes, emotions, and stereotypes toward robots with a combination of measures from HRI (e.g., Godspeed Questionnaire, NARS) and social psychology (e.g., Big Five, Social Threat, Emotions). Results suggest that Number and Type of observed robots had an interaction effect on responses toward robots in general, leading to more positive responses for groups for some robot types, but more negative responses for others.
Marlena R. Fraune, Steven Sherrin, Selma Sabanovic, Eliot R. Smith
HRI3
2014 Designing a Minimalist Socially Aware Robotic Agent for the Home
abstract
We present a minimalist social robot that relies on long timeseries of low resolution data such as mechanical vibration, temperature, lighting, sounds and collisions.Our goal is to develop an experimental system for growing socially situated robotic agents whose behavioral repertoire is subsumed by the social order of the space.To get there we are designing robots that use their simple sensors and motion feedback routines to recognize different classes of human activity and then associate to each class a range of appropriate behaviors.We use the Katie Family of robots, built on the iRobot Create platform, an Arduino Uno, and a Raspberry Pi.We describe its sensor abilities and exploratory tests that allow us to develop hypotheses about what objects (sensor data) correspond to something known and observable by a human subject.We use machine learning methods to classify three social scenarios from over a hundred experiments, demonstrating that it is possible to detect social situations with high accuracy, using the low-resolution sensors from our minimalist robot.
Matthew R. Francisco, Ian B. Wood, Selma Sabanovic, Luis M. Rocha
ALIFE3
2014 Stay on the boundary: artifact analysis exploring researcher and user framing of robot design
abstract
In recent years, HCI researchers have increased their focus on studying the power relationships between researchers and users, and developing methodologies for eliciting design ideas that are sensitive to existing epistemic hierarchies in technology design. The differential value given to expert versus lay knowledge is a central factor in these debates. We apply Artifact Analysis, developed to help designers handle the complexity of digital artifacts, as a method to explore how experts and non-experts understand and frame robots, a technology characterized by significant complexity. Our results show that both non-expert users and expert researchers have knowledge that is significant to future robot development, but they focus on different aspects of the technology - users address mediated and interaction complexity while researchers focus on internal and external complexity. We also found that robots function as boundary objects between experts and users, and suggest that one task designers can perform is to "stay on the boundary" and mediate between the different ways in which experts and non-experts frame emerging technology to develop designs that benefit from insights from both user and researcher perspectives.
Hee Rin Lee, Selma Sabanovic, Erik Stolterman
CHI2
2014 Robot gossip: effects of mode of robot communication on human perceptions of robots
abstract
With robots becoming more prevalent, it is important to understand human attitudes toward robots not only when humans directly interact with the robots as most research examines, but when robots are performing nonsocial tasks (e.g., cleaning) within sight and hearing of humans. This study examined how presumed robot communication style in such situations of human-robot co-location affects human perceptions of a group of robots. Results suggest that communication style of robots did not affect perceptions of robots, but further studies should use different techniques to manipulate supposed communication style.
Marlena R. Fraune, Selma Sabanovic
HRI2
2014 Culturally variable preferences for robot design and use in South Korea, Turkey, and the United States
abstract
Based on the results of an online survey conducted with participants in South Korea (N=73), Turkey (N=46), and the United States (N=99), we show that people's perceptions and preferences regarding acceptable designs and uses for robots are culturally variable on a number of dimensions, including general attitudes towards robots, preferences for robot form, interactivity, intelligence, and sociality. We also explore correlations between these design and use characteristics and factors cited as having an effect on user perceptions and acceptance of robots, such as religious beliefs and media exposure. Our research suggests that culturally variable attitudes and preferences toward robots are not simply reducible to these factors, rather they relate to more specific social dynamics and norms. In conclusion, we discuss potential design and research implications of culturally variable and universally accepted user preferences regarding robots.
Hee Rin Lee, Selma Sabanovic
HRI2
2014 Context congruency and robotic facial expressions: Do effects on human perceptions vary across culture?
abstract
We performed an experimental study (n=48) of the effects of context congruency on human perceptions of robotic facial expressions across cultures (Western and East Asian individuals). We found that context congruency had a significant effect on human perceptions, and that this effect varied by the emotional valence of the context and facial expression. Moreover, these effects occurred regardless of the cultural background of the participants. In short, there were predictable patterns in the effects of congruent/incongruent environmental context on perceptions of robot affect across Western and East Asian individuals. We argue that these findings fit with a dynamical systems view of social cognition as an emergent phenomenon. Taking advantage of such context effects may ease the constraints for developing culturally-specific affective cues in human-robot interaction, opening the possibility to create culture-neutral models of robots and affective interaction.
Casey C. Bennett, Selma Sabanovic, Marlena R. Fraune, Kate Shaw
RO-MAN2
2014 Observational study of naturalistic interactions with the socially assistive robot PARO in a nursing home
abstract
To explore the development and maintenance of human-robot interaction (HRI) in a real-world setting, we conducted a three-month long field study with the socially assistive robot PARO in a nursing home. We placed PARO a public area in the institution and observed naturally occurring interactions of nursing home residents, staff, and visitors with the robot. We collected both quantitative (on-site behavioral coding and interactor counts) and qualitative (observational field notes) data. The results of behavioral analysis complemented with information from our field notes show that interaction with PARO was often not spontaneous and that mediation from staff and family members was integral to successful interactions between residents and PARO. We also observed that individual interactors engaged with the robot in diverse ways based on their needs. We conclude by discussing design and methodological implications for in situ HRI studies.
Wan Ling Chang, Selma Sabanovic, Lesa Huber
RO-MAN2
2014 Negative Attitudes toward minimalistic Robots with intragroup communication styles
abstract
With robots becoming more prevalent in daily life, it is important to understand human attitudes toward robots not only when humans interact with them directly, as most research examines, but also when people are indirectly exposed to robots performing nonsocial tasks (e.g., cleaning) in their vicinity. Because minimalistic robots are at present more likely to be found in households than human-like robots, this study examined human reactions to nonsocial, nonanthropomorphic robots. The specific focus of this study was on how robot communication style during human-robot co-location affects human perceptions of a group of robots. This paper also evaluates the relationship between participants' scores on the Negative Attitudes toward Robots Scale (NARS) and their behavioral response to and perceptions of robots in their environment. Our results suggest that robot communication style did not affect perceptions of robots and that responses on the NARS may not translate directly to behavior toward robots.
Marlena R. Fraune, Selma Sabanovic
RO-MAN2
2014 Designing robots in the wild: in situ prototype evaluation for a break management robot
abstract
As robots move into everyday environments, we need to understand both the social and the technical constraints and affordances for human-robot interaction. We use in situ evaluation of partially functioning prototypes to inform the design of robotic technologies that fit their intended contexts of use and illustrate this method through a case study of iteratively designing a desktop robot for break management in a computerized office. After an initial exploratory study of the office as context of use, we used comparative semi-controlled evaluations of multiple design alternatives to explore how different robot characteristics, specifically embodiment and social interactivity, are perceived by users and affect their break taking. We found evaluating simple prototypes with varying levels of functionality, even when not robust or "complete," provides opportunities for including users in the design process and for identifying emergent factors that impact robot use. Our case study provides insights into the challenges and best practices for performing iterative prototyping and in situ evaluations of robots, which can inform future development of contextually appropriate robotic technologies.
Selma Sabanovic, Sarah Reeder, Bobak Kechavarzi
J. Hum. Robot Interact.1
2013 Perceptions of affective expression in a minimalist robotic face
Casey C. Bennett, Selma Sabanovic
HRI2
2013 Potential use of robots in Taiwanese nursing homes
Wan Ling Chang, Selma Sabanovic
HRI2
2013 Use of seal-like robot PARO in sensory group therapy for older adults with dementia
Wan Ling Chang, Selma Sabanovic, Lesa Huber
HRI2
2013 Weiser's dream in the Korean home: collaborative study of domestic roles, relationships, and ideal technologies
abstract
Following Bell and Dourish's call for a "ubicomp of the present," we visited 14 households in Korea, where Weiser's dreams come true, to study their social dynamics and domestic technologies as a part of these dynamics. We used a participatory research approach in which participants, acting as collaborative ethnographers and co-designers, chose how to describe their homes to us and which existing technologies to discuss. A qualitative analysis of the conversations identified two main themes. The first finding is the highly gendered nature of roles in the Korean home, influenced by traditional Confucian values and reinforced by contemporary neo-liberal norms. The second finding is that domestic technologies are used, adopted, and imagined in the context of these gendered social dynamics rather than just according to functional needs. In conclusion, we emphasize the need to attend to the social dynamics of the home in the design of politically sensitive domestic technologies, which will enable the inclusion of marginalized voices, such as women, in design.
Hee Rin Lee, Selma Sabanovic
UbiComp2
2012 The effect of monitoring by cameras and robots on the privacy enhancing behaviors of older adults
abstract
This paper describes the results of an experimental study in which older adult participants interacted with three monitoring technologies designed to support their ability to age in place in their own home - a camera, a stationary robot, and a mobile robot. The aim of our study was to evaluate users' perceptions of privacy and their tendencies to engage in privacy enhancing behaviors (PEBs) by comparing the three conditions. We found that privacy concerns lead older adults to change their behavior in a home environment while being monitored by cameras or embodied robots. We expected participants to engage in more PEBs when they interacted with a mobile robot, which provided embodied cues of ongoing monitoring; surprisingly, we found the opposite to be true - the camera was the condition in which participants performed more PEBs. We describe the results of quantitative and qualitative analyses of our survey, interview, and observational data and discuss the implications of our study for human-robot interaction, the study of privacy and technology, and the design of assistive robots for monitoring older adults.
Kelly Caine, Selma Sabanovic, Mary Carter
HRI2
2012 The effect of group size on people's attitudes and cooperative behaviors toward robots in interactive gameplay
abstract
This paper contributes to the study of interaction between groups of people and groups of robots by examining the effect of group size on people's attitudes and behaviors toward robots as interaction partners. Our work is motivated by psychological research on human intergroup dynamics, particularly the interindividual-intergroup discontinuity effect, which suggest that interactions among groups are more competitive than interactions among individuals. To test the discontinuity effect in the context of human-robot interaction, we conducted a between-subjects experiment with four conditions, derived by differentiating the ratio of humans to robots in the interaction (one or two humans interacting with one or two robots). Participants played a game with robots in which they were given a chance to exhibit competitive and cooperative behaviors, which we tracked. We also measured changes in participants' attitudes toward robots following gameplay. Our results show that people playing in groups behave more competitively towards the robots than individual human players. However, participants' attitudes toward robots did not change after the short-term interaction.
Wan Ling Chang, Jeremy P. White, Joohyun Park, Anna Holm, Selma Sabanovic
RO-MAN5
2012 Evaluation of control factors affecting the operator's immersion and performance in robotic teleoperation
abstract
This paper presents the results of an experimental study in which users teleoperating a mobile robot evaluated three controllers: a keyboard, a game controller, and a touchpad interface. It is motivated by the need to engage a broader, non-expert user audience in teleoperation as robots become more prevalent in everyday applications. Analysis focuses on how specific control elements and the user's comfort with a device improve the operator's sense of immersion in the task and how this alters performance. Our results show that perceived controllability of the controller, users' level of technological anxiety, and the physical nature of feedback from the controller had an effect on user feelings of immersion and presence. Our findings have implications for the development of controllers that can be used for teleoperating robots by a broad user audience.
Bobak Kechavarzi, Selma Sabanovic, Kurt Weisman
RO-MAN2
2012 Cultural design of domestic robots: A study of user expectations in Korea and the United States
abstract
In this paper, we describe the results of a comparative analysis of user-created designs for future domestic robots made by participants in Korea and the US. We identify their culturally variable expectations and preferences. We use a generative design methodology, which includes users visualizing their designs followed by semi-structured interviews. We describe our results in four areas of design: the look and feel of the robot, interaction mode, social role, and desired task. We identify variable cultural models relating to robotic technology and the cultural meaning of the domestic context as central factors. Finally, we discuss the design implications of our findings to culturally situated robot design.
Hee Rin Lee, JaYoung Sung, Selma Sabanovic, Joenghye Han
RO-MAN3
2011 Designing a robot through prototyping in the wild
abstract
This paper describes the design and initial evaluation of Dewey, a do-it-yourself (DIY) robot prototype aimed to help users manage break-taking in the workplace. We describe the application domain, prototyping and technical implementation, and evaluation of Dewey in a real office environment to show how research using simple prototypes can provide valuable insights into user needs and practices at the early stages of socially assistive robot design.
Selma Sabanovic, Sarah Reeder, Bobak Kechavarzi, Zachary Schall-Zimmerman
HRI1
2010 Breakbot: a social motivator for the workplace
abstract
Workplace injuries commonly result from long periods of inactivity during computer use. Software exists to help remind people to take breaks but is often ineffective. On the basis of design research performed in an office environment, we propose an emotionally expressive companion robot to encourage employees to take breaks and socialize more regularly. Initial reactions to our design were positive, and encourage further investigation.
Sarah Reeder, Lorelei Kelly, Bobak Kechavarzi, Selma Sabanovic
Conference on Designing Interactive Systems4
2010 Predictive State Representations for grounding human-robot communication
abstract
Allowing robots to communicate naturally with humans is an important goal for social robotics. Most approaches have focused on building high-level probabilistic cognitive models. However, research in cognitive science shows that people often build common ground for communication with each other by seeking and providing evidence of understanding through behaviors like mimicry. Predictive State Representations (PSRs) allow one to build explicit, low-level models of the expected outcomes of actions, and are therefore well-suited for tasks that require providing such evidence of understanding. Using human-robot shadow puppetry as a prototype interaction study, we show that PSRs can be used successfully to both model human interactions, and to allow a robot to learn on-line how to engage a human in an interesting interaction.
Eric M. Meisner, Sanmay Das, Volkan Isler, Jeffrey C. Trinkle, Selma Sabanovic, Linnda R. Caporael
ICRA5
2009 ShadowPlay: a generative model for nonverbal human-robot interaction
abstract
Humans rely on a finely tuned ability to recognize and adapt to socially relevant patterns in their everyday face-to-face interactions. This allows them to anticipate the actions of others, coordinate their behaviors, and create shared meaning to communicate. Social robots must likewise be able to recognize and perform relevant social patterns, including interactional synchrony, imitation, and particular sequences of behaviors. We use existing empirical work in the social sciences and observations of human interaction to develop nonverbal interactive capabilities for a robot in the context of shadow puppet play, where people interact through shadows of hands cast against a wall. We show how information theoretic quantities can be used to model interaction between humans and to generate interactive controllers for a robot. Finally, we evaluate the resulting model in an embodied human-robot interaction study. We show the benefit of modeling interaction as a joint process rather than modeling individual agents.
Eric M. Meisner, Selma Sabanovic, Volkan Isler, Linnda R. Caporael, Jeffrey C. Trinkle
HRI2
2008 Tabulation and analysis of questionnaire results of subjective evaluation of seal robot in seven countries
abstract
This paper describes research on a ldquomental commitment robotrdquo. These robots have a different target audience to industrial robots, one that is not so rigidly dependent on objective measures such as accuracy and speed. The main goal of this research is to explore a new area in robotics, with the emphasis on human-robot interaction. In previous research, we classified robots into four categories, which related to their appearance. We then introduced a robot cat and a robot seal, which we evaluated by interviewing a large group of people. The results showed that physical interaction improved their subjective evaluation of the robots. Moreover, a priori knowledge of a subject has a considerable influence on the subjective interpretation and evaluation of mental commitment robots. In this paper, we asked several groups of subjects to evaluate the seal robot known as dasiaParopsila by answering questionnaires that were given out in exhibitions that were held in seven different countries; Japan, U.K., Sweden, Italy, Korea, Brunei and U.S. This paper reports the results of statistical analysis of the evaluation data.
Takanori Shibata, Kazuyoshi Wada, Yousuke Ikeda, Selma Sabanovic
RO-MAN4
2007 A dancing robot for rhythmic social interaction
abstract
This paper describes a robotic system that uses dance as a form of social interaction to explore the properties and importance of rhythmic movement in general social interaction. The system consists of a small creature-like robot whose movement is controlled by a rhythm-based software system. Environmental rhythms can be extracted from auditory or visual sensory stimuli, and the robot synchronizes its movement to a dominant rhythm. The system was demonstrated, and an exploratory study conducted, with children interacting with the robot in a generalized dance task. Through a behavioral analysis of videotaped interactions, we found that the robot's synchronization with the background music had an effect on children's interactive involvement with the robot. Furthermore, we observed a number of expected and unexpected styles and modalities of interactive exploration and play that inform our discussion on the next steps in the design of a socially rhythmic robotic system.
Marek P. Michalowski, Selma Sabanovic, Hideki Kozima
HRI2
2006 Socially distributed perception
abstract
This paper presents a robot search task (social tag) that uses social interaction, in the form of asking for help, as an integral component of task completion. We define socially distributed perception as a robot's ability to augment its limited sensory capacities through social interaction.
Marek P. Michalowski, Carl F. DiSalvo, Dídac Busquets, Laura M. Hiatt, Nik A. Melchior, Reid G. Simmons, Selma Sabanovic
HRI7
2006 Roillo: Creating a Social Robot for Playrooms
abstract
In this paper, we introduce Roillo, a social robotic platform for investigating, in the context of children's playrooms, questions about how to design compelling nonverbal interactive behaviors for social robots. Specifically, we are interested in the importance of rhythm to natural interactions and its role in the expression of affect, attention, and intent. Our design process has consisted of rendering, animation, surveys, mechanical prototyping, and puppeteered interaction with children
Marek P. Michalowski, Selma Sabanovic, Philipp Michel
RO-MAN2