As the population of older adults increases, there is a growing need for support for them to age in place. This is exacerbated by the growing number of individuals struggling with cognitive decline and shrinking number of youth who provide care for them. Artificially intelligent agents could provide cognitive support to older adults experiencing memory problems, and they could help informal caregivers with coordination tasks. To better understand this possible future, we conducted a speed dating with storyboards study to reveal invisible social boundaries that might keep older adults and their caregivers from accepting and using agents. We found that healthy older adults worry that accepting agents into their homes might increase their chances of developing dementia. At the same time, they want immediate access to agents that know them well if they should experience cognitive decline. Older adults in the early stages of cognitive decline expressed a desire for agents that can ease the burden they saw themselves becoming for their caregivers. They also speculated that an agent who really knew them well might be an effective advocate for their needs when they were less able to advocate for themselves. That is, the agent may need to transition from being unremarkable to remarkable. Based on these findings, we present design opportunities and considerations for agents and articulate directions of future research.

The population of older adults experiencing cognitive decline is growing faster than the number of workers who can care for them. Artificially intelligent (AI) agents could assist these older adults, keeping them in their homes longer. For this to happen, older adults must be willing to adopt and rely on agents. Would they trust an agent that might need to report their decline to others? We conducted a speed dating study exploring the impact of agent affiliation (i.e., who the agent should work for). Our healthy and declining participants reacted positively to the idea of agents supporting them. They particularly recognized how the agent would reduce the burden placed on their family caregivers. They viewed affiliation to be dynamic, shifting from the declining older adult and orienting more to their caregivers over the course of cognitive decline. They envisioned the agent modifying its decision-making process to be like their caregivers’.

This paper investigates how conversational agents that are embedded in smart environments should present themselves socially. In an online study, we simulate a future space habitat in which "astronauts" (participants) interact with one or more agents to complete several tasks related to science, maintenance, and inventory. We examine effects of agent affiliation (affiliation with a user, affiliation with a domain, or affiliation with all users and all domains) and narrative perspective (first-vs. third-person references to parts of the environment) on mental models of the smart environment as one or multiple entities, trust, performance, and social variables. Our findings suggest that in this type of setting, interacting with a single agent may increase mental demand, and that agents that speak about embodied interaction in third person are perceived as more trustworthy and competent than agents that speak in first person.

Technology, especially the smartphone, is villainized for taking meaning and time away from in-person interactions and secluding people into “digital bubbles”. We believe this is not an intrinsic property of digital gadgets, but evidence of a lack of imagination in technology design. Leveraging augmented reality (AR) toward this end allows us to create experiences for multiple people, their pets, and their environments. In this work, we explore the design of AR technology that “piggybacks” on everyday leisure to foster co-located interactions among close ties (with other people and pets). We designed, developed, and deployed three such AR applications, and evaluated them through a 41-participant and 19-pet user study. We gained key insights about the ability of AR to spur and enrich interaction in new channels, the importance of customization, and the challenges of designing for the physical aspects of AR devices (e.g., holding smartphones). These insights guide design implications for the novel research space of co-located AR.

Virtual assistants, vacuum robots, security systems, and other smart home technologies are rapidly advancing, evolving, and gaining popularity. This raises questions of how people envision future interactions with smart home systems and how they imagine the future roles of such technologies in society. We deployed an online study that collected fictional short stories from 60 participants about smart home interactions. We identified themes regarding the roles of smart home technologies, social interactions with AI, and concerns about data privacy in the context of the home. We describe our method, discuss insights from the stories that explicitly reflect possible futures and implicitly reflect the present, and make design recommendations based on our findings.

Research has shown how the connections between robots' minds, bodies, and identities can be configured and performed in a variety of ways. In this work, we consider group identity observables: the set of design cues that robot groups use to perform different identity configurations. We explore how group identity observables lead observers to develop different mental models of robot groups. Specifically, we make four key contributions: (1) we define, conceptualize, and taxonomize group identity observables; (2) we use Grounded Theory-informed analysis of qualitative data to produce a taxonomy of users' mental models invoked by variation in those observables; (3) we empirically demonstrate (n=166) how variations in observables lead to different mental models; and (4) we further demonstrate how variations in those observables, and the mental models they evoke, influence key group dynamics constructs like entitativity.

Following the success of the first edition of Robo-Identity, the second edition will provide an opportunity to expand the discussion about artificial identity. This year, we are focusing on emotions that are expressed through speech and voice. Synthetic voices of robots can resemble and are becoming indistinguishable from expressive human voices. This can be an opportunity and a constraint in expressing emotional speech that can (falsely) convey a human-like identity that can mislead people, leading to ethical issues. How should we envision an agent's artificial identity? In what ways should we have robots that maintain a machine-like stance, e.g., through robotic speech, and should emotional expressions that are increasingly human-like be seen as design opportunities? These are not mutually exclusive concerns. As this discussion needs to be conducted in a multidisciplinary manner, we welcome perspectives on challenges and opportunities from variety of fields. For this year's edition, the special theme will be “speech, emotion and artificial identity”.

Designers of smart-home systems must make decisions about the perceived identities and interconnectedness of their various devices. To inform these decisions, we performed an online study to examine whether people perceive multiple devices in a smart home as different interfaces for the same system, devices that talk to each other, or independent devices. We manipulated the types of devices in the system (hetero-geneous!homogeneous), how the devices relayed commands to each other (implicit/explicit), and the task requested. Participants were flexible in how they interpreted the devices, presenting an opportunity for designers to select a suitable model.

With the infusion of computation into workplaces and homes, various service settings, and everyday objects, scholars in human–computer interaction (HCI) and related domains have begun to consider the research and design implications not only of smart “things,” but ofsmart environments. Much of the work on smart environments to date has focused on smart homes; related work in HCI explores user values for smart homes, means of interacting with computation in smart homes (e.g., interfaces and agents), how to balance the needs of multiple stakeholders, and how to preserve user trust and autonomy. However, the smart environments of the future will not always fit the smart home mold of a coalescence of products that exist to automate and ease everyday tasks for the end users. They will be both user-focused and goal-focused, public and private, large and small, and ephemeral and long-lasting. It will benefit the field to look atsmart environmentsas a unit of analysis—including what these different types of environments have in common and what they do not—from a systemic, user experience design-oriented view. In this survey article, we review prior research on smart environments and various related bodies of literature. Informed by our literature review, we articulate fivelensesthat distinguish different types of smart environments from one another. We then propose research directions for future work on this topic.

Social agents and robots are moving into front-line positions in brick and mortar services, taking on roles where they directly interact with customers. These agents could potentially recognize customers to personalize service. Will customers like this, or might they feel monitored and profiled? Robots could also re-embody (move their “personality” between one body and another) in order to take on multiple roles that are typically performed by different people. Will this make customers feel more taken care of, or will it raise concerns about the robot’s competence and expertise? Our work investigates when robots should and should not recognize customers and re-embody. Our online study used storyboards to present possible future interactions between robots and customers across several different service contexts. Our findings suggest that people generally accept robots identifying customers and taking on vastly different roles. However, in some contexts, these robot behaviors seem creepy and untrustworthy.

How are people using current smart home technologies, and how do they conceptualize future ones that are more interconnected and more capable than those available today? We deployed an online survey study to 150 participants to investigate use of and opinions about smart speakers, home robots, virtual assistants, and other smart home devices. We also gauged how impressions of connected smart home devices are shaped by the way the devices interact with one another. Through a mixed-methods qualitative and quantitative approach, we found that people mostly use single devices for single functions, and have simple and brief interactions with virtual assistants. However, they imagine their future devices to have more control over the physical environment (i.e., interact with each other) and envision them interacting with people in more socially complex ways. These findings motivate design considerations and research directions for connected smart home technologies.

We explored different ways in which a multi-robot system might recover after one robot experiences a failure. We compared four recovery conditions: Update (a robot fixes its error and continues the task), Re-embody (a robot transfers its intelligence to a different body), Call (the failed robot summons a second robot to take its place), and Sense (a second robot detects the failure and proactively takes the place of the first robot). We found that trust in the system and perceived competence of the system were higher when a single robot recovered from a failure on its own (by updating or re-embodying) than when a second robot took over the task. We also found evidence that two robots that used the same socially interactive intelligence were perceived more similarly than two robots with different intelligences. Finally, our study revealed a relationship between how people perceive the agency of a robot and how they perceive the performance of the system.

How might the capabilities of voice assistants several decades in the future shape human society? To anticipate the space of possible futures for voice assistants, we asked 149 participants to each complete a story based on a brief story stem set in the year 2050 in one of five different contexts: the home, doctor's office, school, workplace, and public transit. Story completion as a method elicits participants' visions of possible futures, unconstrained by their understanding of current technological capabilities, but still reflective of current sociocultural values. Through a thematic analysis, we find these stories reveal the extremes of the capabilities and concerns of today's voice assistants---and artificial intelligence---such as improving efficiency and offering instantaneous support, but also replacing human jobs, eroding human agency, and causing harm through malfunction. We conclude by discussing how these speculative visions might inform and inspire the design of voice assistants and other artificial intelligence.

People take to social media to share their thoughts, joys, and sorrows. A recent popular trend has been to support and mourn people and pets that have died as well as other objects that have suffered catastrophic damage. As several popular robots have been discontinued, including the Opportunity Rover, Jibo, and Kuri, we are interested in how language used to mourn these robots compares to that to mourn people, animals, and other objects. We performed a study in which we asked participants to categorize deidentified Twitter reactions as referencing the death of a person, an animal, a robot, or another object. Most reactions were labeled as being about humans, which suggests that people use similar language to describe feelings for animate and inanimate entities. We used a natural language toolkit to analyze language from a larger set of tweets. A majority of tweets about Opportunity included second-person ("you") and gendered third-person pronouns (she/he versus it), but terms like "R.I.P" were reserved almost exclusively for humans and animals. Our findings suggest that people verbally mourn robots similarly to living things, but reserve some language for people.

Service robots often perform their main functions in public settings, interacting with more than one person at a time. How these robots should handle the affairs of individual users while also behaving appropriately when others are present is an open question. One option is to design for flexible agent embodiment: letting agents take control of different robots as people move between contexts. Through structured User Enactments, we explored how agents embodied within a single robot might interact with multiple people. Participants interacted with a robot embodied by a singular service agent, agents that re-embody in different robots and devices, and agents that co-embody within the same robot. Findings reveal key insights about the promise of re-embodiment and co-embodiment as design paradigms as well as what people value during interactions with service robots that use personalization.

Interactions with multiple conversational agents and social robots are becoming increasingly common. This raises new design challenges: Should agents and robots be modeled after humans, presenting their entity (i.e., social presence) as bound to a single body, or should they take advantage of non-human capabilities, such as moving their social presence from body to body across service touchpoints and contexts? We conducted a User Enactments study in which participants interacted with agents that had one social presence per body, that could re-embody (move their social presence from body to body), and that could co-embody (move their social presence into a body that already contains another). Reactions showed that participants felt comfortable with re-embodying agents, who created more seamless and efficient experiences. Yet situations that required expertise or concentration raised concerns about non-human behaviors. We report on our insights regarding collaboration and coordination with several agents in multi-step interactions.

We describe an exploration of how kiosk-type stationary robots might provide navigation instructions for blind people. Inspired by a technique used by Orientation & Mobility experts in which a route is traced out on a person's palm, we developed five methods that supplement verbal instructions with physical movements. We explored the usability, strengths, and limitations of each of our methods in two exploratory studies with blind participants. One method, in which the robot used its entire arm to create path gestures while participants held its gripper, was preferred by 5 out of 8 blind participants and performed comparably on a recall task as a verbal-only instruction method. A closer approximation of the original palm method failed. We analyzed interview data to understand the reasons behind the failures and successes. We discuss the lessons learned from our studies about instruction methods, how robots in public settings can be useful for blind people, and the challenges of deploying such systems in public.

Interactions with social robots in public and private spaces are becoming more and more common and varied. As this trend continues, it is important to understand how a robot's embodiment influences its ability to calibrate trust and comfort with its users and behave in accordance with social norms. This is especially true when one social intelligence embodies multiple physical robots (re-embodiment). We have conducted two studies-one quantitative and and one qualitative-which shed light on the way robots should be embodied and re-embodied by intelligences during different types of social interactions. This paper outlines our previous work on elucidating the role of embodiment in social interactions and experimenting with re-embodiment as a design paradigm, and it describes the directions in which we plan to take this research in the near future.

Human-robot interactions that involve multiple robots are becoming common. It is crucial to understand how multiple robots should transfer information and transition users between them. To investigate this, we designed a 3 × 3 mixed-design study in which participants took part in a navigation task. Participants interacted with a stationary robot who summoned a functional (not explicitly social) mobile robot to guide them. Each participant experienced the three types of robot-robot interaction: representative (the stationary robot spoke to the participant on behalf of the mobile robot), direct (the stationary robot delivered the request to the mobile robot in a straightforward manner), and social (the stationary robot delivered the request to the mobile robot in a social manner). Each participant witnessed only one type of robot-robot communication: silent (the robots covertly communicated), explicit (the robots acknowledged that they were communicating), or reciting (the stationary robot said the request aloud). Our results show that it is possible to instill socialness in and improve likability of a functional robot by having a social robot interact socially with it. We also found that covertly exchanging information is less desirable than reciting information aloud.

As robots are increasingly endowed with social and communicative capabilities, they will interact with humans in more settings, both collaborative and competitive. We explore human-robot relationships in the context of a competitive Stackelberg Security Game. We vary humanoid robot expressive language (in the form of “encouraging” or “discouraging” verbal commentary) and measure the impact on participants' rationality, strategy prioritization, mood, and perceptions of the robot. We learn that a robot opponent that makes discouraging comments causes a human to play a game less rationally and to perceive the robot more negatively. We also contribute a simple open source Natural Language Processing framework for generating expressive sentences, which was used to generate the speech of our autonomous social robot.

Autonomous vehicles have been in development for nearly thirty years and recently have begun to operate in real-world, uncontrolled settings. With such advances, more widespread research and evaluation of human interaction with autonomous vehicles (AV) is necessary. Here, we present an interview study of 32 pedestrians who have interacted with Uber AVs. Our findings are focused on understanding and trust of AVs, perceptions of AVs and artificial intelligence, and how the perception of a brand affects these constructs. We found an inherent relationship between favorable perceptions of technology and feelings of trust toward AVs. Trust in AVs was also influenced by a favorable interpretation of the company's brand and facilitated by knowledge about what AV technology is and how it might fit into everyday life. To our knowledge, this paper is the first to surface AV-related interview data from pedestrians in a natural, real-world setting.

Telepresence robots provide remote team members with embodied presence, but whether this improves remote teammate participation, remote users' perceptions of team collaboration, or collocated members' perceptions of remote teammates is an open question. We conducted an experiment in which teams of two collocated members and one telepresent (remote) member solved a word puzzle requiring a translation key. We varied who had access to the key to examine effects of resource accessibility in distributed groups: in the Robot Information condition, the remote pilot (RP) possessed the key; in the Shared Information condition, all team members possessed the key; in the Local Information condition, only collocated participants (CPs) possessed the key. Audio transcripts were analyzed for differences in the number of words spoken by each team member. RPs spoke significantly less than CPs, especially when they lacked the translation key. RPs perceived greater task difficulty and less ease of communication than CPs. CPs rated other CPs as more trustworthy than RPs. This suggests an imbalance between collocated and remote collaborators that can negatively affect collaboration. We discuss implications for the design and use of telepresence robots in the workplace.