We investigate the ethical and privacy implications of voice-first ambient interfaces (VFAIs) for aging in place through an in-depth engagement with five older adults. Our participants were in the process of becoming experienced VFAI users, and had used a VFAI-based design probe for health data reporting. We create and iteratively refine an interview protocol using Privacy Cards. We customize Privacy Cards by drawing on participants’ previous interviews and device usage logs. Using Privacy Cards, we conduct interviews to surface their mental models, and explore their privacy concerns. We find insufficient mental models for proper consent. For example, participants did not know who could access their data, and experienced difficulty distinguishing built-in functionality from third-party apps. Participants initially expressed little worry about VFAI-related ethical concerns, but interviews with Privacy Cards revealed nuanced issues, resulting in various implications for future research and design.

The use of interactive voice assistants (IVAs) in healthcare provides an avenue to address diverse health needs, such as gaps in the medical recovery period for older adult patients who have recently experienced serious illness. By using a voice-assisted medical recovery curriculum, discharged patients can receive ongoing support as they recover. However, there exist significant medical and technology disparities among older adults, particularly among Black older adults. We recruited 26 Black older adults to participate in the design process of an IVA-enacted medical recovery curriculum by providing feedback during the early stages of design. Lack of cultural relevancy, accountability, privacy concerns, and stigmas associated with aging and disability made participants reluctant to engage with the technology unless in a position of extreme need. This study underscored the need for Black cultural representation, whether it regarded the IVA's accent, the types of media featured, or race-specific medical advice, and the need for strategies to address participants' concerns and stigmas. Participants saw the value in the curriculum for those who did not have caregivers and deliberated about the trade-offs the technology presented. We discuss tensions surrounding inclusion and representation and conclude by showing how we enacted the lessons from this study in future design plans.

Motivating children to learn is a major challenge in education. One way to inspire motivation to learn is through immersion. We combine the immersive potential of augmented reality (AR), narrative, and large language models (LLMs) to bridge fantasy with reality in a mobile application, Moon Story, that teaches elementary schoolers astronomy and environmental science. Our system also builds upon learning theories such as culturally-relevant pedagogy. Using our application, a child embarks on a journey inspired by Chinese mythology, engages in real-world AR activities, and converses with a fictional character powered by an LLM. We conducted a controlled experiment (N = 50) with two conditions: one using an LLM and one that was hard-coded. Both conditions resulted in learning gains, high engagement levels, and increased science learning motivation. Participants in the LLM condition also wrote more relevant answers. Finally, participants of both Chinese and non-Chinese heritage found the culturally-based narrative compelling.

From ELIZA to Alexa, Conversational Agents (CAs) have been deliberately designed to elicit or project empathy. Although empathy can help technology better serve human needs, it can also be deceptive and potentially exploitative. In this work, we characterize empathy in interactions with CAs, highlighting the importance of distinguishing evocations of empathy between two humans from ones between a human and a CA. To this end, we systematically prompt CAs backed by large language models (LLMs) to display empathy while conversing with, or about, 65 distinct human identities, and also compare how different LLMs display or model empathy. We find that CAs make value judgments about certain identities, and can be encouraging of identities related to harmful ideologies (e.g., Nazism and xenophobia). Moreover, a computational approach to understanding empathy reveals that despite their ability to display empathy, CAs do poorly when interpreting and exploring a user’s experience, contrasting with their human counterparts.

Voice-first ambient interfaces (VFAIs), such as Alexa, can uniquely meet the health needs of older adults. However, inequitable technology may worsen health disparities and decrease independence, calling for participatory methods to increase the agency of older adults in the design processes of these technologies. We adapt and conduct a participatory design workshop to focus on ambient interfaces for home health with 13 diverse older adults in San Francisco's Tenderloin neighborhood. Using the prototypes they made as discussion catalyzers, participants shared different perspectives entailing stigmatized topics that can be difficult to discuss, such as drug use, sex, isolation, and dementia. They deliberated on the negative implications of VFAIs, such as a justified concern for surveillance, in conjunction with their positive implications, such as receiving always-available ''non-judgmental'' support. Similarly, the risk of leaking drug use data was considered alongside the benefits of sharing important medical information with clinicians. We synthesize our findings into design considerations, such as how we might address varying levels of trust in different stakeholders and reduce stigma that may hinder users from fully benefiting from VFAIs' capabilities.

We focus on the stories of five older adults who became voice assistant users through our study, and with whom we speculated about future interfaces through two design probes, one for health data reporting and one for positive reminiscing. We delivered a voice-first ambient interface (VFAI) to each participant, and closely observed participants’ journeys through periodic themed interviews (16 hours, 21 minutes of transcribed recordings), usage log reviews (4,657 entries), and phone and text support. Participants’ lived experiences impacted their perceptions and interactions with their VFAI, fueling rich insights about how to design for diverse needs. For example, while one participant saw increased potential in the VFAI after interacting with the design probe for health data reporting, another was skeptical of using it to communicate with her doctor. We contribute an in-depth exploration of VFAIs to support aging in place, implications for design, and areas for future work for tailoring VFAIs towards enabling continuity of care in people’s homes.

The rapid pace of technological progress carries with it a heightened risk of ethics and privacy violations, creating an urgent need for mechanisms to address this risk. We approach this problem from the perspective of designers and technologists aiming to design technology that better accounts for ethical implications. We iteratively developed a workshop-based method (N=113, seven workshops) for probing ethical implications of emerging ubiquitous computing technologies. We contribute a method that enables people with varying levels and areas of domain expertise and with a variety of lived experiences to collectively speculate about the ethical implications of emerging technologies, navigate value tensions, and prototype artifacts as a way to grapple with those tensions. We introduce implication design as a means for participants with and without design experience to communicate how a technology might change to better serve them. Lastly, we share our learnings from and reflections on our design process.

Outdoor learning experiences, such as field trips, can improve children’s science achievement and engagement, but these experiences are often difficult to deliver without extensive support. Narrative in educational experiences can provide needed structure, while also increasing engagement. We created a narrative-based, mobile application to investigate how to guide young learners in interacting with their local, outdoor environment. In a second variant, we added augmented reality and image classification to explore the value of these features. A study (n = 44) found that participants using our system demonstrated learning gains and found the experience engaging. Our findings identified several major themes, including participant excitement for hands-on interactions with nature, curiosity about the characters, and enthusiasm toward typing their thoughts and observations. We offer a set of design implications for supporting narrative-based, outdoor learning with immersive technology.

Older adults around the world lack access to a wide range of potentially life-changing digital applications, services, and information that could be provided by voice assistants (such as Amazon’s Alexa, Google’s Assistant, or Apple’s Siri). However, older adults’ needs are underrepresented in the design of voice assistants. Because of this, we are missing opportunities for digital inclusion, and increasing risks of excluding older adults as these devices permeate public settings. In this work, we video record older adults (n=26) interacting with a multi-modal voice assistants while waiting in line at food pantries, and use Interaction Analysis to draw insights from these recordings. We find that by being agnostic to body language, audio-prosodic features, and other contextual factors, voice assistants fail to capture and react to some important aspects of interactions. We discuss design (e.g, interpreting users’ posture as a cue to wake the device when they are leaning towards the device) and research (e.g., surveillance trade-offs) implications, and argue for the use of multi-modal inputs with attention to privacy. Designing and training voice assistants to take in and appropriately respond to non-verbal cues may increase their inclusivity, helping them fulfill important needs of our aging population.

One key technique people use in conversation and collaboration is conversational repair. Self-repair is the recognition and attempted correction of one's own mistakes. We investigate how the self-repair of errors by intelligent voice assistants affects user interaction. In a controlled human-participant study (N =101), participants asked Amazon Alexa to perform four tasks, and we manipulated whether Alexa would "make a mistake'' understanding the participant (for example, playing heavy metal in response to a request for relaxing music) and whether Alexa would perform a correction (for example, stating, "You don't seem pleased. Did I get that wrong?'') We measured the impact of self-repair on the participant's perception of the interaction in four conditions: correction (mistakes made and repair performed), undercorrection (mistakes made, no repair performed), overcorrection (no mistakes made, but repair performed), and control (no mistakes made, and no repair performed). Subsequently, we conducted free-response interviews with each participant about their interactions. This study finds that self-repair greatly improves people's assessment of an intelligent voice assistant if a mistake has been made, but can degrade assessment if no correction is needed. However, we find that the positive impact of self-repair in the wake of an error outweighs the negative impact of overcorrection. In addition, participants who recently experienced an error saw increased value in self-repair as a feature, regardless of whether they experienced a repair themselves.

To help both designers and people with tetraplegia fully realize the benefts of voice assistant technology, we conducted interviews with fve people with tetraplegia in the home to understand how this population currently uses voice-based interfaces as well as other technologies in their everyday tasks. We found that people with tetraplegia use voice assistants in specifc places, such as in their beds, or when traveling in their wheelchair. In addition, we note the inefciencies for people with tetraplegia when using voice assistance.

Dominance is a key aspect of interpersonal relationships. To what extent do nonverbal indicators related to dominance status translate to a nonanthropomorphic robot? An experiment ( N = 25) addressed whether a mobile robot's motion style can influence people's perceptions of its status. Using concepts from improv theater literature, we developed two motion styles across three scenarios (robot makes lateral motions, approaches, and departs) to communicate a robot's dominance status through nonverbal expression. In agreement with the literature, participants described a motion style that was fast, in the foreground, and more animated as higher status than a motion style that was slow, in the periphery, and less animated. Participants used fewer negative emotion words to describe the robot with the purportedly high-status movements versus the purportedly low-status movements, but used more negative emotion words to describe the robot when it made departing motions that occurred in the same style. This result provides evidence that guidelines from improvisational theater for using nonverbal expression to perform interpersonal status can be applied to influence perception of a nonanthropomorphic robot's status, thus suggesting that useful models for more complicated behaviors might similarly be derived from performance literature and theory.