This paper presents a virtual reality (VR) study that examines how multiple road users jointly interact with an autonomous vehicle (AV) in complex traffic scenarios. Moving beyond dyadic studies (e.g., AV-pedestrian or AV-passenger), our multi-user setup simulates ambiguous all-way stop intersections involving a pedestrian, a human driver in a conventional vehicle, and a passenger in an AV, all interacting simultaneously with the AV. We investigated how users perceive and respond to two distinct types of AV behaviors: an efficient AV that proceeds as soon as it is safe to do so, and a prosocial AV that yields to others before entering the intersection. Sixteen groups of three participants (N=48) took part in the study, with each group interacting with a single AV type across four ambiguous traffic scenarios. Our findings show that even simple AV behavior logics can meaningfully shape crossing negotiation dynamics and highlight how trust and perception can vary across different user roles. We conclude by discussing how our methods and insights can inform the research and design of AV interactions in complex multi-agent traffic environments.

Numerous driving simulator systems are available and are continuing to be developed.However, we believe many simulator offerings are built around what is technically possible rather than what is useful to the researchers that might use such systems.This points to a critical need to understand what makes a driving simulator practical and effective for automotive interface design researchers.To remedy this shortcoming, we conducted video interviews with 15 industry and academic researchers engaged in automotive interface design research.We transcribed and performed thematic analysis on the data collected to better understand the different ways that researchers are using driving simulators, and what challenges they still face.We identified needs across three broad dimensions including: (1) Participant Experience, (2) Research Needs, and (3) Operationalization Requirements.By categorizing these needs, we aim to inform the development of future simulation tools that are more accessible to researchers from diverse backgrounds. CCS Concepts• Human-centered computing → Systems and tools for interaction design.

Micromobility vehicles, such as e-scooters, provide ecological and financial advantages over automotive transportation. However, as with car drivers, micromobility users often perform non-driving related tasks (NDRTs), interacting with stereo controls or navigation tasks, which can lead to accidents. It remains unclear what control interfaces are appropriate and safe for micromobility. We evaluated six interface modalities for NDRTs and conducted a within-subjects study with 35 participants (yielding N=210 observations) in an e-scooter simulator to compare modality safety and preferences. Our results align with existing work on gaze and tactility in the automotive NDRTs context. However, unique to e-scooters, interfaces that required users to alter their grip on the handlebars were less preferred as they compromised stability. Social comfort also emerged as a critical factor due to concerns about public visibility. This work aims to encourage the design of safer, more socially acceptable interfaces for e-scooters and other emerging micromobility vehicles.

Figure 1: We conducted a Virtual Reality driving simulation study to explore interactions between a human driver and an autonomous vehicle at traffic intersections in a pseudo-naturalistic setting to understand the influence of contingent driving behaviors in different contexts (two of four scenarios shown above).

In the Wizard-of-Oz study paradigm, human "wizards" perform not-yet-implemented system behavior, simulating, among others, how autonomous robots could interact in public to see how unwitting bystanders respond. This paper analyzes a 60-minute video recording of two wizards in a public plaza who are operating two trash-collecting robots within their line of sight. We take an ethnomethodology and conversation analysis perspective to scrutinize interactions between the wizards and the people in the plaza, focusing on critical instances where one robot gets stuck and requires collaborative intervention by the wizards. Our analysis unpacks how the wizards deal with emergent problems by pushing one robot into the other, how they manage front and backstage interactions, and how they monitor the location of each other's robots. We discuss how scrutinizing the work of wizards can inform explorative Wizard-of-Oz paradigms, the design of multi-agent robot systems, and the operation of urban robots from a distance.

Robots are imperfect and will often fail multiple times during interactions. Despite this, there is a knowledge gap in understanding how humans respond to successive robot failures. In a user study with 26 participants, we explored human responses to successive robot conversational errors. We found that users typically resort to reformulating their prompts, modifying the verbal tone and cadence when encountering repeated failures. A range of emotional displays emerges, including confusion, frustration, and occasional amusement, which evolve throughout the interaction. We further conducted a statistical analysis of participants' behavioral features, revealing insights that could inform automated approaches to detecting and responding to robot errors in successive failure scenarios.

The integration of large language models (LLMs) into conversational robots has made human-robot conversations more dynamic. Yet, LLM-powered conversational robots remain prone to errors, e.g., misunderstanding user intent, prematurely interrupting users, or failing to respond altogether. Detecting and addressing these failures is critical for preventing conversational breakdowns, avoiding task disruptions, and sustaining user trust. To tackle this problem, the ERR@HRI 2.0 Challenge provides a multimodal dataset of LLM-powered conversational robot failures during human-robot conversations and encourages researchers to benchmark machine learning models designed to detect robot failures. The dataset includes 16 hours of dyadic human-robot interactions, incorporating facial, speech, and head movement features. Each interaction is annotated with the presence or absence of robot errors from the system perspective, and perceived user intention to correct for a mismatch between robot behavior and user expectation. Participants are invited to form teams and develop machine learning models that detect these failures using multimodal data. Submissions will be evaluated using various performance metrics, including detection accuracy and false positive rate. This challenge represents another key step toward improving failure detection in human-robot interaction through social signal analysis.

Dancers often prototype movements themselves or with each other during improvisation and choreography. How are these interactions altered when physically manipulable technologies are introduced into the creative process? To understand how dancers design and improvise movements while working with instruments capable of non-humanoid movements, we engaged dancers in workshops to co-create movements with a robot arm in one-human-to-one-robot and three-human-to-one-robot settings. We found that dancers produced more fluid movements in one-to-one scenarios, experiencing a stronger sense of connection and presence with the robot as a co-dancer. In three-to-one scenarios, the dancers divided their attention between the human dancers and the robot, resulting in increased perceived use of space and more stop-and-go movements, perceiving the robot as part of the stage background. This work highlights how technologies can drive creativity in movement artists adapting to new ways of working with physical instruments, contributing design insights supporting artistic collaborations with non-humanoid agents.

The maker movement embodies a resurgence in DIY creation, merging physical craftsmanship and arts with digital technology support. However, mere technological skills and creativity are insufficient for economically and psychologically sustainable practice. By illuminating and smoothing the path from "maker" to "maker entrepreneur," we can help broaden the viability of making as a livelihood. Our research centers on makers who design, produce, and sell physical goods. In this work, we explore the transition to entrepreneurship for these makers and how technology can facilitate this transition online and offline. We present results from interviews with 20 USA-based maker entrepreneurs (i.e., lamps, stickers), six creative service entrepreneurs (i.e., photographers, fabrication), and seven support personnel (i.e., art curator, incubator director). Our findings reveal that many maker entrepreneurs 1) are makers first and entrepreneurs second; 2) struggle with business logistics and learn business skills as they go; and 3) are motivated by non-monetary values. We discuss training and technology-based design implications and opportunities for addressing challenges in developing economically sustainable businesses around making.

Safety-critical data, such as crash and near-crash records, are crucial to improving autonomous vehicle (AV) design and development. Sharing such data across AV companies, academic researchers, regulators, and the public can help make all AVs safer. However, AV companies rarely share safety-critical data externally. This paper aims to pinpoint why AV companies are reluctant to share safety-critical data, with an eye on how these barriers can inform new approaches to promote sharing. We interviewed twelve AV company employees who actively work with such data in their day-to-day work. Findings suggest two key, previously unknown barriers to data sharing: (1) Datasets inherently embed salient knowledge that is key to improving AV safety and are resource-intensive. Therefore, data sharing, even within a company, is fraught with politics. (2) Interviewees believed AV safety knowledge is private knowledge that brings competitive edges to their companies, rather than public knowledge for social good. We discuss the implications of these findings for incentivizing and enabling safety-critical AV data sharing, specifically, implications for new approaches to (1) debating and stratifying public and private AV safety knowledge, (2) innovating data tools and data sharing pipelines that enable easier sharing of public AV safety data and knowledge ; (3) offsetting costs of curating safety-critical data and incentivizing data sharing.

In this work, we analyze video data and interviews from a public deployment of two trash barrel robots in a large public space to better understand the sensemaking activities people perform when they encounter robots in public spaces. Based on an analysis of 274 human–robot interactions and interviews with N = 65 individuals or groups, we discovered that people were responding not only to the robots or their behavior, but also to the general idea of deploying robots as trashcans, and the larger social implications of that idea. They wanted to understand details about the deployment because having that knowledge would change how they interact with the robot. Based on our data and analysis, we have provided implications for design that may be topics for future human–robot design researchers who are exploring robots for public space deployment. Furthermore, our work offers a practical example of analyzing field data to make sense of robots in public spaces.

Goods are designed, produced, and eventually purchased by consumers who will typically use them as their marketing and branding suggests. What happens when, instead, existing products are used to provide the foundational components for novel devices? We argue that post-consumer product reuse should be a central and ongoing concern in the design of interactive devices. This paper examines four case studies of interactive devices produced by reusing and repurposing existing consumer products. By critically analyzing these case studies, we demonstrate how reuse is leveraged in practice to exploit macro and micro economic niches. These case studies illustrate factors beyond noble ecological intent that make reuse sustainable: mechanical robustness, affordability, and technological bootstrapping. We conclude with a discussion of the challenges and opportunities inherent to this mode of production.

We have developed a Communal eXtended-Reality (CXR) system that enables groups of people in a shared moving vehicle to view a common geo-synchronized tour. This paper describes the geo-synchronized multi-user extended reality system we created to provide a situated and shared experience to promote community engagement. This paper describes: (a) the technical implementation of the CXR system, which geo-locates and orients the view of the participant within the moving vehicle; (b) the immersive digital twin tour, critically aligned with the real-life location; (c) our fallback system, which allows people who feel disoriented or motion-sick to continue along with the content of the tour. We validated the sense of communality, comfort, and effectiveness of the system through in-ride observation and post-ride surveys. Our intent is to enable development of similar systems to foster communal engagement in communities worldwide.

Amidst the replication crisis, it is increasingly clear that we need to understand contextual factors that drive participant behavior, because those factors influence the applicability of study findings more broadly. For AutoUI, as we conduct interaction studies involving drivers, pedestrians, and other traffic participants, it is useful to characterize the traffic contexts that human participants are familiar with, because their prior experiences with traffic are likely to influence their behaviors within the context of a controlled study.

The design of self-driving vehicles requires an understanding of the social interactions between drivers in resolving vague encounters, such as at un-signalized intersections. In this paper, we make the case for social situation awareness as a model for understanding everyday driving interaction. Using a dual-participant VR driving simulator, we collected data from driving encounter scenarios to understand how (N=170) participant drivers behave with respect to one another. Using a social situation awareness questionnaire we developed, we assessed the participants’ social awareness of other driver’s direction of approach to the intersection, and also logged signaling, speed and speed change, and heading of the vehicle. Drawing upon the statistically significant relationships in the variables in the study data, we propose a Social Situation Awareness model based on the approach, speed, change of speed, heading and explicit signaling from drivers.

Computing devices become waste for a variety of reasons. They breakdown, become obsolete, or are no longer trendy. These events are so common that currently, e-waste has become the largest consumer waste stream in the world. However, taking apart e-waste devices reveals how they often contain many useful parts and components that could be scrapped and creatively integrated into new forms. These include highly expressive materials like sensors, displays, micro-controllers, etc. This workshop will explore processes in creative making with e-waste, examining the unique materiality of e-waste and how its reuse differs and/or converges with other material reuse processes. To do so, our workshop will combine hands-on activities (tear downs, rapid prototyping, and tutorials) with discussion of the challenges and opportunities in this space. We aim to use these activities to explore how HCI can better support the creative acts of making with e-waste across a wide audience.

The introduction of robots in public spaces raises many questions concerning emergent interactions with robots. In this paper, we use video analysis to study two robotic trashcans deployed in a busy city square. We focus on the movement-based practices that emerged between the robot, the robot operators, and the inhabitants of the square. These practices spanned ways of attracting the robot and disposing of trash, the robot ’asking’ for trash, ’demonstrations’ by those in the square, as well as passersby in the square navigating around and in coordination with the robots. In discussion, we document these ’spontaneous simple sequential systematics’ - interactions that were systematic (they had an order), sequential (they had parts that happened one at a time), simple (in that they could be understood and copied by an observer) and spontaneous (they could be produced with no prompting or training). Building on this we discuss how we might think of robotic motion as a design space, along with HCI contributions to urban robotics.

In automotive user interface design, testing often starts with lab-based driving simulators and migrates toward on-road studies to mitigate risks. Mixed reality (XR) helps translate virtual study designs to the real road to increase ecological validity. However, researchers rarely run the same study in both in-lab and on-road simulators due to the challenges of replicating studies in both physical and virtual worlds. To provide a common infrastructure to port in-lab study designs on-road, we built a platform-portable infrastructure, Portobello, to enable us to run twinned physical-virtual studies. As a proof-of-concept, we extended the on-road simulator XR-OOM with Portobello. We ran a within-subjects, autonomous-vehicle crosswalk cooperation study (N=32) both in-lab and on-road to investigate study design portability and platform-driven influences on study outcomes. To our knowledge, this is the first system that enables the twinning of studies originally designed for in-lab simulators to be carried out in an on-road platform.

External Human-Machine Interfaces (eHMIs) have been evaluated to facilitate interactions between Automated Vehicles (AVs) and pedestrians. Most eHMIs are, however, visual/ light-based solutions, and multi-modal eHMIs have received little attention to date. We ran an experimental video study (N = 29) to systematically understand the effect on pedestrian’s willingness to cross the road and user preferences of a light-based eHMI (light bar on the bumper) and two sound-based eHMIs (bell sound and droning sound), and combinations thereof. We found no objective change in pedestrians’ willingness to cross the road based on the nature of eHMI, although people expressed different subjective preferences for the different ways an eHMI may communicate, and sometimes even strong dislike for multi-modal eHMIs. This shows that the modality of the evaluated eHMI concepts had relatively little impact on their effectiveness. Consequently, this lays an important groundwork for accessibility considerations of future eHMIs, and points towards the insight that provisions can be made for taking user preferences into account without compromising effectiveness.

Despite the recent advancements in robotics and machine learning (ML), the deployment of autonomous robots in our everyday lives is still an open challenge. This is due to multiple reasons among which are their frequent mistakes, such as interrupting people or having delayed responses, as well as their limited ability to understand human speech, i.e., failure in tasks like transcribing speech to text. These mistakes may disrupt interactions and negatively influence human perception of these robots. To address this problem, robots need to have the ability to detect human-robot interaction (HRI) failures. The ERR@HRI 2024 challenge tackles this by offering a benchmark multimodal dataset of robot failures during human-robot interactions, encouraging researchers to develop and benchmark multimodal machine learning models to detect these failures. We created a dataset featuring multimodal non-verbal interaction data, including facial, speech, and pose features from video clips of interactions with a robotic coach, annotated with labels indicating the presence or absence of robot mistakes, user awkwardness, and interaction ruptures, allowing for the training and evaluation of predictive models. Challenge participants have been invited to submit their multimodal ML models for detection of robot errors, to be evaluated against various performance metrics such as accuracy, precision, recall, F1 score, with and without a margin of error reflecting the time-sensitivity of these metrics. The results of this challenge will help the research field in better understanding the robot failures in human-robot interactions and designing autonomous robots that can mitigate their own errors after successfully detecting them.

Humans have the ability to anticipate what will happen in their environment based on perceived information. Their anticipation is often manifested as an externally observable behavioral reaction, which cues other people in the environment that something bad might happen. As robots become more prevalent in human spaces, robots can leverage these visible anticipatory responses to assess whether their own actions might be "a bad idea?" In this study, we delved into the potential of human anticipatory reaction recognition to predict outcomes. We conducted a user study wherein 30 participants watched videos of action scenarios and were asked about their anticipated outcome of the situation shown in each video ("good" or "bad"). We collected video and audio data of the participants reactions as they were watching these videos. We then carefully analyzed the participants’ behavioral anticipatory responses; this data was used to train machine learning models to predict anticipated outcomes based on human observable behavior. Reactions are multimodal, compound and diverse, and we find significant differences in facial reactions. Model performances are around 0.5-0.6 test accuracy, and increase notably when nonreactive participants are excluded from the dataset. We discuss the implications of these findings and future work. This research offers insights into improving the safety and efficiency of human-robot interactions, contributing to the evolving field of robotics and human-robot collaboration.

Driven by the vision of future responsive environments, where everyday surroundings can perceive human behaviors and respond through intelligent robotic actuation, we propose Wizard of Props (WoP): a human-centered design workflow for creating expressive, implicit, and meaningful interactions. This collaborative experience prototyping approach integrates full-scale physical props with Mixed Reality (MR) to support ideation, prototyping, and rapid testing of responsive environments. We present two design explorations that showcase our investigations of diverse design solutions based on varying technology resources, contextual considerations, and target audiences. Design Exploration One focuses on mixed environment building, where we observe fluid prototyping methods. In Design Exploration Two, we explore how novice designers approach WoP, and illustrate their design ideas and behaviors. Our findings reveal that WoP complements conventional design methods, enabling intuitive body-storming, supporting flexible prototyping fidelity, and fostering expressive environment-human interactions through in-situ improvisational performance.

Farm Management Information Systems (FMIS) integrate data from a variety of sources, including sensors, for the purpose of enabling farmers to interpret past activity and predict future performance. FMIS is traditionally designed for and used by large farms, given their capital and need for automation and scale-up. This paper examines the current data collection practices on small and medium farms so that FMIS systems can be better designed to their needs. Our empirical research comprises interviews conducted during 10 farm visits. Our semi-structured interviews incorporated questions about daily activities, points of decision-making, data sharing, and incentives for data collection. We analyzed the interviews by focusing on possible obstacles to adopting expanding digital data collection practices and how expanded data collection might help fulfill farmers' goals and motivations. We found that farmers use their own bespoke data collection techniques instead of or in parallel to more formalized methods and often hold key observations and hypotheses in their heads rather than committing them to any data collection system at all. Key barriers to FMIS adoption include technology skepticism, technical hurdles, lack of support, and self-doubt in technical skills. Based on this empirical work and analysis, we recommend that FMIS systems can best address the needs of small and medium farms by 1) accounting for the farmers' different approaches to memorizing vs. storing data, 2) integrating rather than trying to replace existing practices, and 3) considering the economic and political motivations driving farm decision-making and practices.

The volatility of modern economics, and marketing paired with highly extended and globalized supply chains produces goods with explicit market inefficiencies. This pictorial explores a design process, “Garbatrage,” wherein designers exploit the difference in value between parts in waste product and those used in new products. Drawing upon our own material re-use of hoverboards, we inductively outline a framework to help designers consider the micro- and macro-economic context of material recapture of waste components, necessary to make for more circular economies and sustainable supply chains at scale. We then apply this framework to identify other potential targets for Garbatrage. We conclude with opportunities for design to make the practice of Garbatrage more widely adoptable.

This study examines three modalities for designing live-in van interiors. Participants (N=18) situated within an empty van were asked to explore potential designs using physical cardboard prototyping, a commercial software application (Vanspace 3D) for planning van interiors, and an augmented reality application that we developed. Participants were asked to think aloud as they designed van interiors for fictive journeys using each modality. A qualitative evaluation was conducted to assess how participants’ conceptualizations of space shifted across the use of each prototyping method. The results demonstrate that each design method influenced design outcomes due to the physicality of the task. This study highlights the importance of considering the role of physicality in the selection of prototyping modality for the design exploration process.

For a robot to repair its own error, it must first know it has made a mistake. One way that people detect errors is from the implicit reactions from bystanders - their confusion, smirks, or giggles clue us in that something unexpected occurred. To enable robots to detect and act on bystander responses to task failures, we developed a novel method to elicit bystander responses to human and robot errors. Using 46 different stimulus videos featuring a variety of human and machine task failures, we collected a total of 2,452 webcam videos of human reactions from 54 participants. To test the viability of the collected data, we used the bystander reaction dataset as input to a deep-learning model, BADNet, to predict failure occurrence. We tested different data labeling methods and learned how they affect model performance, achieving precisions above 90%. We discuss strategies (manual labelling, failure-vs-control, and failure-time) used to model bystander reactions and predict failure, and how this approach can be used in real-world robotic deployments to detect errors and improve robot performance. As part of this work, we also contribute with the “Bystander Affect Detection” (BAD) dataset of bystander reactions, supporting the development of better prediction models.

High-fidelity driving simulators can act as testbeds for designing in-vehicle interfaces or validating the safety of novel driver assistance features. In this system paper, we develop and validate the safety of a mixed reality driving simulator system that enables us to superimpose virtual objects and events into the view of participants engaging in real-world driving in unmodified vehicles. To this end, we have validated the mixed reality system for basic driver cockpit and low-speed driving tasks, comparing the use of the system with non-headset and with the headset driving conditions, to ensure that participants behave and perform similarly using this system as they would otherwise. This paper outlines the operational procedures and protocols for using such systems for cockpit tasks (like using the parking brake, reading the instrument panel, and turn signaling) as well as basic low-speed driving exercises (such as steering around corners, weaving around obstacles, and stopping at a fixed line) in ways that are safe, effective, and lead to accurate, repeatable data collection about behavioral responses in real-world driving tasks.

Design has been used to contest existing socio-technical arrangements, provoke conversations around matters of concern, and operationalize radical theories such as agonism, which embraces difference and contention. However, the focus is usually on creating something new: a product, interface or artifact. In this paper, we investigate what happens when critical unmaking is deployed as a deliberate design strategy in an intergenerational, agonistic urban context. Specifically, we report on how youth in a six-week design internship used unmaking as a design move to subvert conventional narratives about their surrounding urban context. We analyze how this led to conflictual encounters at the local senior center, and compare it to the other, making-centric proposals which received favorable feedback but failed to raise the same important discussions. Through this ethnographic account, we argue that critical unmaking is important yet overlooked, and should be in the repertoire of design moves available for agonism and provocation.

We developed a system to explore expressiveness for a robot playing Tic-Tac-Toe against a human. Our robot is based around a pen plotter which performs expressions through the modalities of motion and drawing, aiming to enhance the social engagement of the human-robot interaction.

We present an exploratory human participant study (N=3) examining how people interact with a pen-plotting robot that interactively plays Tic-Tac-Toe on a shared physical sheet of paper. Each participant played a round of Tic-Tac-Toe against the robot, while we observed. We particularly focused our observations on the participants’ physical and social behaviors during game interaction, as well as in-moment reactions from the participants. Following each game, we performed semi-structured qualitative interviews to understand the user’s experience interacting with the robot. Our questions were designed to elicit comparisons of their experience with less tangible interactions that players might have with a computer or phone-based app, as well as more traditional interactions that players might have with other people. We found that participants directly addressed the robot by talking to it during play and openly expressed competitiveness against the robot. Furthermore, participants displayed careful movements around the robot and attentively observed its behaviors. Based on these initial insights from our exploratory study, we are planning future experiments to investigate the effect that the mutuality of the physical Tic-Tac-Toe interaction has on social responses to the robot to understand what this implies for embodied and tangible interaction design.

Most robots are unclothed. However, we believe that robot clothes present an underutilized opportunity for the field of designing interactive systems. Clothes can help robots become better robots––by helping them be useful in a new, wider array of contexts, or better adapt and function in the contexts they are already in. In this paper, we provide a foundation for a research area of robot clothing by speculating on its potential. We systematically present functional requirements of robot clothing, considerations, and parameters for robot clothing designers, as well as key reference cases of robots in clothes. We then discuss what robot clothes can do specifically for the field of designing interactive systems.

One key technique people use in conversation and collaboration is conversational repair. Self-repair is the recognition and attempted correction of one's own mistakes. We investigate how the self-repair of errors by intelligent voice assistants affects user interaction. In a controlled human-participant study (N =101), participants asked Amazon Alexa to perform four tasks, and we manipulated whether Alexa would "make a mistake'' understanding the participant (for example, playing heavy metal in response to a request for relaxing music) and whether Alexa would perform a correction (for example, stating, "You don't seem pleased. Did I get that wrong?'') We measured the impact of self-repair on the participant's perception of the interaction in four conditions: correction (mistakes made and repair performed), undercorrection (mistakes made, no repair performed), overcorrection (no mistakes made, but repair performed), and control (no mistakes made, and no repair performed). Subsequently, we conducted free-response interviews with each participant about their interactions. This study finds that self-repair greatly improves people's assessment of an intelligent voice assistant if a mistake has been made, but can degrade assessment if no correction is needed. However, we find that the positive impact of self-repair in the wake of an error outweighs the negative impact of overcorrection. In addition, participants who recently experienced an error saw increased value in self-repair as a feature, regardless of whether they experienced a repair themselves.

It can be difficult for user researchers to explore how people might interact with interactive systems in everyday contexts; time and space limitations make it hard to be present everywhere that technology is used. Digital music services are one domain where designing for context is important given the myriad places people listen to music. One novel method to help design researchers embed themselves in everyday contexts is through remote-controlled speech agents. This paper describes a practitioner-centered case study of music service interaction researchers using a remote-controlled speech agent, called DJ Bot, to explore people's music interaction in the car and the home. DJ Bot allowed the team to conduct remote user research and contextual inquiry and to quickly explore new interactions. However, challenges using a remote speech-agent arose when adapting DJ Bot from the constrained environment of the car to the unconstrained home environment.

Human-Computer Integration (HInt) is an emerging paradigm in which computational and human systems are closely interwoven. Integrating computers with the human body is not new. however, we believe that with rapid technological advancements, increasing real-world deployments, and growing ethical and societal implications, it is critical to identify an agenda for future research. We present a set of challenges for HInt research, formulated over the course of a five-day workshop consisting of 29 experts who have designed, deployed and studied HInt systems. This agenda aims to guide researchers in a structured way towards a more coordinated and conscientious future of human-computer integration.

A growing number of studies use a "ghost-driver" vehicle driven by a person in a car seat costume to simulate an autonomous vehicle. Using a hidden-driver vehicle in a field study in the Netherlands, Study 1 (N = 130) confirmed that the ghostdriver methodology is valid in Europe and confirmed that European pedestrians change their behavior when encountering a hidden-driver vehicle. As an important extension to past research, we find pedestrian group size is associated with their behavior: groups look longer than singletons when encountering an autonomous vehicle, but look for less time than singletons when encountering a normal vehicle. Study 2 (N = 101) adapted and extended the hidden-driver method to test whether it is believable as online video stimuli and whether car characteristics and participant feelings are related to the beliefs and behavior of pedestrians who see hidden-driver vehicles. As expected, belief rates were lower for hidden-driver vehicles seen in videos compared to in a field study. Importantly, we found noticing no driver was the only significant predictor of belief in car autonomy, which reinforces prior justification for the use of the ghostdriver method. Our contributions are a replication of the hidden-driver method in Europe and comparisons with past US and Mexico data; an extension and evaluation of the ghostdriver method in video form; evidence of the necessity of the hidden driver in creating the illusion of vehicle autonomy; and an extended analysis of how pedestrian group size and feelings relate to pedestrian behavior when encountering a hidden-driver vehicle.

Many producers of automated vehicle systems have begun testing autonomous vehicles on the road. In order to ensure safety and prevent crashes, human drivers are enlisted to monitor autonomous vehicles. However, operators of autonomous systems exhibit negative behavior adaptations in response to prolonged supervision of automation. To prevent the onset of undesirable behaviors in safety drivers, we must investigate driver state and behavior changes during the operation of highly automated vehicles. In the study presented here, we examine the effects of theoretical and practical training on the drivers' response to potentially critical situations in a longitudinal driving simulator study. We also present the effects of encountering a failure of the automated vehicle on driver state and behavior. We conducted a two-part panel driving simulator study (N=28), with an interval of 20-30 days between the training and testing sessions. We found that while participants with training are better prepared for a potential failure of the automation, participants in both conditions show a rise in sleepy or drowsy behavior before a potential failure of automation.

This paper presents a novel and exploratory investigation of how users might control future autonomous vehicles with user-defined in-air static hand gestures. In the era of autonomous vehicles, how to support “driving” without steering control may become a key issue affecting user experience. As the navigation interface of future autonomous cars will be wholly dependent on GPS, and passengers will be unable to make manual control adjustments, verbal or gestural communication will become a primary means to assist them in vehicle control. We thus focus on gesture as an innovative solution for this “final 100 meters” problem in automated navigation. A study (N = 24) conducted in a full chassis simulator shows that hand gesture control is feasible for autonomous vehicle navigation. It further reveals that hand gestures are influenced by task types, local regional norms, and participant culture. In particular, the spatial region where gestures occur can affect execution time, gender can impact user preference and demand, and culture differences and priorities can affect adoption, ease of learning, and user comfort, influencing longer-term use.

To help both designers and people with tetraplegia fully realize the benefts of voice assistant technology, we conducted interviews with fve people with tetraplegia in the home to understand how this population currently uses voice-based interfaces as well as other technologies in their everyday tasks. We found that people with tetraplegia use voice assistants in specifc places, such as in their beds, or when traveling in their wheelchair. In addition, we note the inefciencies for people with tetraplegia when using voice assistance.

Top-down simulations of autonomous intersections neglect considerations for the human experience of being in cars driving through these autonomous intersections. To understand the impact that perspective has on perception of autonomous intersections, we conducted a driving simulator experiment and studied the experience in terms of perception, feelings, and pleasure. Based on this data, we discuss experiential factors of autonomous intersections that are perceived as beneficial or detrimental for the future driver. Furthermore, we present what the change of perspective implies for designing intersection models, future in-car interfaces and simulation techniques.

In fields where in situ performance cannot be measured, ecological validity is difficult to estimate. Drawing on theory from social psychology and virtual reality, we argue that face validity can be a useful proxy for ecological validity. We provide illustrative examples of this relationship from work in search-and-rescue HRI, and conclude with some practical guidelines for the construction of immersive simulations in general.

Recent research suggests that a robot's motors make sounds that can influence users' perception of the robot's characteristics. To more deeply understand users' associations with specific sonic characteristics, we adapted methods from sensory science including Check All That Apply (CATA) questions and Polarized Sensory Positioning (PSP) to tease out small differences in motor sounds in an online survey. These methods are straightforward for untrained people to do in an online setting, mathematically rigorous, and can explore a variety of subtle auditory and perceptual stimuli. We describe how to use these methods, interpret the results with several intuitive visual representations, and show that the results align with a previous study of the same dataset. We close by discussing benefits and limitations of applying these methods to study subtle phenomena in the HCI community.

Advances in automotive sensing systems and speech interfaces provide new opportunities for smarter driving assistants or infotainment systems. For both safety and consumer satisfaction reasons, any new system which interacts with drivers must do so at appropriate times. We asked 63 drivers, ''Is now a good time?'' to receive non-driving information during a 50-minute drive. We analyzed 2,734 responses and synchronized automotive and video data, and show that while the chances of choosing a good time can be determined with better success using easily accessible automotive data, certain nuances in the problem require a richer understanding of the driver and environment states in order to achieve higher performance. We illustrate several of these nuances with quantitative and qualitative analyses to contribute to the understanding of how to design a system that might simultaneously minimize the risk of interacting at a bad time while maximizing the window of allowable interruption.

A Hidden Markov Model framework is introduced to formalize the beliefs that humans may have about the mode in which a semi-automated vehicle is operating. Previous research has identified various “levels of automation,” which serve to clarify the different degrees of a vehicle’s automation capabilities and expected operator involvement. However, a vehicle that is designed to perform at a certain level of automation can actually operate across different modes of automation within its designated level, and its operational mode might also change over time. Confusion can arise when the user fails to understand the mode of automation that is in operation at any given time, and this potential for confusion is not captured in models that simply identify levels of automation. In contrast, the Hidden Markov Model framework provides a systematic and formal specification of mode confusion due to incorrect user beliefs. The framework aligns with theory and practice in various interdisciplinary approaches to the field of vehicle automation. Therefore, it contributes to the principled design and evaluation of automated systems and future transportation systems.

Dominance is a key aspect of interpersonal relationships. To what extent do nonverbal indicators related to dominance status translate to a nonanthropomorphic robot? An experiment ( N = 25) addressed whether a mobile robot's motion style can influence people's perceptions of its status. Using concepts from improv theater literature, we developed two motion styles across three scenarios (robot makes lateral motions, approaches, and departs) to communicate a robot's dominance status through nonverbal expression. In agreement with the literature, participants described a motion style that was fast, in the foreground, and more animated as higher status than a motion style that was slow, in the periphery, and less animated. Participants used fewer negative emotion words to describe the robot with the purportedly high-status movements versus the purportedly low-status movements, but used more negative emotion words to describe the robot when it made departing motions that occurred in the same style. This result provides evidence that guidelines from improvisational theater for using nonverbal expression to perform interpersonal status can be applied to influence perception of a nonanthropomorphic robot's status, thus suggesting that useful models for more complicated behaviors might similarly be derived from performance literature and theory.

We propose a novel method for reliably inducing stress in drivers for the purpose of generating real-world participant data for machine learning, using both scripted in-vehicle stressor events and unscripted on-road stressors such as pedestrians and construction zones. On-road drives took place in a vehicle outfitted with an experimental display that lead drivers to believe they had prematurely ran out of charge on an isolated road. We describe the elicitation method, course design, instrumentation, data collection procedure and the post-hoc labeling of unplanned road events to illustrate how rich data about a variety of stress-related events can be elicited from study participants on-road. We validate this method with data including psychophysiological measurements, video, voice, and GPS data from (N=20) participants. Results from algorithmic psychophysiological stress analysis were validated using participant self-reports. Results of stress elicitation analysis show that our method elicited a stress-state in 89% of participants.

How will pedestrians from different regions interact with an approaching autonomous vehicle? Understanding differences in pedestrian culture and responses can help inform autonomous cars how to behave appropriately in different regional contexts. We conducted a field study comparing the behavioral response of pedestrians between metropolitan Mexico City (N=113) and Colima, a smaller coastal city (N=81). We hid a driver in a car seat costume as a Wizard-of-Oz prototype to evoke pedestrian interaction behavior at a crosswalk or street. Pedestrian interactions were coded for crossing decision, crossing pathway, pacing, and observational behavior. Most distinctly, pedestrians in Mexico City kept their pace and more often crossed in front of the vehicle, while those in Colima stopped in front of the car more often.

Lack of trust can arise when people do not know what autonomous vehicles perceive in the environment. To convey this information without causing alarm or compelling people to act, we designed and evaluated a way to sonify an autonomous vehicle's perception of salient driving events using abstract auditory icons, or "earcons." These are localized in space using an in-car quadraphonic speaker array to correspond with the direction of events. We describe the interaction design for these awareness cues and a validation experiment (N=28) examining the effects of sonified events on drivers' sense of situation awareness, comfort, and trust. Overall, this work suggests that our designed earcons do improve people's awareness of in-simulation events. The effect of the increased situational awareness on trust and comfort is inconclusive. However, post-study design feedback suggests that sounds should have low levels of intensity and dissonance, and a sense of belonging to a common family.

Researchers and designers of in-vehicle interactions and interfaces currently have to choose between performing evaluation and human factors experiments in laboratory driving simulators or on-road experiments. To enjoy the benefit of customizable course design in controlled experiments with the immediacy and rich sensations of on-road driving, we have developed a new method and tools to enable VR driving simulation in a vehicle as it travels on a road. In this paper, we describe how the cost-effective and flexible implementation of this platform allows for rapid prototyping. A preliminary pilot test (N = 6), centered on an autonomous driving scenario, yields promising results, illustrating proof of concept and indicating that a basic implementation of the system can invoke genuine responses from test participants.

Stress affects the lives of millions of people every day. In-situ sensing could enable just-in-time stress management interventions. We present the first work to detect stress using the movements of a car's existing steering wheel. We extend prior work on PC peripherals and demonstrate that stress, expressed through muscle tension in the limbs, can be measured through the way we drive a car. We collected data in a driving simulator under controlled circumstances to vary the levels of induced stress, within subjects. We analyze angular displacement data to estimate coefficients related to muscle tension using an inverse filtering technique. We prove that the damped frequency of a mass spring damper model representing the arm is significantly higher during stress. Stress can be detected with only a few turns during driving. We validate these measures against a known stressor and calibrate our sensor against known stress measurements.

Automotive interaction design is undergoing a major shift due to the disruptive forces of automation and information technology. This paper reviews current challenges in human vehicle interaction design and argues that these challenges demand that interaction become a primary consideration in designing automotive user experiences. We survey exploratory interaction design techniques for human vehicle interactions, showing examples from our research of each technique in action. The techniques are enactments, contextual inquiry, scale scenarios, Wizard of Oz, field experiments and video and animation prototyping. We reflect upon our experiences with these methods, and discuss as yet unmet needs in interaction design for the road ahead.

The advent of automated features in modern vehicles requires human factors researchers to find measures other than driving behavior to anticipate the response of drivers in various contexts. Functional near-infrared spectroscopy (fNIRS) is one research tool that allows us to quantify the driver's mental state. However, the underlying mechanisms of fNIRS technology can limit the possible contexts for its application. The pervasive question arises, whether the measurement device at hand is suitable for the research topic in question and is it capable of detecting the phenomenon under investigation? We provide a proof of concept study demonstrating that significant habituation is present when drivers operate new automated driving systems and that fNIRS technology is suitable to detect said driver habituation effects. The study presented here was conducted in a driving simulator and investigated the drivers' cortical activation in three different modes of automation: manual, partially autonomous, and fully autonomous modes.

To explore cultural differences in driver behavior for the purposes of vehicle automation, we used eye tracking to measure fixation patterns of Japanese and US participants (N = 98) viewing video simulations of automated driving through San Francisco and Osaka. After each drive, we asked participants questions about objects and events from the video.

In partially automated driving, rapid transitions of control present a severe hazard. How long does it take a driver to take back control of the vehicle when engaged with other non-driving tasks? In this driving simulator study, we examined the performance of participants (N=30) after an abrupt loss of automated vehicle control. We tested three transition time conditions, with an unstructured transition of control occurring 2s, 5s, or 8s before entering a curve. As participants were occupied with an active secondary task (playing a game on a tablet) while the automated driving mode was enabled, they needed to disengage from the task and regain control of the car when the transition occurred. Few drivers in the 2 second condition were able to safely negotiate the road hazard situation, while the majority of drivers in the 5 or 8 second conditions were able to navigate the hazard situation safely.

Until vehicles are fully autonomous, safety, legal and ethical obligations require that drivers remain aware of the driving situation. Key decisions about whether a driver can take over when the vehicle is confused, or its capabilities are degraded, depend on understanding whether he or she is responsive and aware of external conditions. The leading techniques for measuring situation awareness in simulated environments are ill-suited to autonomous driving scenarios, and particularly to on-road testing. We have developed a technique, named Daze, to measure situation awareness through real-time, in-situ event alerts. The technique is ecologically valid: it resembles applications people use in actual driving. It is also flexible: it can be used in both simulator and on-road research settings. We performed simulator-based and on-road test deployments to (a) check that Daze could characterize drivers' awareness of their immediate environment and (b) understand practical aspects of the technique's use. Our contributions include the Daze technique, examples of collected data, and ways to analyze such data.

Interaction designers often have difficulty understanding people's real-world experiences with ubiquitous systems. The automobile is a great example of these challenges, where on-road testing is time-consuming and provides little ability for rapid prototyping of interface behavior. We introduce WoZ Way, a system to connect designers to remote drivers. We use live video, audio, car data, and Wizard of Oz speech and interfaces to enable remote observation and interaction prototyping on the road. Our implementation integrates environmental, system level, and social information to make the invisible visible. We tested across three example deployments highlighting usage in interaction prototyping and observational studies. Our findings illustrate how designers explored the situated experiences of people on the road, and how they experimented with different improvisational Wizard of Oz interactions. WoZ Way is both a design research and a design prototyping tool, which can support the work of interaction designers through naturalistic observations, contextual inquiry, and responsive interaction prototyping.

How do sounds shape interaction with robots? The present study explores aural impressions associated with servo motors commonly used to prototype robotic motion. This exploratory analysis constructs a framework to objectively and subjectively characterize sound using acoustic analyses and novice evaluators on Amazon Mechanical Turk. Participants evaluated unfamiliar sounds through pairwise comparison, resulting in subjective ratings of servo motor sounds. In this study, subjective measures of sound correlated well internally, but correlated weakly with objective measures. Moreover, qualitative commentary offered by participants suggests both anthropomorphic associations with sounds as well as negative impressions of the sounds overall. We conclude with a roadmap for exploration into the field of consequential sonic interaction design.

There is a growing need to study the interactions between drivers and their increasingly autonomous vehicles. This paper describes a method of using a low-cost, portable, and versatile driver interaction system in commercial passenger vehicles to enable on-road partial and fully autonomous driving interaction studies. By conducting on-road Wizard-of-Oz studies in naturalistic settings, we can explore a range of driving conditions and scenarios far beyond what can be conducted in laboratory simulator environments. The Marionette system uses off-the-shelf components to cre- ate bidirectional communication between the driving controls of a Wizard-of-Oz vehicle operator and a driving study participant. It signals to the study participant what the car is doing and enables researchers to study participant intervention in driving activity. Mar- ionette is designed to be easily replicated for researchers studying partially autonomous driving interaction. This paper describes the design and evaluation of this system.

Information about drivers' mental states can be vital to the design of interfaces for highly automated vehicles. Functional near infrared spectroscopy (fNIRS) is a neuroimaging tool that is fast becoming popular to study the cortical activity of participants in HCI experiments and driving simulator studies in particular. The analysis methods of the fNIRS data create requirements in the experimental design such as repeated measures. In this paper, we present a study of the event related cortical activity of the drivers of manual, partially autonomous, and fully autonomous cars when performing lane changes using functional near infrared spectroscopic measures. We also present the experimental methodology that was adopted to meet the needs of the fNIRS measurement and the subsequent analysis. The study (N=28) was conducted in a driving simulator. Participants drove for approximately 7 minutes and performed 8 lane change maneuvers in each mode of automation. Multiple streams of data including 4 time-synced video recordings, NASA TLX questionnaires and fNIRS data were recorded and analyzed. It was found that the dorsolateral prefrontal cortex activation during lane changes performed in a partially autonomous mode of operation was just as high as that during a manual lane change, showing that drivers of partially automated systems are as cognitively engaged as drivers of manually operated vehicles.

How could a rearranging chair convince you to let it by? This paper explores how robotic chairs might negotiate passage in shared spaces with people, using motion as an expressive cue. The user study evaluates the efficacy of three gestures at convincing a busy participant to let it by. This within-participants study consisted of three subsequent trials, in which a person is completing a puzzle on a standing desk and a robotic chair approaches to squeeze by. The measure was whether participants moved out of the robot's way or not. People deferred to the robot in slightly less than half the trials as they were engaged in the activity. The main finding, however, is that over-communication cues more blocking behaviors, perhaps because it is annoying or because people want chairs to know their place (socially speaking). The Forward-Back gesture that was most effective at negotiating passage in the first trail was least effective in the second and third trial. The more subtle Pause and the slightly loud but less-aggressive Side-to-Side gesture, were much more likely to be deferred to in later trials, but not a single participant deferred to them in the first trial. The results demonstrate that the Forward-Back gesture was the clearest way to communicate the robot's intent, however, they also give evidence that there is a communicative trade-off between clarity and politeness, particularly when direct communication has an association with aggression. The takeaway for robot design is: be informative initially, but avoid over-communicating later.

How does a robot's sound shape our perception of it? We overlaid sound from high-end and low-end robot arms on videos of the high-end KUKA youBot desktop robotic arm moving a small block in functional (working in isolation) and social (interacting with a human) contexts. The low-end audio was sourced from an inexpensive OWI arm. Crowdsourced participants watched one video each and rated the robot along dimensions of competence, trust, aesthetic, and human-likeness. We found that the presence and quality of sound shapes subjective perception of the KUKA arm. The presence of any sound reduced human-likeness and aesthetic ratings, however the high-end sound rated better in the competence evaluation in the social context measures when compared to no sound. Overall, the social context increased the perceived competence, trust, aesthetic and human-likeness of the robot. Based on motor sound's significant mixed impact on visual perception of robots, we discuss implications for sound design of interactive systems.

In this paper, we introduce two different transforming steering wheel systems that can be utilized to augment user experience for future partially autonomous and fully autonomous vehicles. The first one is a robotic steering wheel that can mechanically transform by using its actuators to move the various components into different positions. The second system is a LED steering wheel that can visually transform by using LEDs embedded along the rim of wheel to change colors. Both steering wheel systems contain onboard microcontrollers developed to interface with our driving simulator. The main function of these two systems is to provide emergency warnings to drivers in a variety of safety critical scenarios, although the design space that we propose for these steering wheel systems also includes the use as interactive user interfaces. To evaluate the effectiveness of the emergency alerts, we conducted a driving simulator study examining the performance of participants (N=56) after an abrupt loss of autonomous vehicle control. Drivers who experienced the robotic steering wheel performed significantly better than those who experienced the LED steering wheel. The results of this study suggest that alerts utilizing mechanical movement are more effective than purely visual warnings.

A large literature describes the use of robots' physical bodies to support communication with people. Touch is a natural channel for physical interaction, yet it is not understood how principles of interpersonal touch might carry over to a robot. Exploring how interpersonal rules surrounding body accessibility and touch apply to a robot is critical toward understanding the extent to which people treat the act of touching body regions as a sign of closeness---even if the body belongs to a robot---and is important to the field of humanoid social robotics. Thirty-one students participated in an interactive anatomy lesson with a small, humanoid robot. Participants either touched or pointed to an anatomical region of the robot in each of 26 trials while their skin conductance response was measured. Touching less accessible regions of a robot's body (e.g., its buttocks and genitals) was more physiologically arousing than touching more accessible regions (e.g., its hands and feet). No differences in physiological arousal were found when just pointing to those same anatomical regions. A social robot elicited tactile responses in human physiology, a result that signals people treat touching body parts as an act of closeness in itself that does not require a human recipient. The power of touching a humanoid body with identifiable body parts should caution mechanical and interaction designers about the positive and negative effects of human-robot interaction.

Automated driving systems that share control with human drivers by using haptic feedback through the steering wheel have been shown to have advantages over fully automated systems and manual driving. Here, we describe an experiment to elicit tacit expectations of behavior from such a system. A gaming steering wheel electronically coupled to the steering wheel in a full-car driving simulator allows two participants to share control of the vehicle. One participant was asked to use the gaming wheel to act as the automated driving agent while another participant acted as the car driver. The course provided different information and visuals to the driving agent and the driver to simulate possible automation failures and conflict situations between automation and the driver. The driving agent was also given prompts that specified a communicative goal at various points along the course. Both participants were interviewed before and after the drive, and vehicle data and drive video were collected. Our results suggest that drivers were able to interpret simple trajectory intentions, such as a lane change, conveyed by the driving agent. However, the driving agent was not able to effectively communicate more nuanced, higher level ideas such as availability, primarily due to the steering wheel being the control mechanism. Torque on the steering wheel without warning was seen most often as a failure of automation. Gentle and steady steering movements were viewed more favorably.

Instructional video content is being created in many different languages. A robot is a generic interface that can deliver translated lecture content in a person's space. In an exploratory study, 40 participants viewed a lecture delivered by a robot lecturer that was either located in front of them or displayed on a screen, in either a real-world environment or while wearing an immersive virtual reality headset. Initial results did not find differences in test performance or in how much participants liked a robot lecturer that was in front of them compared to on a screen, perhaps because the robot spoke the lecture but did not interact with the person.

Previous HRI research has established that trust, disclosure, and a sense of companionship lead to positive outcomes. In this study, we extend existing work by exploring behavioral approaches to increasing these three aspects of HRI. We increased the expressivity and vulnerability of a robot and measured the effects on trust, disclosure, and companionship during human-robot interaction. We engaged (N = 61) high school aged students in a 2 (vulnerability of robot: high vs. low) × 2 (expressivity of robot: high vs. low) between-subjects study where participants engaged in a short electronics learning activity with a robotic tutor. Our results show that students had more trust and feelings of companionship with a vulnerable robot, and reported disclosing more with an expressive robot. Additionally, we found that trust mediated the relationship between vulnerability and companionship. These findings suggest that vulnerability and expressivity may improve peoples' relationships with robots, but that they each have different effects.

Previous HRI research has established that trust, disclosure, and a sense of companionship lead to positive outcomes. We explored improving these aspects of HRI through robot behavior. Specifically, we increased the expressivity and vulnerability of a robot and measured the effects on trust, disclosure, and companionship during human-robot interaction. We engaged (N = 61) high school aged students in a 2 (vulnerability of robot: high vs. low) × 2 (expressivity of robot: high vs. low) between-subjects study where participants engaged in a short electronics learning activity with a robotic tutor. Our results show that students had more trust and feelings of companionship with a vulnerable robot, and reported disclosing more with an expressive robot. This video highlights some of the qualitative interactions students had with the robot as well as a discussion of their experiences around trust, disclosure, and companionship with the robot. The video clips of student interactions show how vulnerability and expressivity can be used to engender trust, disclosure, and companionship between a person and robot.

This tutorial is a hands-on introduction to human-centered design topics and practices for human-robot interaction. It is intended for researchers with a variety of backgrounds, particularly those with little or no prior experience in design. In the morning, participants will learn about user needs and needfinding, as ways to understand the stakeholders in research outcomes, guide the selection of participants, and as possible measures of success. We then focus on design sketching, including ways to represent objects, people and their interactions through storyboards. Design sketching is not intended to be art, rather a way to develop and build upon ideas with oneself, and quickly communicate with colleagues. In the afternoon, participants will use the tools and materials, and learn techniques for lightweight physical prototyping and improvisation. Participants will build a small paper robot (not actuated) of their own design, to practice puppeteering, explore bodily movement and prototype interactions.

Lateral skin stretch is a promising technology for haptic display of information between an autonomous or semi-autonomous car and a driver. We present the design of a steering wheel with an embedded lateral skin stretch display and report on the results of tests (N=10) conducted in a driving vehicle in suburban traffic. Results are generally consistent with previous results utilizing skin stretch in stationary applications, but a slightly higher, and particularly a faster rate of stretch application is preferred for accurate detection of direction and approximate magnitude.

While automated driving systems will become increasingly capable and common in the future, there will still be instances when human drivers want or need to make corrections to the car's automated driving behavior. We conducted two studies exploring how driving interfaces could be designed to better execute the drivers' intentions. In our first study, adult participants (N=40) experienced a simulated driving scenario that varied the behavior of the car's automation (perfect driving and imperfect driving) and the intervention modalities (takeover and takeover+influence). At certain segments, the car's automation would drive perfectly or weave within the lane. During those times, participants could intervene using the available modalities. When experiencing instances of imperfect driving, drivers who had the ability to takeover+influence intervened more often than drivers who were only given the option to takeover. As intervening would require them to resume full control, drivers in the takeover condition were more tolerant of the imperfect driving. Also, most drivers tried to intervene initially by influencing the car, even those drivers who were only given the ability to takeover. In our second study, we examined how participants (N=40) of different demographics (high school students and seniors) would respond when they were subjected to the imperfect driving scenarios. High school drivers intervened just as much as the adult drivers. However, senior drivers intervened far less. These two studies suggest that when intervention is necessary, human drivers have a desire for shared control, which allows them to act as supervisors rather than operators of automated vehicles.

In partially automated cars, it is vital to understand the driver state, especially the driver's cognitive load. This might indicate whether the driver is alert or distracted, and if the car can safely transfer control of driving. In order to better understand the relationship between cognitive load and the driver performance in a partially autonomous vehicle, functional near infrared spectroscopy (fNIRS) measures were employed to study the activation of the prefrontal cortex of drivers in a simulated environment. We studied a total of 14 participants while they drove a partially autonomous car and performed common secondary tasks. We observed that when participants were asked to monitor the driving of an autonomous car they had low cognitive load compared to when the same participants were asked to perform a secondary reading or video watching task on a brought in device. This observation was in line with the increased drowsy behavior observed during intervals of autonomous system monitoring in previous studies. Results demonstrate that fNIRS signals from prefrontal cortex indicate additional cognitive load during manual driving compared to autonomous. Such brain function metrics could be used with minimally intrusive and low cost sensors to enable real-time assessment of driver state in future autonomous vehicles to improve safety and efficacy of transfer of control.

How will pedestrians and bicyclists interact with autonomous vehicles when there is no human driver? In this paper, we outline a novel method for performing observational field experiments to investigate interactions with driverless cars. We provide a proof-of-concept study (N=67), conducted at a crosswalk and a traffic circle, which applies this method. In the study, participants encountered a vehicle that appeared to have no driver, but which in fact was driven by a human confederate hidden inside. We constructed a car seat costume to conceal the driver, who was specially trained to emulate an autonomous system. Data included video recordings and participant responses to post-interaction questionnaires. Pedestrians who encountered the car reported that they saw no driver, yet they managed interactions smoothly, except when the car misbehaved by moving into the crosswalk just as they were about to cross. This method is the first of its kind, and we believe that it contributes a valuable technique for safely acquiring empirical data and insights about driverless vehicle interactions. These insights can then be used to design vehicle behaviors well in advance of the broad deployment of autonomous technology.

In this workshop, we will familiarize participants with the Interaction Engine, a system for prototyping connected, interactive devices using low cost, single-board Linux computers and Arduino microcontrollers. Our main objective is to introduce participants to the basic architecture of connected devices and provide hands-on experience creating networked, physical hardware. The Interaction Engine is a generic framework, not a specialized toolkit. We employ widely available, community-supported tools that can enable web-connected hardware capable of merging tangible interfaces with audio/visual web interfaces. We view low-cost single-board computers as an enabling technology, representing the next step for tangible, embedded, and embodied designs enabling deep interaction between physical and digital worlds. This workshop will be a starting point for participants to begin exploring connected device development and will provide a basic set of tools and skills that participants can use in their own applications.

To design proactive and autonomous interactive objects, designers deal with the design of the object's behavior. In this paper, we propose a design method, called Personality, to help designers develop interactive objects' behaviors with a focus on aesthetics of interaction; the method focuses on tangible and bodily interaction, and it includes four main steps. The "unguided improvisation" step consists of an initial interplay with the interactive object in order to size up the interaction; a brainstorming step, in which we use stereotypes of personalities to create metaphors, to support the discussion around, and the description of, possible behaviors; the "guided improvisation" step iterates over several improvisation sessions to act out interaction scenarios and behaviors; and the behavior synthesis step, in which we provide a final description of the object's behavior. To illustrate Personality we will describe the sofa-bot case study. We will report a lab study, in which we observed people reaction to the different behaviors of the sofa.

This platform paper introduces a methodology for simulating an autonomous vehicle on open public roads. The paper outlines the technology and protocol needed for running these simulations, and describes an instance where the Real Road Autonomous Driving Simulator (RRADS) was used to evaluate 3 prototypes in a between-participant study design. 35 participants were interviewed at length before and after entering the RRADS. Although our study did not use overt deception---the consent form clearly states that a licensed driver is operating the vehicle---the protocol was designed to support suspension of disbelief. Several participants who did not read the consent form clearly strongly believed that they were interacting with a fully autonomous vehicle.

Dialog between drivers and speech-based vehicle interfaces can be used as an instrument to find out what drivers might be concerned, confused or curious about in driving simulator studies. Eliciting on-going conversation with drivers about topics that go beyond navigation, control of entertainment systems, or other traditional driving related tasks is important to getting drivers to engage with the activity in an open-ended fashion. In a structured improvisational Wizard of Oz study that took place in a highly immersive driving simulator, we engaged participant drivers (N=6) in an autonomous driving course where the vehicle spoke to drivers using computer-generated natural language speech. Using microanalyses of the drivers’ responses to the car’s utter- ances, we identify a set of topics that are expected and treated as appropriate by the participants in our study, as well as a set of topics and conversational strategies that are treated as inappropriate. We also show that it is just these unexpected, inappropriate utterances that eventually increase users’ trust in the system, make them more at ease, and raise the system’s acceptability as a communication partner.

How people view relationships between humans and robots is an important consideration for the design and acceptance of social robots. Two studies investigated the effect of relational behavior in a human-robot dyad. In Study 1, participants watched videos of a human confederate discussing the Desert Survival Task with either another human confederate or a humanoid robot. Participants were less trusting of both the robot and the person in a human-robot relationship where the robot was dominant toward the person than when the person was dominant toward the robot; these differences were not found for a human pair. In Study 2, participants watched videos of a human confederate having an everyday conversation with either another human confederate or a humanoid robot. Participants who saw a confederate mirror the gestures of a robot found the robot less attractive than when the robot mirrored the confederate; the opposite effect was found for a human pair. Exploratory findings suggest that human-robot relationships are viewed differently than human dyads.

This paper describes our approach to designing, developing behaviors for, and exploring the use of, a robotic footstool, which we named the mechanical ottoman. By approaching unsuspecting participants and attempting to get them to place their feet on the footstool, and then later attempting to break the engagement and get people to take their feet down, we sought to understand whether and how motion can be used by non-anthropomorphic robots to engage people in joint action. In several embodied design improvisation sessions, we observed a tension between people perceiving the ottoman as a living being, such as a pet, and simultaneously as a functional object, which requests that they place their feet on it-something they would not ordinarily do with a pet. In a follow-up lab study (N=20), we found that most participants did make use of the footstool, although several chose not to place their feet on it for this reason. We also found that participants who rested their feet understood a brief lift and drop movement as a request to withdraw, and formed detailed notions about the footstool's agenda, ascribing intentions based on its movement alone.

With automated driving systems, drivers may still be expected to resume full control of the vehicle. While structured transitions where drivers are given warning are desirable, it is critical to benchmark how drivers perform when transition of control is unstructured and occurs without advanced warning. In this study, we observed how participants (N=27) in a driving simulator performed after they were subjected to an emergency loss of automation. We tested three transition time conditions, with an unstructured transition of vehicle control occurring 2 seconds, 5 seconds, or 8 seconds before the participants encountered a road hazard that required the drivers' intervention. Few drivers in the 2 second condition were able to safely negotiate the road hazard situation, while the majority of drivers in 5 or 8 second conditions were able to navigate the hazard safely. Similarly, drivers in 2 second condition rated the vehicle to be less likeable than drivers in 5 and 8 second conditions. From the study results, we are able to narrow in on a minimum amount of time in which drivers can take over the control of vehicle safely and comfortably from the automated system in the advent of an impending road hazard.

In this study, we examined how participants (N=20) interacted and collaborated with a set of robotic drawers to accomplish an assembly task. The drawers' behavior varied along two dimensions - proactivity and expressivity of motions. The results of our study indicate that participants consider an expressive robot to be more involved and interested in the interaction. We also found that while proactive or expressive robots could dominate the interaction, proactivity might negatively affect the participants' perception of their social status relative to that of the robot's, while expressiveness did not. This shows the importance of utilizing expressive movements when designing socially appropriate robots that collaborate with human users.

Service robots in public places need to both understand environmental cues and move in ways that people can understand and predict. We developed and tested interactions with a trash barrel robot to better understand the implicit protocols for public interaction. In eight lunch-time sessions spread across two crowded campus dining destinations, we experimented with piloting our robot in Wizard of Oz fashion, initiating and responding to requests for impromptu interactions centered on collecting people's trash. Our studies progressed from open-ended experimentation to testing specific interaction strategies that seemed to evoke clear engagement and responses, both positive and negative. Observations and interviews show that a) people most welcome the robot's presence when they need its services and it actively advertises its intent through movement; b) people create mental models of the trash barrel as having intentions and desires; c) mistakes in navigation are indicators of autonomous control, rather than a remote operator; and d) repeated mistakes and struggling behavior polarized responses as either ignoring or endearing.

To investigate how children interact differently when interactions are mediated with screen-based video communication versus a robot-mediated communication, we conducted a study with elementary students in Korea, comparing the use of both technologies to introduce classroom students with peer-aged individuals in America. Our findings show that the classroom children showed more positive emotion during certain tasks and exhibited more interest to remote participants in the context of robot-mediated communication than with video-mediated communication.

The role of human-robot interaction is becoming more important as everyday robotic devices begin to permeate into our lives. In this study, we video-prototyped a user's interactions with a set of robotic drawers. The user and robot each displayed one of five emotional states - angry, happy, indifferent, sad, and timid. The results of our study indicated that the participants of our online questionnaire preferred empathetic drawers to neutral ones. They disliked robotic drawers that displayed emotions orthogonal to the user's emotions. This showed the importance of displaying emotions, and empathy in particular, when designing robotic devices that share our living and working spaces.

What effect will periods of automated driving will have on driver performance after transfer of control? In our driving simulator experiment (N = 48) participants in four different automation conditions (fully autonomous vehicle, autonomous steering, autonomous speed control, no automation) were evaluated based on their post-transition accident avoidance, situational awareness, and feelings of trust in and comfort with autonomous or partially autonomous driving. Preliminary results from behavioral data show significant differences in time to initiate evasive action across conditions. Participants in the fully autonomous condition showed greater trust and comfort with the car's autonomous features than those in the autonomous speed control condition.

This paper makes the case for designing interactive robots with their expressive movement in mind. As people are highly sensitive to physical movement and spatiotemporal affordances, well-designed robot motion can communicate, engage, and offer dynamic possibilities beyond the machines' surface appearance or pragmatic motion paths. We present techniques for movement centric design, including character animation sketches, video prototyping, interactive movement explorations, Wizard of Oz studies, and skeletal prototypes. To illustrate our design approach, we discuss four case studies: a social head for a robotic musician, a robotic speaker dock listening companion, a desktop telepresence robot, and a service robot performing assistive and communicative tasks. We then relate our approach to the design of non-anthropomorphic robots and robotic objects, a design strategy that could facilitate the feasibility of real-world human-robot interaction.

This paper describes a series of research probe games developed to investigate how real-world activity could be incorporated into digital game systems. These culminated in the design of our final game, Forest, which was conceived for the San Francisco non-profit Friends of the Urban Forest (FUF), who have been planting and caring for the city's street trees for 30 years. By incorporating real-world actions and behaviors into digital games, we can create experiences that both enhance our understanding of the world around us and provide incentive structures towards our personal, community, or societal goals.

This paper describes a cooperative design project to develop ways to use Nintendo Wii Remotes as inexpensive data acquisition tools for science. In collaboration with a 6th grade physics instructor and his students, we have developed software tools and curriculum that enable science teachers and students to repurpose gaming technologies to study concepts such as velocity and acceleration. The project involved a year's observation of students' project based learning in a 6th grade physics class, followed by a year of design experimentation to engage students in integrating game controllers into their projects. Using the insights from their observations and suggestions, we created three different Wii Remote-based setups that used the IR camera and the accelerometer to help students glean data from their projects. In this paper, we provide an overview of the project, and then offer data that demonstrates the added value across the material, social, experiential, and temporal aspects of inquiry science activity. We conclude by identifying key design opportunities within this space.

Does augmented movement capability improve people's experiences with telepresent meeting participants? We performed two web-based studies featuring videos of a telepresence robot. In the first study (N=164), participants observed clips of typical conversational gestures performed a) on a stationary screen only, b) with an actuated screen moving in physical space, or c) both on-screen and in-space. In the second study (N=103), participants viewed scenario videos depicting two people interacting with a remote collaborator through a telepresence robot, whose distant actions were a) visible on the screen only, or b) accompanied by local physical motion. These studies suggest that synchronized on-screen and in-space gestures significantly improved viewers' interpretation of the action compared to on-screen or in-space gestures alone, and that in-space gestures positively influenced perceptions of both local and remote participants.

This paper describes a digital learning tool that engages math teachers and students with geometry through physical movement, tangible controls, and computer visualization. It was developed through iterative prototype testing in actual grade 6--10 math classes. Students actively create geometries using their own movements which are captured by a simple web camera. A tangible interface allows them to transform the captured images and create complex patterns through mathematical relationships. We evaluated the collaborative and kinesthetic participation promoted by the tool and the way teachers and students reacted to using it. Pattern Poses was deployed at a learning workshop with a group of 35 students of grade 5 and grade 8 and their parents in the San Francisco Bay Area.

The animation techniques of anticipation and reaction can help create robot behaviors that are human readable such that people can figure out what the robot is doing, reasonably predict what the robot will do next, and ultimately interact with the robot in an effective way. By showing forethought before action and expressing a reaction to the task outcome (success or failure), we prototyped a set of human-robot interaction behaviors. In a 2 (forethought vs. none: between) x 2 (reaction to outcome vs. none: between) x 2 (success vs. failure task outcome: within) experiment, we tested the influences of forethought and reaction upon people's perceptions of the robot and the robot's readability. In this online video prototype experiment (N=273), we have found support for the hypothesis that perceptions of robots are influenced by robots showing forethought, the task outcome (success or failure), and showing goal-oriented reactions to those task outcomes. Implications for theory and design are discussed.

This study builds upon previous work regarding people's attitudes toward robot workers, identifying the characteristics of occupations for which people believe robots are qualified and desired. This research updates prior research and adds a new dimension of respondent expertise in the domain of robotics (N=392, which includes 134 robotics experts and 258 non-experts). We deployed a web-based survey that asked respondents about their attitudes toward robots' suitability for a variety of jobs (n=812) from the U.S. Department of Labor's O*NET occupational information database. There were different responses from experts and non-experts about what types of jobs robots: (a) could, but should not do and (b) should, but could not do. Implications for the robotics community are discussed.

This paper describes a series of interaction design sketches we created to supplement mathematics curricula. These sketches were deployed in a variety of secondary school math classrooms in the San Francisco Bay Area. The activities purposefully use visualization and embodiment to engage students with the math concepts of geometric transformations and symmetrical patterning. These experiments exemplify how applying embodiment and visualization to traditionally impersonal and abstract subjects like math can make the learning experience more fun and active for students and offer new pedagogical strategies for teachers.

Data visualization, commonly used to make large sets of numerical data more legible, also has enormous potential as a storytelling tool to elicit insights on long-standing social problems. It can help to synthesize diverse personal narratives about history, causes and impacts, and thereby give a voice to populations seeking to create change.In this work, we explore the potential for using data visualization as a vehicle for social change through creative engagement. Our intent is to design and deploy an interactive visualization of development in the Dominican Republic which brings empathy to the society's cultural psychology, helps frame limitations and challenges, and highlights opportunities for progress. Some of the major challenges in designing this work lie in layering both the big picture perspective - historical events and statistical trends - with personal narratives - vivid stories that illuminate the current state of the society. We discuss how this work can foster conversations and promote creative thought, motivating actions that can transform the current state of the country.

An important challenge in designing ubiquitous computing experiences is negotiating transitions between explicit and implicit interaction, such as how and when to provide users with notifications. While the paradigm of implicit interaction has important benefits, it is also susceptible to difficulties with hidden modes, unexpected action, and misunderstood intent. To address these issues, this work presents a framework for implicit interaction and applies it to the design of an interactive whiteboard application called Range. Range is a public interactive whiteboard designed to support co-located, ad-hoc meetings. It employs proximity sensing capability to proactively transition between display and authoring modes, to clear space for writing, and to cluster ink strokes. We show how the implicit interaction techniques of user reflection (how systems indicate to users what they perceive or infer), system demonstration (how systems indicate what they are doing), and override (how users can interrupt or stop a proactive system action) can prevent, mitigate, and correct errors in the whiteboard's proactive behaviors. These techniques can be generalized to improve the designs of a wide array of ubiquitous computing experiences.

We present a study of people's attitudes toward robot workers, identifying the characteristics of occupations for which people believe robots are qualified and desired. We deployed a web-based public-opinion survey that asked respondents (n=250) about their attitudes regarding robots' suitability for a variety of jobs (n=812) from the U.S. Department of Labor's O*NET occupational information database. We found that public opinion favors robots for jobs that require memorization, keen perceptual abilities, and service-orientation. People are preferred for occupations that require artistry, evaluation, judgment and diplomacy. In addition, we found that people will feel more positively toward robots doing jobs with people rather than in place of people.

We have built and tested WorkspaceNavigator, which supports knowledge capture and reuse for teams engaged in unstructured, dispersed, and prolonged collaborative design activity in a dedicated physical workspace. It provides a coherent unified interface for post-facto retrieval of multiple streams of data from the work environment, including overview snapshots of the workspace, screenshots of in-space computers, whiteboard images, and digital photos of physical objects. This paper describes the design of WorkspaceNavigator and identifies key considerations for knowledge capture tools for design workspaces, which differ from those of more structured meeting or classroom environments. Iterative field tests in workspace environments for student teams in two graduate Mechanical Engineering design courses helped to identify features that augment the work of both course participants and design researchers.