As AI advances, and can draw upon previously disparate information sources, automated vehicles are gaining the ability to proactively collaborate with humans to accomplish everyday tasks. This collaboration will be based on increased sensing and information and will feature greater proactive behavior by the vehicle. However, there is little guidance on how vehicles should behave proactively, or even a shared understanding of what that means. Using prior work, we develop a framework of proactive interaction comprising three dimensions: initiation of action, alignment with users’ goals, and communication of context awareness. We apply the framework in a video-based online study (N=351) of a vehicle navigation system that exhibits proactive behavior. The system advises or acts, supports or counters users’ goals in two scenarios (to reach a work meeting quickly or carry groceries without walking too far), and provides a more or less detailed rationale. We find that when goals are misaligned, communicating contextual awareness decreases user experience and acceptance. As a result, if alignment of (immediate) goals is not possible—such as when drivers want to reduce walking distance but the closest parking lot is full—systems should exercise care in communicating their situation awareness, as this can reinforce users’ negative perceptions. Dispositional traits such as propensity to trust and locus of control do predict users’ trust, distrust, and experience in our scenarios. We also find that participants’ primary rationales for accepting the system’s decision are largely similar (saving time, trusting, and making sense) but rationales overall, especially for rejecting the system’s decision, are more varied than expected and often based on experience. Therefore, systems that look to personalize decision-making—such as whether to proactively execute versus merely suggest an action—could take into account users’ dispositional traits and other preferences, such as those as exemplified by these rationales.

As the navigation interface of fully autonomous vehicles (FAV) will be wholly dependent on On-board Navigation Systems (ONS), it is foreseeable that ONS will replace steering wheels as the primary control mode of future FAVs. Thus, the ease of setting precise destinations using ONS may become a key factor affecting FAV user acceptance. A survey (N = 200) comprising Navigation Anxiety Questionnaire and User Preference Questionnaire reveals the user’s acceptable measurements of distance error and execution time when navigating FAVs in six common situations. Subsequently, an in-vehicle simulation test (N = 24) found most participants in the simulation test performed worse than the user’s acceptable measurements obtained through the questionnaire survey, which indicated that the existing ONS failed to meet users’ expectations for the FAV navigation experience. Findings of the questionnaire survey and the simulation test reveal a severe “last-100 meters” issue when controlling FAVs by ONS, and that destination familiarity, gender difference, text input behavior, map scale, and map display mode should be seriously considered when designing ONS for FAV control.

Research has characterized mind-wandering as humans’ natural mental state, with moments of task-focused attention being the exception. With this framing, mind-wandering while driving likely occurs more than generally acknowledged, and seems poised to increase with higher levels of automation. This in turn may have adverse effects on drivers’ abilities to regain situation awareness or resume control when needed. Of the prior work on detecting mind-wandering while driving, none focuses on automation or complex urban environments. We ran an exploratory study (N = 14) of an automated drive through New York City in a two-dimensional virtual reality context, focusing on physiological measures such as gaze distribution, pupillometry, and heart rate. We also explored how drivers missing critical events may be a potential new measure. Results varied between focused and mind-wandering mental states and between moving and stopped driving contexts. These observations are an initial step toward understanding mind-wandering across diverse driving scenarios.

Conditionally automated driving systems face two main safety challenges: the inability to autonomously handle all situations the vehicle encounters, and the allowed inattention of drivers during these critical moments. Our study focuses on enhancing drivers’ situation awareness at such times by embedding information about system status and the road environment in the visual signals displayed when control is transferred from the automated driving system. Six visual signals, each including different levels of situation awareness information, were compared to examine how they influence drivers’ levels of situation awareness in a simulated environment. The results show that signals incorporating higher levels of situation awareness information about the environment significantly facilitate the recovery of situation awareness after engaging in non-driving related tasks. This research provides insights into how visual cues can be optimized to facilitate quicker recovery of situation awareness for drivers transitioning from non-driving tasks in conditionally automated vehicles.

The Development of Fully Autonomous Vehicles (AVs) would fundamentally change the nature of in-vehicle user interactions, behaviors, needs, and activities. Passengers free from driving would expect to undertake diverse Non-Driving-Related Tasks to keep themselves occupied. Introducing Conversational Artificial Intelligence (CAI) in Level 5 AVs could improve the in-vehicle user experience (UX). To explore this, firstly, we identify what roles and relationships can CAI play towards end-users of AVs through end-user interviews and thematic analysis. Secondly, we examine how end-users qualitatively assess the embodied UX of the CAI roles and relationships through guided brainstorming, post simulator interaction experiments employing Wizard of Oz setup and Participant Enactment methods. Results show that Tour Guide, Mentor, and Storyteller were the most preferred CAI roles, and that Human-CAI relationships are maintained if the CAI mediates in-vehicle user activities, interactions, sharing of vehicle control, and deep conversations. We discuss the research implications and propose design guidelines.

On-demand mental health services-including counseling, crisis hotlines, and peer support programs-are vital to the healthcare system, providing acute and ongoing support through telephone, online chats, and text messaging. Although such services have proven effective at reducing hopelessness, psychological pain, and suicidality, they put the providers of these services at high risk of burnout, secondary traumatic stress, and compassion fatigue. Our interviews with professionals from four mental health organizations revealed that while these workers have a strong motivation to help clients with mental health care needs, they face various challenges themselves, particularly regarding heavy caseloads, difficult crisis clients, and coping with repeated exposure to abuse and harassment. To overcome challenges, participants identify the need to be self-reliant and engage in self-care practices ranging from socializing with coworkers to yoga and meditation. Although organizations spend significant time training workers prior to their involvement with clients, the training typically lacks components on self-compassion and self-care. Designers might see technology as an opportunity to promote such practices; however, while technology is an integral part of their work routine, participants, irrespective of age, had misapprehensions regarding technology use in the mental health care space, including for managing their psychological well-being. We recommend design guidelines for HCI researchers, including developing contextualized just-in-time adaptive interventions to promote self-compassion and educating workers regarding the use of various technologies to manage their well-being.

Modern vehicles are using AI and increasingly sophisticated sensor suites to improve Advanced Driving Assistance Systems (ADAS) and support automated driving capabilities. Heads-Up-Displays (HUDs) provide an opportunity to visually inform drivers about vehicle perception and interpretation of the driving environment. One approach to HUD design may be to reveal to drivers the vehicle’s full contextual understanding, though it is not clear if the benefits of additional information outweigh the drawbacks of added complexity, or if this balance holds across drivers. We designed and tested an Augmented Reality (AR) HUD in an online study (N = 298), focusing on the influence of HUD visualizations on drivers’ situation awareness and perceptions. Participants viewed two driving scenes with one of three HUD conditions. Results were nuanced: situation awareness declined with increasing driving context complexity, and contrary to expectation, also declined with the presence of a HUD compared to no HUD. Significant differences were found by varying HUD complexity, which led us to explore different characterizations of complexity, including counts of scene items, item categories, and illuminated pixels. Our analysis finds that driving style interacts with driving context and HUD complexity, warranting further study.

Electric vehicles’ (EVs) nearly silent operation has proved to be dangerous for bicyclists and pedestrians, who often use an internal combustion engine’s sound as one of many signals to locate nearby vehicles and predict their behavior. Inspired by regulations currently being implemented that will require EVs and hybrid vehicles (HVs) to play synthetic sound, we used a Wizard-of-Oz AV setup to explore how adding synthetic engine sound to a hybrid autonomous vehicle (AV) will influence how pedestrians interact with the AV in a naturalistic field study. Pedestrians reported increased interaction quality and clarity of intent of the vehicle to yield compared to a baseline condition without any added sound. These findings suggest that synthetic engine sound will not only be effective at helping pedestrians to hear EVs, but also may help AV developers implicitly signal to pedestrians when the vehicle will yield.

Autonomous vehicle system performance is limited by uncertainties inherent in the driving environment and challenges in processing sensor data. Engineers thus face the design decision of biasing systems toward lower sensitivity to potential threats (more misses) or higher sensitivity (more false alarms). We explored this problem for Automatic Emergency Braking systems in Level 3 autonomous vehicles, where the driver is required to monitor the system for failures. Participants (N=48) drove through a simulated suburban environment and experienced detection misses, perfect performance, or false alarms. We found that driver vigilance was greater for less-sensitive braking systems, resulting in improved performance during a potentially fatal failure. In addition, regardless of system bias, greater levels of autonomy resulted in significantly worse driver performance. Our results demonstrate that accounting for the effects of system bias on driver vigilance and performance will be critical design considerations as vehicle autonomy levels increase.

Autonomous vehicle (AV) systems are developing at a rapid pace, not only in technological capabilities, but also in human-centered directions. Despite this development, we lack a nuanced understanding of driver preference in decision scenarios that semi-AVs will face, and of possible misalignment between semi-AV decisions and user preference. Using an online survey, we explore how participants would like semi-AVs to act and alert them of the vehicles' decisions in various scenarios. Participants reported varying levels of comfort with autonomy, desire to takeover control, and desire for AV informing. Individual differences, including level of experience with autonomy and situation awareness, affected perceptions of the vehicle. Our results highlight the importance of considering driver preference in AV decision-making, and we present an influence diagram that situates this factor among others. We also derive five design principles, including that a previous positive AV experience can lead to more harmful consequences for AVs when not aligned with driver preference.

As autonomous vehicles (AVs) become a reality on public roads, researchers and designers are beginning to see unexpected reactions from the public ranging from curiosity to vandalism. These behaviors are concerning, as AV platforms will need to know how to deal with people behaving unexpectedly or aggressively. We call this griefing of AVs, adopting the term from harassment in online gaming. We discuss several examples of griefing observed in on-road field studies using a Wizard-of-Oz driverless car. While Uber and Waymo have anecdotally mentioned vandalism towards AVs, we believe this to be the first public video available of AV griefing ranging from playful to aggressive. To stimulate discussion, we propose speculative design principles to address griefing.

A growing number of studies use a "ghost-driver" vehicle driven by a person in a car seat costume to simulate an autonomous vehicle. Using a hidden-driver vehicle in a field study in the Netherlands, Study 1 (N = 130) confirmed that the ghostdriver methodology is valid in Europe and confirmed that European pedestrians change their behavior when encountering a hidden-driver vehicle. As an important extension to past research, we find pedestrian group size is associated with their behavior: groups look longer than singletons when encountering an autonomous vehicle, but look for less time than singletons when encountering a normal vehicle. Study 2 (N = 101) adapted and extended the hidden-driver method to test whether it is believable as online video stimuli and whether car characteristics and participant feelings are related to the beliefs and behavior of pedestrians who see hidden-driver vehicles. As expected, belief rates were lower for hidden-driver vehicles seen in videos compared to in a field study. Importantly, we found noticing no driver was the only significant predictor of belief in car autonomy, which reinforces prior justification for the use of the ghostdriver method. Our contributions are a replication of the hidden-driver method in Europe and comparisons with past US and Mexico data; an extension and evaluation of the ghostdriver method in video form; evidence of the necessity of the hidden driver in creating the illusion of vehicle autonomy; and an extended analysis of how pedestrian group size and feelings relate to pedestrian behavior when encountering a hidden-driver vehicle.

Autonomous vehicle systems and their users need to collaborate to navigate the driving environment, particularly during an unstructured transition of control from automation, when the system releases control and expects the human to immediately assume driving responsibility. We investigated how such transitions affect the user's trust in the system and subsequent performance. In a full-vehicle driving simulator, participants encountered two system failures: the first varied in severity (mild or severe failure), and the second required a transition of control that was either detected and alerted (loud failure) or not (silent failure). We observed (i) significant changes in user trust in the system over time and between events, and (ii) that the first failure's severity level did not affect user performance in the subsequent failure; rather, the system's detection and alert both times was sufficient to successfully complete the transition of control.

This paper presents a novel and exploratory investigation of how users might control future autonomous vehicles with user-defined in-air static hand gestures. In the era of autonomous vehicles, how to support “driving” without steering control may become a key issue affecting user experience. As the navigation interface of future autonomous cars will be wholly dependent on GPS, and passengers will be unable to make manual control adjustments, verbal or gestural communication will become a primary means to assist them in vehicle control. We thus focus on gesture as an innovative solution for this “final 100 meters” problem in automated navigation. A study (N = 24) conducted in a full chassis simulator shows that hand gesture control is feasible for autonomous vehicle navigation. It further reveals that hand gestures are influenced by task types, local regional norms, and participant culture. In particular, the spatial region where gestures occur can affect execution time, gender can impact user preference and demand, and culture differences and priorities can affect adoption, ease of learning, and user comfort, influencing longer-term use.

Autonomous vehicles' (AVs) interactions with pedestrians remain an ongoing uncertainty. Several studies have claimed the need for explicit external human-machine interfaces (eHMI) such as lights or displays to replace the lack of eye contact with and explicit gestures from drivers, however this need is not thoroughly understood. We review literature on explicit and implicit eHMI, and discuss results from a field study with a Wizard-of-Oz driverless vehicle that tested pedestrians' reactions in everyday traffic without explicit eHMI. While some pedestrians were surprised by the vehicle, others did not notice its autonomous nature, and all crossed in front without explicit signaling, suggesting that pedestrians may not need explicit eHMI in routine interactions---the car's implicit eHMI (its motion) may suffice.

HRI researchers have explored how people behave toward technology agents, advancing the concept that people can attain “closeness” with technology itself in addition to a living social partner. Yet the topic of closeness with robots has not been fully explored or organized into a discrete area of study. This seems particularly important to the design of robots that are expressive, to the implementation of technologies that use new social signal processing or reciprocal social touch, and to the study of how people respond to robots. This half-day workshop is a forum to discuss the future of “closeness” with robots, conversational agents, autonomous vehicles, Internet of Things devices and other technologies that act as social partners-designs, applications, responses and societal concerns.

Automated emergency braking (AEB) systems-which alert a driver to approaching hazards and automatically brake-are currently available in some vehicles and will soon be widespread. Due to the uncertainties inherent in any environment and difficulties in processing sensor data, these systems are prone to both false alarms and system misses of hazards. A pressing design concern is whether to bias these systems toward a higher likelihood of false alarms versus system misses for non-fatal events. We investigated how drivers form mental models of the AEB system and how that influences their reliance on the system during a critical, potentially fatal, failure. In a full vehicle driving simulator, participants experienced nine interactions that reflected the system's level of bias toward false alarms or toward misses for non-fatal events or that demonstrated perfect performance. When a potentially fatal event occurred, participants trained to expect misses were better able to avoid a pedestrian in the road after a detection failure than those using a system with perfect performance or false alarms. These findings suggest that systems biased toward misses in non-fatal events encourage driver vigilance and preparedness for potentially fatal events.

Until vehicles are fully autonomous, safety, legal and ethical obligations require that drivers remain aware of the driving situation. Key decisions about whether a driver can take over when the vehicle is confused, or its capabilities are degraded, depend on understanding whether he or she is responsive and aware of external conditions. The leading techniques for measuring situation awareness in simulated environments are ill-suited to autonomous driving scenarios, and particularly to on-road testing. We have developed a technique, named Daze, to measure situation awareness through real-time, in-situ event alerts. The technique is ecologically valid: it resembles applications people use in actual driving. It is also flexible: it can be used in both simulator and on-road research settings. We performed simulator-based and on-road test deployments to (a) check that Daze could characterize drivers' awareness of their immediate environment and (b) understand practical aspects of the technique's use. Our contributions include the Daze technique, examples of collected data, and ways to analyze such data.

Automated driving systems that share control with human drivers by using haptic feedback through the steering wheel have been shown to have advantages over fully automated systems and manual driving. Here, we describe an experiment to elicit tacit expectations of behavior from such a system. A gaming steering wheel electronically coupled to the steering wheel in a full-car driving simulator allows two participants to share control of the vehicle. One participant was asked to use the gaming wheel to act as the automated driving agent while another participant acted as the car driver. The course provided different information and visuals to the driving agent and the driver to simulate possible automation failures and conflict situations between automation and the driver. The driving agent was also given prompts that specified a communicative goal at various points along the course. Both participants were interviewed before and after the drive, and vehicle data and drive video were collected. Our results suggest that drivers were able to interpret simple trajectory intentions, such as a lane change, conveyed by the driving agent. However, the driving agent was not able to effectively communicate more nuanced, higher level ideas such as availability, primarily due to the steering wheel being the control mechanism. Torque on the steering wheel without warning was seen most often as a failure of automation. Gentle and steady steering movements were viewed more favorably.

This tutorial is a hands-on introduction to human-centered design topics and practices for human-robot interaction. It is intended for researchers with a variety of backgrounds, particularly those with little or no prior experience in design. In the morning, participants will learn about user needs and needfinding, as ways to understand the stakeholders in research outcomes, guide the selection of participants, and as possible measures of success. We then focus on design sketching, including ways to represent objects, people and their interactions through storyboards. Design sketching is not intended to be art, rather a way to develop and build upon ideas with oneself, and quickly communicate with colleagues. In the afternoon, participants will use the tools and materials, and learn techniques for lightweight physical prototyping and improvisation. Participants will build a small paper robot (not actuated) of their own design, to practice puppeteering, explore bodily movement and prototype interactions.

In partially automated cars, it is vital to understand the driver state, especially the driver's cognitive load. This might indicate whether the driver is alert or distracted, and if the car can safely transfer control of driving. In order to better understand the relationship between cognitive load and the driver performance in a partially autonomous vehicle, functional near infrared spectroscopy (fNIRS) measures were employed to study the activation of the prefrontal cortex of drivers in a simulated environment. We studied a total of 14 participants while they drove a partially autonomous car and performed common secondary tasks. We observed that when participants were asked to monitor the driving of an autonomous car they had low cognitive load compared to when the same participants were asked to perform a secondary reading or video watching task on a brought in device. This observation was in line with the increased drowsy behavior observed during intervals of autonomous system monitoring in previous studies. Results demonstrate that fNIRS signals from prefrontal cortex indicate additional cognitive load during manual driving compared to autonomous. Such brain function metrics could be used with minimally intrusive and low cost sensors to enable real-time assessment of driver state in future autonomous vehicles to improve safety and efficacy of transfer of control.

How will pedestrians and bicyclists interact with autonomous vehicles when there is no human driver? In this paper, we outline a novel method for performing observational field experiments to investigate interactions with driverless cars. We provide a proof-of-concept study (N=67), conducted at a crosswalk and a traffic circle, which applies this method. In the study, participants encountered a vehicle that appeared to have no driver, but which in fact was driven by a human confederate hidden inside. We constructed a car seat costume to conceal the driver, who was specially trained to emulate an autonomous system. Data included video recordings and participant responses to post-interaction questionnaires. Pedestrians who encountered the car reported that they saw no driver, yet they managed interactions smoothly, except when the car misbehaved by moving into the crosswalk just as they were about to cross. This method is the first of its kind, and we believe that it contributes a valuable technique for safely acquiring empirical data and insights about driverless vehicle interactions. These insights can then be used to design vehicle behaviors well in advance of the broad deployment of autonomous technology.

To design proactive and autonomous interactive objects, designers deal with the design of the object's behavior. In this paper, we propose a design method, called Personality, to help designers develop interactive objects' behaviors with a focus on aesthetics of interaction; the method focuses on tangible and bodily interaction, and it includes four main steps. The "unguided improvisation" step consists of an initial interplay with the interactive object in order to size up the interaction; a brainstorming step, in which we use stereotypes of personalities to create metaphors, to support the discussion around, and the description of, possible behaviors; the "guided improvisation" step iterates over several improvisation sessions to act out interaction scenarios and behaviors; and the behavior synthesis step, in which we provide a final description of the object's behavior. To illustrate Personality we will describe the sofa-bot case study. We will report a lab study, in which we observed people reaction to the different behaviors of the sofa.

We explore the impact of the displayed precision of instrumentation estimates of range and state-of-charge on drivers' attitudes towards an all-electric vehicle (EV), on their driving experience, and driving behavior under varying conditions of resource availability. Participants (N=73) completed a 19-mile long drive through highway, rural town and mountain road conditions in an EV that displayed high vs. low remaining range, and gave estimates of that range with high and low information ambiguity. We found that an ambiguous display of range preserved drivers' feelings of trust towards the vehicle, despite encountering situations intended to induce severe range anxiety. Furthermore, compared to drivers facing an unambiguous display of range, drivers presented with an ambiguous range display reported improved driving experience, and exhibited driving behavior better adapted to road and remaining range conditions.

Dialog between drivers and speech-based vehicle interfaces can be used as an instrument to find out what drivers might be concerned, confused or curious about in driving simulator studies. Eliciting on-going conversation with drivers about topics that go beyond navigation, control of entertainment systems, or other traditional driving related tasks is important to getting drivers to engage with the activity in an open-ended fashion. In a structured improvisational Wizard of Oz study that took place in a highly immersive driving simulator, we engaged participant drivers (N=6) in an autonomous driving course where the vehicle spoke to drivers using computer-generated natural language speech. Using microanalyses of the drivers’ responses to the car’s utter- ances, we identify a set of topics that are expected and treated as appropriate by the participants in our study, as well as a set of topics and conversational strategies that are treated as inappropriate. We also show that it is just these unexpected, inappropriate utterances that eventually increase users’ trust in the system, make them more at ease, and raise the system’s acceptability as a communication partner.

This paper describes our approach to designing, developing behaviors for, and exploring the use of, a robotic footstool, which we named the mechanical ottoman. By approaching unsuspecting participants and attempting to get them to place their feet on the footstool, and then later attempting to break the engagement and get people to take their feet down, we sought to understand whether and how motion can be used by non-anthropomorphic robots to engage people in joint action. In several embodied design improvisation sessions, we observed a tension between people perceiving the ottoman as a living being, such as a pet, and simultaneously as a functional object, which requests that they place their feet on it-something they would not ordinarily do with a pet. In a follow-up lab study (N=20), we found that most participants did make use of the footstool, although several chose not to place their feet on it for this reason. We also found that participants who rested their feet understood a brief lift and drop movement as a request to withdraw, and formed detailed notions about the footstool's agenda, ascribing intentions based on its movement alone.

In this exploratory study, participants taking cover from a simulated earthquake interacted with a search-and-rescue robot that assumed one of four different identities, distinguished by varying degrees of autonomy, and whose communications were either clear or noisy. Results showed that identities with low autonomy elicited greater hopefulness from participants than identities with high autonomy. Discussion focuses on design recommendations for search-and-rescue robots, and on the design of immersive HRI experiments.

In this study, we examined how participants (N=20) interacted and collaborated with a set of robotic drawers to accomplish an assembly task. The drawers' behavior varied along two dimensions - proactivity and expressivity of motions. The results of our study indicate that participants consider an expressive robot to be more involved and interested in the interaction. We also found that while proactive or expressive robots could dominate the interaction, proactivity might negatively affect the participants' perception of their social status relative to that of the robot's, while expressiveness did not. This shows the importance of utilizing expressive movements when designing socially appropriate robots that collaborate with human users.

Service robots in public places need to both understand environmental cues and move in ways that people can understand and predict. We developed and tested interactions with a trash barrel robot to better understand the implicit protocols for public interaction. In eight lunch-time sessions spread across two crowded campus dining destinations, we experimented with piloting our robot in Wizard of Oz fashion, initiating and responding to requests for impromptu interactions centered on collecting people's trash. Our studies progressed from open-ended experimentation to testing specific interaction strategies that seemed to evoke clear engagement and responses, both positive and negative. Observations and interviews show that a) people most welcome the robot's presence when they need its services and it actively advertises its intent through movement; b) people create mental models of the trash barrel as having intentions and desires; c) mistakes in navigation are indicators of autonomous control, rather than a remote operator; and d) repeated mistakes and struggling behavior polarized responses as either ignoring or endearing.

The role of human-robot interaction is becoming more important as everyday robotic devices begin to permeate into our lives. In this study, we video-prototyped a user's interactions with a set of robotic drawers. The user and robot each displayed one of five emotional states - angry, happy, indifferent, sad, and timid. The results of our study indicated that the participants of our online questionnaire preferred empathetic drawers to neutral ones. They disliked robotic drawers that displayed emotions orthogonal to the user's emotions. This showed the importance of displaying emotions, and empathy in particular, when designing robotic devices that share our living and working spaces.

Does augmented movement capability improve people's experiences with telepresent meeting participants? We performed two web-based studies featuring videos of a telepresence robot. In the first study (N=164), participants observed clips of typical conversational gestures performed a) on a stationary screen only, b) with an actuated screen moving in physical space, or c) both on-screen and in-space. In the second study (N=103), participants viewed scenario videos depicting two people interacting with a remote collaborator through a telepresence robot, whose distant actions were a) visible on the screen only, or b) accompanied by local physical motion. These studies suggest that synchronized on-screen and in-space gestures significantly improved viewers' interpretation of the action compared to on-screen or in-space gestures alone, and that in-space gestures positively influenced perceptions of both local and remote participants.