Methods for understanding the felt and situated experience of controlling autonomous systems are crucial for designing systems that are adjusted to the complexity of human interaction. Prior work has tended to overlook the bodily and experiential dimensions of monitoring systems at a distance, particularly in moments of losing control. We combined two approaches, ethnomethodology and conversation analysis, and soma design, to explore a case where a semi-autonomous system crashed when controlled by an inexperienced operator, captured in video ethnographic fieldwork. We report on our methodological approach, combining sequential video analysis of the unfolding sequence and interviews inspired by microphenomenology to unpack the operator’s experience of losing control. We contribute methodological considerations for interaction designers seeking to explore the felt experience of having and losing control of autonomous systems and discuss how insights gained through this combination of methods, rooted in phenomenology, support a designerly appreciation of safety operators’ work.

The proliferation of robots in public spaces necessitates a deeper understanding of how these robots can interact with those they share the space with. In this paper, we present findings from video analysis of publicly deployed cleaning robots in a transit space—a major commercial airport, using their navigational “troubles” as a tool to document what robots currently lack in interactional competence. We demonstrate that these robots, while technically proficient, can disrupt the social order of a space due to their inability to understand core aspects of human movement: mutual adjustment to others, the significance of understanding social groups, and the purpose of different locations. In discussion we argue for exploring a new design space of movement: socially-aware movement. By developing “strong concepts” that treat movement as an interactional and collaborative accomplishment, we can create systems that better integrate into the everyday rhythms of public life.

In the Wizard-of-Oz study paradigm, human "wizards" perform not-yet-implemented system behavior, simulating, among others, how autonomous robots could interact in public to see how unwitting bystanders respond. This paper analyzes a 60-minute video recording of two wizards in a public plaza who are operating two trash-collecting robots within their line of sight. We take an ethnomethodology and conversation analysis perspective to scrutinize interactions between the wizards and the people in the plaza, focusing on critical instances where one robot gets stuck and requires collaborative intervention by the wizards. Our analysis unpacks how the wizards deal with emergent problems by pushing one robot into the other, how they manage front and backstage interactions, and how they monitor the location of each other's robots. We discuss how scrutinizing the work of wizards can inform explorative Wizard-of-Oz paradigms, the design of multi-agent robot systems, and the operation of urban robots from a distance.

“Sustainability −4- HRI, HRI −4-Sustainability” offers hands-on engagement with the HRI 2025 conference theme, “Robots for a Sustainable World”. This workshop will explore the relationship between HRI and sustainable development and stimulate discussion on how we can make our own research practices more sustainable. We propose a full-day workshop, featuring morning discussions with sustainability experts and activists, and an afternoon hands-on activity aimed at understanding how robotics research can help in creating sustainable futures. We will broadly answer the following questions: How can we, as robotic researchers, help address sustainable development responsibly? Equally, how can we ensure that our HRI research practices minimises ecological footprints and operates within ethical and sustainable frameworks? We envision two practical outcomes of this workshop: a “sustainability statement” that can be submitted together with future HRI research papers, and a paper gathering insights and reflections from the workshop. We welcome researchers and students at any career stage and from any subfield of HRI to attend and contribute.

The concept of the “user” is central to research in Human- Robot Interaction (HRI). Drawing on the field studies of functional service robots in public spaces, the paper contributes a conceptual framework for describing positions and roles humans can take beyond being robot users. We rely on the concept of “subject position” from Human-Computer Interaction (HCI) to map out people beyond users into three ecologies: Service, Street Life, and Public Discourse. Through the lens of these ecologies, we highlight how people encounter the robot as (i) an actor in service provision, (ii) a (social and material) actor moving with other members of the street, and (iii) a matter of public concern. The paper contributes a vocabulary for describing the diverse roles of humans and offers methodological suggestions for more holistic research and design of HRI. Further, we outline disciplinary and ethical considerations relating to designing with ecologies in mind.

If robots are to be deployed in public places, we need to understand what factors their design should consider. Informed by sociological studies of urban settings, particularly the work of William H. Whyte and the Street Life Project, we describe four characteristics of public places that affect and are affected by robot design: (1) localism-how robot design aligns with the identity, culture, and character of the place(s) they reside within; (2) environments-the physical characteristics of the environment in which public robots operate; (3) activities- consideration for the various daily, occasional, and situational activities that are tied to place(s) robots inhabit; and (4) sociability- how people collectively and individually relate to, interact with, and make sense of robots deployed in public places. Throughout, we illustrate these characteristics with examples drawn from empirical studies of public robots. We discuss how these key characteristics of public places can inform HRI design.

As we see robots being deployed into new places in everyday life, questions arise about what 'human-robot collaboration' (HRC) might look like there. At the same time, HRC researchers are looking to CSCW for better conceptualisations of 'collaboration', and recent work has called for more CSCW-oriented studies of HRC to support this. We address this via an ethnomethodological study of encounters between pedestrians and food delivery robots on public streets. Our analysis--using video recorded fragments of what happens on the streets--demonstrates how passers-by continuously manage walking trajectories in ways that account for robot actions; specifically we articulate how people accomplish practices of following and overtaking robots, passing by and crossing paths with them. We then show that the picture of human-robot collaboration is drawn with distinct asymmetries of action and intelligibility, where humans contribute considerable work to get something that looks like 'collaboration' achieved. This raises fundamental questions for how we talk about concepts of collaboration in HRC from a CSCW perspective, and how such notions can and should be applied to activities which include robots.

This paper analyzes honks directed at autonomous vehicles (AVs) by other drivers. As honks often mark problems, this focus allows us to better understand the challenges that AVs face in real traffic. Performing a sequential video analysis of 63 honk incidents uploaded by Tesla beta testers on YouTube, we identify how problematic situations emerge as honkable Traffic Gestalts. We identify four types of situated problems with AV driving performance marked by other drivers’ honks: they may wait too long, steer inconsistently, stop instead of going, and go too fast. We further show how a honk may be understandable as a warning, a nudge or a reprimand. Our work suggests designing honks for AVs to focus on relevant contexts, supported by developing bidirectional interfaces and audio analysis methods that consider the interplay of auditory and visual information in traffic.

Robots deployed in public settings enter spaces that humans live and work in. Studies of HRI in public tend to prioritise direct and deliberate interactions. Yet this misses the most common form of response to robots, which ranges from subtle fleeting interactions to virtually ignoring them. Taking an ethnomethodological approach building on video recordings, we show how robots become embedded in urban spaces both from a perspective of the social assembly of the physical environment (the streetscape) and the socially organised nature of everyday street life. We show how such robots are effectively 'granted passage' through these spaces as a result of the practical work of the streets' human inhabitants. We detail the contingent nature of the streetscape, drawing attention to its various members and the accommodation work they are doing. We demonstrate the importance of studying robots during their whole deployment, and approaches that focus on members' interactional work.

Horns and sirens are important tools for communicating on the road, which are still understudied in autonomous vehicles. While HRI has explored different ways in which robots could sound, we focus on the range of actions that a single sound can accomplish in interaction. In a Research through Design study involving autonomous shuttle buses in public transport, we explored sound design with the help of voice-overs to video recordings of the buses on the road and Wizard-of-Oz tests in live traffic. The buses are slowed down by (unnecessary) braking in response to people getting close. We found that prolonged jingles draw attention to the bus and invite interaction, while repeated short beeps and bell sounds can instruct the movement of others away from the bus. We highlight the importance of designing sound in sequential interaction and describe a new method for embedding video interaction analysis in the design process.

INTRODUCTION: Sound is an important interaction modality and a large part of human interaction happens in the aural domain. While research in Human-Robot Interaction (HRI) has long explored spoken language for interacting with humans, sound as a broader-and, to a significant degree, nonlexical (i.e., without words)-medium has been given comparably less attention. Yet, the range of sounds that robots can produce is vast, encompassing, among others, mechanical noise, music, and utterances that mimic human and animal vocalizations with varying degrees of realism. The sound of a robot's machinery can shape our perceptions and expectations [11, 17], music serves as a medium for robots to engage and communicate [18], and shared musical experiences can strengthen the bond between humans and robots [5]. Sonifications may enhance the legibility of movement and gestures [4, 7, 15] and beep sounds may be used to communicate emotions [2, 14]. Getting closer to themargins of language, robotsmay take inspiration from non-lexical fillers such as "uh" [13, 16] and backchannels such as "mhmm" [8, 12]. More generally, pitch, intensity, and other human prosodic variations may be drawn on in robot sound design [3, 10]. The information that can be extracted from sound in a robot's environment is equally rich. Beyond the recognition of semantic content, robots use, for example, sound source localization to gain a better understanding of their environment [9], or analyze a human's voice timbre and tone to distinguish speakers [6] and detect emotion [1].

Delays in the completion of joint actions are sometimes unavoidable. How should a robot communicate that it cannot immediately act or respond in a collaborative task? Drawing on video recordings of a face-scanning activity in family homes, we investigate how humans make sense of a Cozmo robot’s delays on a moment-by-moment basis. Cozmo’s sounds and embodied actions are recognized as indicators of delay but encourage human participants to act in ways that undermine the scanning process. In comparing the robot’s delay management strategies with human-human vocal and embodied practices, we demonstrate key differences in the sequences that impact how the robot is understood. The study demonstrates how delay events are accomplished as embodied displays that are distributed across co-participants. We present a framework for making delay transparent through situated explanations, particularly in the form of non-lexical sounds and bodily actions.

This paper explores how humans interpret displays of emotion produced by a social robot in real world situated interaction. Taking a multimodal conversation analytic approach, we analyze video data of families interacting with a Cozmo robot in their homes. Focusing on one happy and one sad robot animation, we study, on a turn-by-turn basis, how participants respond to audible and visible robot behavior designed to display emotion. We show how emotion animations are consequential for interactional progressivity: While displays of happiness typically move the interaction forward, displays of sadness regularly lead to a reconsideration of previous actions by humans. Furthermore, in making sense of the robot animations people may move beyond the designer's reported intentions, actually broadening the opportunities for their subsequent engagement. We discuss how sadness functions as an interactional "rewind button" and how the inherent vagueness of emotion displays can be deployed in design.

Drawing on ethnographic fieldwork in two different teaching hospitals that deployed the da Vinci surgical robot, this paper traces how the introduction of robotics reconfigures the sensory environment of surgery and how surgeons and their teams recalibrate their work in response. We explore the entangled and mutually supportive nature of sensing within and between individual actors and the broader world of people and things (with emphasis on vision and touch) and illustrate how such inter-sensory dependencies are challenged and sometimes extended under the conditions of robotic surgery. We illustrate how sensory (re)articulations and compensations allow the surgeon and surgical teams to adapt to a more-than-human sensorium and conclude by advocating new forms of sensory-aware design capable of enhancing and supporting embodied sensory conditions both individually and across teams.

This paper investigates how a teleoperated surgical robot reconfigures teamwork in the operating room by spatially redistributing team members. We report on findings from two years of fieldwork at two hospitals, including interviews and video data. We find that while in non-robotic cases team members huddle together, physically touching, introduction of a surgical robot increases physical and sensory distance between team members. This spatial rearrangement has implications for both cognitive and affective dimensions of collaborative surgical work. Cognitive distance is increased, necessitating new efforts to maintain situation awareness and common ground. Moreover, affective distance is introduced, decreasing sensitivity to shared and non-shared affective states and leading to new practices aimed at restoring affective connection within the team. We describe new forms of physical, cognitive, and affective distance associated with teleoperated robotic surgery, and the effects these have on power distribution, practice, and collaborative experience within the surgical team.

This paper explores how humans adapt to a conventional humanoid robot. Video data of participants playing a charade game with a Nao robot were analyzed from a multimodal conversation analysis perspective. Participants soon adjust aspects of turn-design such as word selection, turn length and prosody, thereby adapting to the robot's limited perceptive abilities as they become apparent in the interaction. However, coordination of turns-at-talk remains troublesome throughout the encounter, as evidenced by overlapping turns and lengthy silences around possible turn endings. The study discusses how the robot design can be improved to support the problematic taking of turns-at-talk with humans. Two programming strategies to address the identified problems are presented: 1. to program the robot so that it will be systematically receptive at the equivalence to transition relevance places in human-human interaction, and 2. to make the robot preferably produce verbal actions that require a response in a conditional way, rather than making a response only possible.