Methods for understanding the felt and situated experience of controlling autonomous systems are crucial for designing systems that are adjusted to the complexity of human interaction. Prior work has tended to overlook the bodily and experiential dimensions of monitoring systems at a distance, particularly in moments of losing control. We combined two approaches, ethnomethodology and conversation analysis, and soma design, to explore a case where a semi-autonomous system crashed when controlled by an inexperienced operator, captured in video ethnographic fieldwork. We report on our methodological approach, combining sequential video analysis of the unfolding sequence and interviews inspired by microphenomenology to unpack the operator’s experience of losing control. We contribute methodological considerations for interaction designers seeking to explore the felt experience of having and losing control of autonomous systems and discuss how insights gained through this combination of methods, rooted in phenomenology, support a designerly appreciation of safety operators’ work.

Feeling socially connected is important for personal well-being, yet many older adults report increasing loneliness and decreasing social connections. We explored how robots and their behaviors can support group interactions and foster social participation among older adults in a community center setting over repeated interactions. We developed a semi-autonomous collaborative and discussion-based variant of the game "With Other Words" for groups of three to four older adults and two robots. A facilitator robot (Furhat) mediated discussions using gaze and verbal support, while a guesser robot (Misty) attempted to guess the words that group members described 'with other words'. We invited 34 older adults aged 65+ to play the game in groups of three or four, three times over two to five weeks. An explorative mixed-method analysis, combining quantitative metrics with Ethnomethodological Conversation Analysis (EMCA), shows that robot gaze and verbal behaviors as well as negotiations around "wrangling" the guesser robot encouraged participation in the game. Further, verbal supporting behaviors elicited shared laughter but also led to breakdowns. While no direct significant improvement in social connectedness was observed, this work contributes to our understanding of how robot behaviors might shape interactions among older adults.

Delta Fleet AI is a speculative VR demo exploring the hidden human labor behind autonomous mobility systems.Set in a future where autonomous vehicles rely on remote human "RoadRunners" to manage edge cases, users engage in high-stakes scenarios including route selection, manual driving, and obstacle disambiguation.Framed as a corporate onboarding simulation, the experience critiques the myth of full autonomy and exposes the cognitive and emotional demands placed on undervalued human operators.Built for MetaQuest 3, the project showcases VR's potential for provocative speculative design, inviting reflection on automation, labor, and the ethics of human-autonomy teaming.

Voice Agents (VAs) are touted as being able to help users in complex tasks such as cooking and interacting as a conversational partner to provide information and advice while the task is ongoing. Through conversation analysis of 7 cooking sessions with a commercial VA, we identify challenges caused by a lack of contextual awareness leading to irrelevant responses, misinterpretation of requests, and information overload. Informed by this, we evaluated 16 cooking sessions with a wizard-led context-aware VA. We observed more fluent interaction between humans and agents, including more complex requests, explicit grounding within utterances, and complex social responses. We discuss reasons for this, the potential for personalisation, and the division of labour in VA communication and proactivity. Then, we discuss the recent advances in generative models and the VAs interaction challenges. We propose limited context awareness in VAs as a step toward explainable, explorable conversational interfaces.

The concept of ‘bodywork´ refers to the work individuals undertake on their own bodies and the bodies of others. One aspect is attending to bodily needs, which is often overlooked in the workplace and HCI/CSCW research on work practices. Yet, this labour can be a significant barrier to work, consequential to work, and prone to spill over into other aspects of life. We present three empirical cases of bodywork: gig-based food delivery, shift work in hospitals and bars, and office-based knowledge work. We describe what attending to bodily needs at work entails and illustrate tactics employed so that work can be carried on, even when the body (or technology optimising it) breaks down. Arguing that all systems are bodily systems, we conclude with a call to acknowledge the centrality of bodies in all work and the roles technologies can play in supporting or constraining bodywork differently for different workers.

With the increasing adoption of body tracking technology, users are able to collect bio-data which designers struggle to make legible or actionable. This work focused on increasing this technology-mediated bodily understanding through exploring the material properties of the body rhythms that govern the sleep behaviours being tracked.

The emphasis on one-to-one speak/wait spoken conversational interaction with intelligent agents leads to long pauses between conversational turns, undermines the flow and naturalness of the interaction, and undermines the user experience. Despite ground breaking advances in the area of generating and understanding natural language with techniques such as LLMs, conversational interaction has remained relatively overlooked. In this workshop we will discuss and review the challenges, recent work and potential impact of improving conversational interaction with artificial systems. We hope to share experiences of poor human/system interaction, best practices with third party tools, and generate design guidance for the community.

Human-Robot Interaction (HRI) is inherently a human-centric field of technology. The role of feminist theories in related fields (e.g. Human-Computer Interaction, Data Science) are taken as a starting point to present a vision for Feminist HRI which can support better, more ethical HRI practice everyday, as well as a more activist research and design stance. We first define feminist design for an HRI audience and use a set of feminist principles from neighboring fields to examine existent HRI literature, showing the progress that has been made already alongside some additional potential ways forward. Following this we identify a set of reflexive questions to be posed throughout the HRI design, research and development pipeline, encouraging a sensitivity to power and to individuals' goals and values. Importantly, we do not look to present a definitive, fixed notion of Feminist HRI, but rather demonstrate the ways in which bringing feminist principles to our field can lead to better, more ethical HRI, and to discuss how we, the HRI community, might do this in practice.

Performing with technology is a complex and challenging task. Artists who use novel technologies, such as Virtual Reality, have to develop strategies of monitoring, maintenance, and recovery from errors with as minimal impact on the ongoing performance as possible. In this paper we draw on two case studies of mixed-reality performances and document strategies of Stage Managing VR Performance, Choreographing for Cables, Consistency & Charging, Improvising Interventions, and Priming Participants. We discuss how these practices expose areas ripe with potential for tool development, and how they can also be used to inform the design of interaction with other technologies, such as the Internet of Things.

Voice interaction has long been envisioned as enabling users to transform physical interaction into hands-free, such as allowing fine-grained control of instructional videos without physically disengaging from the task at hand. While significant engineering advances have brought us closer to this ideal, we do not fully understand the user requirements for voice interactions that should be supported in such contexts. This paper presents an ecologically-valid wizard-of-oz elicitation study exploring realistic user requirements for an ideal instructional video playback control while cooking. Through the analysis of the issued commands and performed actions during this non-linear and complex task, we identify (1) patterns of command formulation, (2) challenges for design, and (3) how task and voice-based commands are interwoven in real-life. We discuss implications for the design and research of voice interactions for navigating instructional videos while performing complex tasks.

This paper presents Fractured Objects for the design of virtual and mixed-reality experiences. Drawing on the qualitative analysis of three weeks of artistic activities within a residency program, we present six types of Fractured Objects that were used in sketching a mixed-reality performance. Building on these Fractured Objects, as they were articulated by the artists, we present speculative designs for their use in scenarios inspired by research within the IMX community. In discussion, we look to expand the concept of Fractured Objects by relating it to other design concepts such as Seamful Design and Wabi-Sabi, and explore the relationship to the temporality of interaction. We introduce Kintsugi VR with Fractured Objects, drawing on the concept of ‘golden repair’ in which the act of reconnecting fractured parts improves the resulting whole object.

With growing interest in how technology can make sense of our body and bodily experiences, this work looks at how these experiences are communicated through and with the help of technology. We present the ways in which knowledge about sleep, and how to manipulate it, is collectively shared online. This paper documents the sleep-change practices of four groups of 'Sleep Hackers' including Nurses, Polyphasic Sleeper, Over-sleepers, and Biohackers. Our thematic analysis uses 1002 posts taken from public forums discussing sleep change. This work reveals the different ways individuals share their experiences and build communal knowledge on how to 'hack' their sleep -- from using drugs, external stimulation, isolation, and polyphasic sleeping practices where segmented sleep schedules are shared between peers. We describe how communal discussions around the body and sleep can inform the development of body sensing technology. We discuss the opportunities and implications for designing for bodily agency over sleep changes both in relation to collaboratively developed understandings of the body and social context of the user. We also discuss notions of slowly changing bodily processes and sensory manipulation in relation to how they can build on the exploration of soma-technology.

Recent developments in gaze tracking present new opportunities for social computing. This paper presents a study of Tama, a gaze actuated smart speaker. Tama was designed taking advantage of research on gaze in conversation. Rather than being activated with a wake word (such as "Ok Google") Tama detects the gaze of a user, moving an articulated 'head' to achieve mutual gaze. We tested Tama's use in a multi-party conversation task, with users successfully activating and receiving a response to over 371 queries (over 10 trials). When Tama worked well, there was no significant difference in length of interaction. However, interactions with Tama had a higher rate of repeated queries, causing longer interactions overall. Video analysis lets us explain the problems users had interacting with gaze. In the discussion, we describe implications for designing new gaze systems, using gaze both as input and output. We also discuss how the relationship to anthropomorphic design and taking advantage of learned skills of interaction. Finally, two paths for future work are proposed, one in the field of speech agents, and the second in using human gaze as an interaction modality more widely.

While lightweight text messaging applications have been researched extensively, new messaging applications such as iMessage, WhatsApp, and Snapchat offer some new functionality and potential uses. Moreover, the role messaging plays in interaction and talk with those who are co-present has been neglected. In this article, we draw upon a corpus of naturalistic recordings of text message reading and composition to document the face-to-face life of text messages. Messages, both sent and received, share similarities with reported speech in conversation; they can become topical resource for local conversation–supporting verbatim reading aloud or adaptive summaries. Yet with text messages, their verifiability creates a distinctive resource. Similarly, in message composition, what to write may be discussed with collocated others. We conclude with discussion of designs for messaging in both face-to-face, and remote, communication.

Drawing on 168 hours of video recordings of smartwatch use, this paper studies how context influences smartwatch use. We explore the effects of the presence of others, activity, location and time of day on 1,009 instances of use. Watch interaction is significantly shorter when in conversation than when alone. Activity also influences watch use with significantly longer use while eating than when socialising or performing domestic tasks. One surprising finding is that length of use is similar at home and work. We note that usage peaks around lunchtime, with an average of 5.3 watch uses per hour throughout a day. We supplement these findings with qualitative analysis of the videos, focusing on how use is modified by the presence of others, and the lack of impact of watch glances on conversation. Watch use is clearly a context-sensitive activity and in discussion we explore how smartwatches could be designed taking this into consideration.

Designing for interactive performances is challenging both in terms of technology design, and of understanding the interplay between technology, narration, and audience interactions. Bio-sensors and bodily tracking technologies afford new ways for artists to engage with audiences, and for audiences to become part of the artwork. Their deployment raises a number of issues for designers of interactive performances. This paper explores such issues by presenting five design ideas for interactive performance afforded by bio-sensing and bodily tracking (i.e. Microsoft Kinect) developed during two design workshops. We use these ideas, and the related scenarios to discuss three emerging issues namely: temporality of input, autonomy and control, and visibility of input in relation to the deployment of bio-sensors and bodily tracking technologies in the context of interactive performances.

We present five provocations for ethics, and ethical research, in HCI. We discuss, in turn, informed consent, the researcher-participant power differential, presentation of data in publications, the role of ethical review boards, and, lastly, corporate-facilitated projects. By pointing to unintended consequences of regulation and oversimplifications of unresolvable moral conflicts, we propose these provocations not as guidelines or recommendations but as instruments for challenging our views on what it means to do ethical research in HCI. We then suggest an alternative grounded in the sensitivities of those being studied and based on everyday practice and judgement, rather than one driven by bureaucratic, legal, or philosophical concerns. In conclusion, we call for a wider and more practical discussion on ethics within the community, and suggest that we should be more supportive of low-risk ethical experimentation to further the field.

We consider how data is produced and used in cities. We draw on our experiences working with city authorities, along with twenty interviews across four cities to understand the role that data plays in city government. Following the development and deployment of innovative data-driven technology projects in the cities, we look in particular at collaborations around open and crowdsourced data, issues with the politicisation of data, and problems in innovating within the highly regulated public sphere. We discuss what this means for cities, citizens, innovators, and for visions of big data in the smart city as a whole.

In recent years, the smartwatch has returned as a form factor for mobile computing with some success. Yet it is not clear how smartwatches are used and integrated into everyday life differently from mobile phones. For this paper, we used wearable cameras to record twelve participants' daily use of smartwatches, collecting and analysing incidents where watches were used from over 34 days of user recording. This allows us to analyse in detail 1009 watch uses. Using the watch as a timepiece was the most common use, making up 50% of interactions, but only 14% of total watch usage time. The videos also let us examine why and how smartwatches are used for activity tracking, notifications, and in combination with smartphones. In discussion, we return to a key question in the study of mobile devices: how are smartwatches integrated into everyday life, in both the actions that we take and the social interactions we are part of?

This paper is an introduction to the "Future IKEA Catalogue", enclosed here as an example of a design fiction produced from a long standing industrial-academic collaboration. We introduce the catalogue here by discussing some of our experiences using design fiction` with companies and public sector bodies, giving some background to the catalogue and the collaboration which produced it. We have found design fiction to be a useful tool to support collaboration with industrial partners in research projects - it provides a way of thinking and talking about present day concepts, and present day constraints, without being overly concerned with contemporary challenges, or the requirements of academic validation. In particular, there are two main aspects of this we will discuss here, aspects that are visible in the enclosed catalogue itself. The first is the potential of design fiction as a sort of 'boundary object' in industry and academic collaboration, and second the role of critique. After this introduction to the paper we enclose the output of our collaboration in the form of the catalogue itself.

Voice interaction with mobile devices has been focused on hands-free interaction or situations where visual interfaces are not applicable. In this paper we explore a subtler means of interaction -- speech recognition from continual, in the background, audio recording of conversations. We call this the 'continuous speech stream' and explore how it could be repurposed as user input. We analyse ten days of recorded audio from our participants, alongside corresponding interviews, to explore how systems might make use of extracts from this stream. Rather than containing directly actionable items, our data suggests that the continuous speech stream is a rich resource for identifying users' next actions, along with the interests and dispositions of those being recorded. Through design workshops we explored new interactions using the speech stream, and describe concepts for individual, shared and distributed use.

This paper examines mobile internet search, presenting search not as a process of information retrieval, but as part of conversation and talk. Through video extracts of mobile search we explore how mobile phones are interwoven into talk, and how searchers manage the participation of other conversationalists alongside the search itself. We introduce the notion of a 'searchable object' -- an object that arises in conversation that can be searched for online -- and document how such an object occasions a search. In turn we discuss the differing roles of the device 'driver' and 'passenger', and how participation is managed through questions and narration. Rather than search being solely about getting correct information, conversations around search may be just as important. We conclude by critiquing some of the pessimistic views of interaction around mobile phones and their use in ordinary life and talk.

The explosion of mobile applications and services presents challenges for evaluation and user study. One successful approach has been to deploy instrumented applications, logging their use over long periods of time. We present an expansion of this by remotely recording video and audio of use, while also capturing device and app context. In vivo combines five data collection techniques -- screen recording, ambient audio recording, wearable cameras, data logging and distributed remote uploads. This data provides a range of insights and we discuss examples from previous work which reveal interaction design issues where interface confusions or task mismatches occur. We see how apps are integrated into ongoing activity and environment (such as how maps are used in situ), and how recorded conversations around and about apps may be used for evaluation purposes. We conclude by arguing that this combinative method helps us to move from considering app use in isolation, to studying app use in interaction.

The transition from local and personally owned file-based media management to cloud-based streaming services such as Spotify and Netflix brings new opportunities for users, but also leaves gaps in their understanding and practice. In this paper we present findings from an interview study that explored early adopters' complex relationships with their collections which spanned physical, digital and cloud media. From this we entered a design process focussing on new material forms for cloud based media. Based on this we discuss our design and point to areas where, tangible or not, affordances from physical and digital media are available to be explored in the cloud. Looking in particular at the concepts of scarcity, gifting, and identity we outline possible reasons why, and why not, they could be incorporated into cloud media services.

Internet connected mobile devices are an increasingly ubiquitous part of our everyday lives and we present here the results from unobtrusive audio-video recordings of iPhone use -- over 100 days of device use collected from 15 users. The data reveals for analysis the everyday, moment-by-moment use of contemporary mobile phones. Through video analysis of usage we observed how messages, social media and internet use are integrated and threaded into daily life, interaction with others, and everyday events such as transport, delays, establishment choice and entertainment. We document various aspects of end-user mobile device usage, starting with understanding how it is occasioned by context. We then characterise the temporal and sequential nature of use. Lastly, we discuss the social nature of mobile phone usage. Beyond this analysis, we reflect on how to draw these points into ideas for design.

The recent rise in large scale trials of mobile software using 'app stores' has moved current researcher practice beyond available ethical guidelines. By surveying this recent and growing body of literature, as well as established professional principles adopted in psychology, we propose a set of ethical guidelines for large scale HCI user trials. These guidelines come in two parts: a set of general principles and a framework into which individual app store-based trials can be assessed and ethical concerns exposed. We categorise existing literature using our scheme, and explain how researchers could use our framework to classify their future user trials to determine ethical responsibility, and the steps required to meet these obligations.

User trials of mobile applications have followed a steady march out of the lab, and progressively further ''into the wild', recently involving ''app store'-style releases of software to the general public. Yet from our experiences on these mass participation systems and a survey of the literature, we identify a number of reported difficulties. We propose a hybrid methodology that aims to address these, by combining a global software release with a concurrent local trial. A phone-based game, created to explore the uptake and use of ad hoc peer-to-peer networking, was evaluated using this new hybrid trial method, combining a small-scale local trial (11 users) with a ''mass participation' trial (over 10,000 users). Our hybrid method offers many benefits, allowing locally observed findings to be verified, patterns in globally collected data to be explained and addresses ethical issues raised by the mass participation approach. We note trends in the local trial that did not appear in the larger scale deployment, and which would therefore have led to misleading results were the application trialed using ''traditional' methods alone. Based on this study and previous experience, we provide a set of guidelines to researchers working in this area.