To provide long-term personalized assistance to users, AI agents must have effective long-term memory (LTM). However, there is little understanding of users’ perceptions, practices, and challenges with LTM in agents. We interviewed 21 users of agents such as ChatGPT and Claude to understand people’s everyday experiences with agent LTM. Our findings shed light on the flow of memory in agents as a three-stage process consisting of (1) information intake, (2) storage and management, and (3) retrieval and interpretation. Users’ perceptions of agent LTM are mainly influenced by Stage 3, and thus users’ interactions with agent LTM are mainly attempts at influencing and understanding how the agent retrieves and interprets information from memory. Therefore, we recommend that technological approaches to user interaction with agent LTM focus at least as much on memory retrieval and interpretation as they do on memory intake, storage, and management.

Hybrid video calls include attendees in a conference room with loudspeakers and remote attendees using headsets, each with different options for rendering sound spatially. Two studies explored the listener experience with spatial audio in video calls. One study examined the in-room experience using loudspeakers, comparing among spatialization algorithms spreading voices out horizontally. A second study compared varying degrees of horizontal separation of binaurally rendered voices for a remote participant using a headset. In-room participants preferred the widest spatialization over monophonic, stereo, and stereo-binary audio in metrics related to intelligibility and helpfulness. Remote participants preferred different widths of the audio stage depending on the number of voices. In both studies, rendering sound spatially increased performance in speech stream identification. Results indicate spatial audio benefits for in-room and remote attendees in video calls, although the in-room attendees accepted a wider audio stage than remote users.

Empowered by the rich sensory capabilities and the advancements in artificial intelligence (AI), head-worn displays (HWD) could understand the user’s contexts and provide just-in-time assistance to users’ tasks to augment their everyday lives. However, there has been limited understanding of how users perceive interacting with AI services, and how different factors impact the user experience in HWD applications. In this research, we investigated broadly what user and system factors play important roles in human-AI experiences during an AI-assisted spatial task. We conducted a user study to simulate an everyday scenario where augmented reality (AR) glasses could provide suggestions/assistance. We researched three AI system factors (performance, initiation, transparency) with multiple user factors (personality traits, trust propensity, and prior trust with AI). We not only identified the impact of user traits such as the levels of conscientiousness and prior trust with the AI, but also found interesting interactions between them and system factors such as AI’s performance and initiation strategy. Based on the findings, we suggest that future AI assistance on HWD needs to take users’ individual characteristics into account and customize the system design accordingly.

Wilderness search and rescue (WSAR) is a command-and-control activity where a Command team manages field teams scattered across a large area looking for a lost person. The challenge is that it can be difficult for Command to maintain awareness of field teams and the conditions of the field. We designed RescueCASTR, an interface that explores the idea of deploying field teams with wearable cameras that stream live video or sequential photos periodically to Command that aid contextual awareness. We ran a remote user study with WSAR managers to understand the opportunities and challenges of such a system. We found that the awareness provided by the footage could give additional confidence and comfort to Command, while reducing the need for explicit communications. However, it could also impact workers' traditional roles and shift the burden of responsibility toward Command. We conclude that, while wearable-camera footage could be beneficial to Command, they need to have the tools and means to narrow their focus within the abundance of information provided. Furthermore, camera streams should not be thought of as a replacement for more direct communications, but rather as another tool available to help Command supplement their understanding of events in the field and help them narrow their focus.

The world is entering a new normal of hybrid organisations, in which it will be common that some members are co-located and others are remote. Hybridity is rife with asymmetries that affect our sense of belonging in an organisational space. This paper reports a study of an XR Telepresence technology probe to explore how remote workers might present themselves and be perceived as an equal and unique embodied being in a workplace. VROOM (Virtual Robot Overlay for Online Meetings) augments a standard Mobile Robotic Telepresence experience by (1) adding a virtual avatar overlay of the remote person to the local space, viewable through a HoloLens worn by the local user, through which the remote user can gesture and express themselves, and (2) giving the remote user an immersive 360° view of the local space, captured by a 360° camera on the robot, which they can view through a VR headset. We ran a study to understand how pairs of participants (one local and one remote) collaborate using VROOM in a search and word-guessing game. Our findings illustrate that there is much potential for a system like VROOM to support dynamic collaborative activities in which embodiment, gesturing, mobility, spatial awareness, and non-verbal expressions are important. However, there are also challenges to be addressed, specifically around proprioception, the mixing of a physical robot body with a virtual human avatar, uncertainties of others' views and capabilities, fidelity of expressions, and the appearance of the avatar. We conclude with further design suggestions and recommendations for future work.

Telepresence robots allow people to participate in remote spaces, yet they can be difficult to manoeuvre with people and obstacles around. We designed a haptic-feedback system called "FeetBack," which users place their feet in when driving a telepresence robot. When the robot approaches people or obstacles, haptic proximity and collision feedback are provided on the respective sides of the feet, helping inform users about events that are hard to notice through the robot's camera views. We conducted two studies: one to explore the usage of FeetBack in virtual environments, another focused on real environments. We found that FeetBack can increase spatial presence in simple virtual environments. Users valued the feedback to adjust their behaviour in both types of environments, though it was sometimes too frequent or unneeded for certain situations after a period of time. These results point to the value of foot-based haptic feedback for telepresence robot systems, while also the need to design context-sensitive haptic feedback.

Wilderness search and rescue (WSAR) requires careful communication between workers in different locations. To understand the contexts from which WSAR workers communicate and the challenges they face, we interviewed WSAR workers and observed a mock-WSAR scenario. Our findings illustrate that WSAR workers face challenges in maintaining a shared mental model. This is primarily done through distributed communication using two-way radios and cell phones for text and photo messaging; yet both implicit and explicit communication suffer. WSAR workers send messages for various reasons and share different types of information with varying levels of urgency. This warrants the use of multiple communication modalities and information streams. However, bringing in more modalities introduces the risk of information overload, and thus WSAR workers today still primarily communicate remotely via the radio. Our work demonstrates opportunities for technology to provide implicit communication and awareness remotely, and to help teams maintain a shared mental model even when synchronous realtime communication is sparse. Furthermore, technology should be designed to bring together multiple streams of information and communication while making sure that they are presented in ways that aid WSAR workers rather than overwhelming them.

People often enjoy sharing outdoor activities together such as walking and hiking. However, when family and friends are separated by distance it can be difficult if not impossible to share such activities. We explore this design space by investigating the benefits and challenges of using a telepresence robot to support outdoor leisure activities. In our study, participants participated in the outdoor activity of geocaching where one person geocached with the help of a remote partner via a telepresence robot. We compared a wide field of view (WFOV) camera to a 360° camera. Results show the benefits of having a physical embodiment and a sense of immersion with the 360° view. Yet challenges related to a lack of environmental awareness, safety issues, and privacy concerns resulting from bystander interactions. These findings illustrate the need to design telepresence robots with the environment and public in mind to provide an enhanced sensory experience while balancing safety and privacy issues resulting from being amongst the general public.

In the coming years, emergency calling services in North America will begin to incorporate new modalities for reporting emergencies, including video-based calling. The challenge is that we know little of how video calling systems should be designed and what benefits or challenges video calling might bring. We conducted observations and contextual interviews within three emergency response call centres to investigate these points. We focused on the work practices of call takers and dispatchers. Results show that video calls could provide valuable contextual information about a situation and help to overcome call taker challenges with information ambiguity, location, deceit, and communication issues. Yet video calls have the potential to introduce issues around control, information overload, and privacy if systems are not designed well. These results point to the need to think about emergency video calling along a continuum of visual modalities ranging from audio calls accompanied with images or video clips to one-way video streams to two-way video streams where camera control and camera work need to be carefully designed.

Computer mediated-communication tools (CMC) support loved ones in maintaining connections with one another over distance, yet it can be difficult to do activities together. We studied the use of telepresence robots for supporting distance-separated loved ones in engaging in the joint activity of shopping over distance. One partner shopped in person while the other used either a telepresence robot or a tablet from a remote location. Compared to the tablet group, we found that when partners communicated through a telepresence robot, the remote partner's personality and presence were expressed through the movements and physicality of the medium. However, the use of the telepresence robot introduced tension between partners regarding responsibility, dependency, and contribution to the act of shopping. These results demonstrate the benefits of a mobile embodiment for remote partners, as well as the need for greater physical capabilities to support both physical connection and remote contribution to leisure activities.

People are increasingly using mobile video to communicate, collaborate, and share experiences while on the go. Yet this presents challenges in adequately sharing camera views with remote users. In this paper, we study the use of semi-autonomous drones for video conferencing, where an outdoor user (using a smartphone) is connected to a desktop user who can explore the environment from the drone's perspective. We describe findings from a study where pairs collaborated to complete shared navigation and search tasks. We illustrate the benefits of providing the desktop user with a view that is elevated, manipulable, and decoupled from the outdoor user. In addition, we articulate how participants overcame challenges in communicating environmental information and navigational cues, negotiated control of the view, and used the drone as a tool for sharing experiences. This provides a new way of thinking about mobile video conferencing where cameras that are decoupled from both users play an integral role in communication, collaboration, and sharing experiences.

Mobile video conferencing, where one or more participants are moving about in the real world, enables entirely new interaction scenarios (e.g., asking for help to construct or repair an object, or showing a physical location). While we have a good understanding of the challenges of video conferencing in office or home environments, we do not fully understand the mechanics of camera work-how people use mobile devices to communicate with one another-during mobile video calls. To provide an understanding of what people do in mobile video collaboration, we conducted an observational study where pairs of participants completed tasks using a mobile video conferencing system. Our analysis suggests that people use the camera view deliberately to support their interactions-for example, to convey a message or to ask questions-but the limited field of view, and the lack of camera control can make it a frustrating experience.