Volumetric telepresence aims to create a shared space, allowing people in local and remote settings to collaborate seamlessly. Prior telepresence examples typically have asymmetrical designs, with volumetric capture in one location and objects in one format. In this paper, we present a volumetric telepresence mixed reality system that supports real-time, symmetrical, multi-user, partially distributed interactions, using objects in multiple formats, across multiple locations. We align two volumetric environments around a common spatial feature to create a shared workspace for remote and co-located people using objects in three formats: physical, virtual, and volumetric. We conducted a study with 18 participants over 6 sessions, evaluating how telepresence workspaces support spatial coordination and hybrid communication for co-located and remote users undertaking collaborative tasks. Our findings demonstrate the successful integration of remote spaces, effective use of proxemics and deixis to support negotiation, and strategies to manage interactivity in hybrid workspaces.

Mixed-reality telepresence allows local and remote users feel as if they are present together in the same space. In this paper we report on a mixed-reality volumetric telepresence system that is adaptable, multi-user and cross-modal, i.e. combining augmented and virtual reality technologies with face-to-face interactions. The system extends state-of-art by creating full-body and environmental volumetric renderings in real-time over local enterprise networks. We report findings of an evaluation in a training scenario which was adapted for remote delivery and led by an industry professional. Analysis of interviews and observed behaviours identify varying attitudes towards virtually mediated full-body experiences and highlight the impact of volumetric mixed-reality telepresence to facilitate personal experiences of co-presence and to ground communication with interlocutors.

Temporal target selection requires users to wait and trigger the selection input within a bounded time window, with a selection cursor that is expected to be delayed. This task conceptualizes, for example, a variety of game scenarios such as determining the timing of shooting a projectile towards a moving object. In this work, we explore models that predict “when” users typically perform a selection (i.e., user selection distribution) and their selection error rates in such tasks. We hypothesize that users react to temporal factors including “distance”, “width”, and “delay” as how they treat the corresponding variables in spatial target selection. The derived models are evaluated in a controlled experiment and an MTurk-based online study. Our research contributes new knowledge on user behavior in temporal target selection tasks and its potential connection with its spatial correspondence. Our models and conclusions can benefit both users and designers of relevant interactive applications.

This paper explores using mixed reality (MR) mirrors for supporting improvisational dance making. Motivated by the prevalence of mirrors in dance studios and inspired by Forsythe’s Improvisation Technologies, we conducted workshops with 13 dancers and choreographers to inform the design of future MR visualisation and annotation tools for dance. The workshops involved using a prototype MR mirror as a technology probe that reveals the spatial and temporal relationships between the reflected dancing body and its surroundings during improvisation; speed dating group interviews around future design ideas; follow-up surveys and extended interviews with a digital media dance artist and a dance educator. Our findings highlight how the MR mirror enriches dancers’ temporal and spatial perception, creates multi-layered presence, and affords appropriation by dancers. We also discuss the unique place of MR mirrors in the theoretical context of dance and in the history of movement visualisation, and distil lessons for broader HCI research.

Mirror reflections offer an intuitive and realistic Mixed Reality (MR) experience comparable to other MR interfaces. Their high visual fidelity, and the sensorimotor contingency from the reflected moving body, make the mirror an ideal instrument for MR movement guidance. The translucent two-way mirror display enables users to follow a virtual humanoid instructor’s movement accurately by visually matching it with their reflections. In this work, we conduct the first formal evaluation of movement acquisition performance with simple motor tasks, using visual guidance from an MR mirror and a humanoid virtual instructor. Our results of performance and subjective ratings indicate that, comparing with simulated virtual mirror and with traditional screen-based movement guidance, the real MR mirror yields better acquisition performance and stronger sense of embodiment with the reflection, for upper-body movement. But the benefits diminish with larger-range head movements. We provide design guidelines for future mirror movement guidance interfaces and MR mirror experiences at large.

As extended reality (XR) systems become increasingly available, XR-based remote instruction is being adopted for diverse purposes in professional settings such as surgery and field servicing. Hobbyists have been well-studied in HCI and may similarly benefit from remote skill-sharing. However, little is known about how XR technologies might support expert-novice collaboration for skilled hobby activities. This paper examines the potential and limitations of XR to connect experts and novices for one such activity: gardening. Through two studies involving 27 expert and novice gardeners, we designed prototypes to understand 1) practitioner perceptions of XR and remote skill-sharing in the garden and 2) what kinds of interactions can be supported in XR for expert-novice groups. We discuss design opportunities and challenges for XR systems in supporting informal connecting interactions and meaningful sensory interactions with a remote environment during skill-sharing.

We propose a multi-scale Mixed Reality (MR) collaboration between the Giant, a local Augmented Reality user, and the Miniature, a remote Virtual Reality user, in Giant-Miniature Collaboration (GMC). The Miniature is immersed in a 360-video shared by the Giant who can physically manipulate the Miniature through a tangible interface, a combined 360-camera with a 6 DOF tracker. We implemented a prototype system as a proof of concept and conducted a user study (n=24) comprising of four parts comparing: A) two types of virtual representations, B) three levels of Miniature control, C) three levels of 360-video view dependencies, and D) four 360-camera placement positions on the Giant. The results show users prefer a shoulder mounted camera view, while a view frustum with a complimentary avatar is a good visualization for the Miniature virtual representation. From the results, we give design recommendations and demonstrate an example Giant-Miniature Interaction.

Spatial Augmented Reality (SAR) systems can be suitably combined with other existing Extended Reality (xR) technologies to support collaboration. In existing strategies, users unencumbered by a viewing technology, such as a tablet interface or a head-mounted display, must rely on the transmission of their collaborators' positioning through interpreting a first-person camera view. This design creates a seam between a user's experience of the augmented physical environment in SAR, and their collaborators' experience inside the virtual environment. To assist in development and evaluation of spatial cues to support spatial awareness in SAR environments, an egocentric spatial-communication taxonomy is presented given two determining dimensions, a cue's attachment (physical/virtual) and animation (local/world). We developed four egocentric cues which characterize the four independent dimensions of the matrix: arrow, path, glow, and radial, and a single exocentric world in miniature visualization. Our study shows that virtual attachment cues are preferred, providing the highest accuracy, highest performance when collaborators are occluded, and produce the least mental effort when used with a single virtual collaborator. For multiple collaborators however, the virtual attached, world animated radial cue produces significant increases in mental load and reductions in preference, demonstrating the impact of visual augmentation clutter. The single exocentric visualization produced higher levels of head movement, and poorer accuracy, however the novelty of the visualization produced positive qualitative results.

In this paper we present a floor-based user interface (UI) that allows multiple users to explore a spatial augmented reality (SAR) environment with both monoscopic and stereoscopic projections. Such environments are characterized by a low level of user instrumentation and the capability of providing a shared interaction space for multiple users. However, projector-based systems using stereoscopic display are usually single-user setups, since they can provide the correct perspective for only one tracked person. To address this problem, we developed a set of guidance cues, which are projected onto the floor in order to assist multiple users regarding (i) the interaction with the SAR system, (ii) the identification of regions of interest and ideal viewpoints, and (iii) the collaboration with each other. In a user study with 40 participants all cues were evaluated and a set of feedback elements, which are essential to guarantee an intuitive self-explaining interaction, was identified. The results of the study also indicate that the developed UI guides users to more favorable viewpoints and therefore is able to improve the experience in a multi-user SAR environment.

This paper presents the results of two cognitive load studies comparing three augmented reality display technologies: spatial augmented reality, the optical see-through Microsoft HoloLens, and the video see-through Samsung Gear VR. In particular, the two experiments focused on isolating the cognitive load cost of receiving instructions for a button-pressing procedural task. The studies employed a self-assessment cognitive load methodology, as well as an additional dual-task cognitive load methodology. The results showed that spatial augmented reality led to increased performance and reduced cognitive load. Additionally, it was discovered that a limited field of view can introduce increased cognitive load requirements. The findings suggest that some of the inherent restrictions of head-mounted displays materialize as increased user cognitive load.

Mobile devices such as some recent phones are now fitted with projection capabilities that support Spatial Augmented Reality (SAR) and require investigation to uncover new interaction possibilities. This paper presents a study measuring user performance in a search and select task using a tracked handheld projector and data tags, a 3D physical cue. This physical cue is used to mark the location of hidden SAR information. The experiment required participants to search for virtual symbols presented on two 5ft, multi-sided control panels. Two methods of presenting AR information were employed, SAR alone and SAR with the inclusion of physical cues to indicate the location of the information. The results showed that attaching data tags, compared to virtual content alone lowered the overall task completion time and reduced handheld projector movement. Subjectively, participants also preferred the combination of virtual data with data tags across both task variations

Industrial uses of Augmented Reality (AR) are growing, however their uses are consistently fashioned with an emphasis on consumption, delivering additional information to the worker to assist them in the completion of their job. A promising alternative is to allow user data creation during the actual process by the worker performing their duties. This not only allows spatially located annotations to be produced, it also allows an AR scene to be developed in-situ and in real-time. Tangible markers offer a physical interface while also creating physical containers to allow for fluent interactions. This form factor allows both attached and detached annotations, whilst allowing the creation of an AR scene during the process. This annotated scene will allow asynchronous collaboration to be conducted between multiple stakeholders, both locally and remotely. In this paper we discuss our reasoning behind such an approach, and present the current work on our prototype created to test and validate our proposition.

This paper presents results of a study measuring user performance in a procedural task using Spatial Augmented Reality (SAR). The task required participants to press sequences of buttons on two control panel designs in the correct order. Instructions for the task were shown either on a computer monitor, or projected directly onto the control panels. This work was motivated by discrepancies between the expectations from AR proponents and experimental findings. AR is often promoted as a way of improving user performance and understanding. With notable exceptions however, experimental results do not confirm these expectations. Reasons cited for results include limitations of current display technologies and misregistration caused by tracking and calibration errors. Our experiment utilizes SAR to remove these effects. Our results show that augmented annotations lead to significantly faster task completion speed, fewer errors, and reduced head movement, when compared to monitor based instructions. Subjectively, our results show augmented annotations are preferred by users.