Collaborative Mixed Reality (MR) systems that help extend expertise for physical tasks to remote environments often situate experts in an immersive view of the task environment to bring the collaboration closer to collocated settings. In this paper, we design UnMapped, an alternative interface for remote experts that combines a live 3D view of the active space within the novice’s environment with a static 3D recreation of the expert’s own workspace to leverage their existing spatial memories within it. We evaluate the impact of this approach on single and repeated use of collaborative MR systems for remote guidance through a comparative study. Our results indicate that despite having a limited understanding of the novice’s environment, using an UnMapped interface increased performance and communication efficiency while reducing experts’ task load. We also outline the various affordances of providing remote experts with a familiar and spatially-stable environment to assist novices.

Eight middle- and high-school Computer Science (CS) teachers in San Diego County were interviewed about the major challenges their students commonly encounter in learning computer programming. We identified strategic design opportunities -- that is, challenges and needs that can be addressed in innovative ways through the affordances of Augmented and Virtual Reality (AR/VR). Thematic Analysis of the interviews yielded six thematic clusters: Tools for Learning, Visualization and Representation, Pedagogical Approaches, Classroom Culture, Motivation, and Community Connections. Within the theme of visualization, focal clusters centered on visualizing problem spaces and using metaphors to explain computational concepts, indicating that an AR/VR coding system could help users to represent computational problems by allowing them to build from existing embodied experiences and knowledge. Additionally, codes clustered within the theme of learning tools reflected educators’ preference for web-based IDEs, which involve minimal start-up costs, as well as concern over the degree of transfer in learning between block- and text-based interfaces. Finally, themes related to motivation, community, and pedagogical practices indicated that the design of an AR coding platform should support collaboration, self-expression, and autonomy in learning. It should also foster selfefficacy and learners’ ability to address lived experience and real-world problems through computational means.

Traumatic injuries require timely intervention, but medical expertise is not always available at the patient’s location. Despite recent advances in telecommunications, surgeons still have limited tools to remotely help inexperienced surgeons. Mixed Reality hints at a future where remote collaborators work side-by-side as if co-located; however, we still do not know how current technology can improve remote surgical collaboration. Through role-playing and iterative-prototyping, we identify collaboration practices used by expert surgeons to aid novice surgeons as well as technical requirements to facilitate these practices. We then introduce ARTEMIS, an AR-VR collaboration system that supports these key practices. Through an observational study with two expert surgeons and five novice surgeons operating on cadavers, we find that ARTEMIS supports remote surgical mentoring of novices through synchronous point, draw, and look affordances and asynchronous video clips. Most participants found that ARTEMIS facilitates collaboration despite existing technology limitations explored in this paper.

Mixed Reality has been shown to enhance remote guidance and is especially well-suited for physical tasks. Conversations during these tasks are heavily anchored around task objects and their spatial relationships in the real world, making referencing - the ability to refer to an object in a way that is understood by others - a crucial process that warrants explicit support in collaborative Mixed Reality systems. This paper presents a 2x2 mixed factorial experiment that explores the effects of providing spatial information and system-generated guidance to task objects. It also investigates the effects of such guidance on the remote collaborator’s need for spatial information. Our results show that guidance increases performance and communication efficiency while reducing the need for spatial information, especially in unfamiliar environments. Our results also demonstrate a reduced need for remote experts to be in immersive environments, making guidance more scalable, and expertise more accessible.