The literature has demonstrated the advantages of Mixed Reality (MR) for product configuration by providing a more engaging and effective end-user experience. While collaborative and remote design tools in MR have been widely explored in previous studies, a noticeable gap remains in the exploration of co-located product configuration for couples. This gap is noteworthy since in many industries, couples (e.g., friends, partners) often make purchasing decisions together in physical retail environments. In this paper, we introduce MR-CoCo, an open MR testbed designed to explore collaborative configurations by co-located couples, both in the role of customers. The testbed is developed in Unity and features: (i) a shared MR space with virtual product 3D model anchoring, (ii) shared visualization of the current configuration, (iii) a versatile UI for selecting configuration areas, (iv) hand gestures for 3D drag and drop of colors and materials from 3D catalog to the product. A case study of the personalization of a sailboat is provided as proof of concept. The user study involved 24 couples (48 participants in total), simulating a purchasing experience and the related configuration using MR-CoCo. We assessed usability through post-experience evaluations, with the System Usability Scale (SUS) and the Co-Presence Configuration Questionnaire (CCQ) to measure collaboration and decision-making. The results demonstrated a high level of usability and perceived quality of collaboration. We also explore guidelines that can be used for remote collaboration applications, enabling configuration across a wide range of industries (e.g., automotive and clothing).

Extended Reality (XR) offers the potential to transform the passenger experience by allowing users to inhabit varied virtual spaces for entertainment, work or social interaction, whilst escaping the constrained transit environment. XR allows remote collaborators to feel like they are together and enables them to perform complex 3D tasks. However, the social and physical constraints of the passenger space pose unique challenges to productive and socially acceptable collaboration. Using a collaborative VR puzzle task, we examined the effects of five different f-formations of collaborator placement and orientation in an interactive workspace on social presence, task workload, and implications for social acceptability. Our quantitative and qualitative results showed that face-to-face formations were preferred for tasks with a high need for verbal communication but may lead to social collisions, such as inadvertently staring at a neighbouring passenger, or physical intrusions, such as gesturing in another passenger’s personal space. More restrictive f-formations, however, were preferred for passenger use as they caused fewer intrusions on other passengers’ visual and physical space.

Extended Reality (XR) technology brings exciting possibilities for aeroplane passengers, allowing them to escape their limited cabin space. Using nearby physical surfaces enables a connection with the real world while improving the XR experience through touch. However, available surfaces may be located in awkward positions, reducing comfort and input performance and thus limiting their long-term use. We explore the usability of passive haptic surfaces in different orientations, assessing their effects on input performance, user experience and comfort. We then overcome ergonomic issues caused by the confined space by using perceptual manipulation techniques that remap the position and rotation of physical surfaces and user movements, assessing their effects on task workload, comfort and presence. Our results show that the challenges posed by constrained seating environments can be overcome by a combination of passive haptics and remapping the workspace with moderate translation and rotation manipulations. These manipulations allow for good input performance, low workload and comfortable interaction, opening up XR use while in transit.

Remote collaborative work has become pervasive in many settings, ranging from engineering to medical professions. Users are im-mersed in virtual environments and communicate through life-sized avatars that enable face-to-face collaboration. Within this context, users often collaboratively view and interact with virtual 3D models, for example to assist in the design of new devices such as cus-tomized prosthetics, vehicles or buildings. Discussing such shared 3D content face-to-face, however, has a variety of challenges such as ambiguities, occlusions, and different viewpoints that all decrease mutual awareness, which in turn leads to decreased task performance and increased errors. To address this challenge, we introduce MAGIC, a novel approach for understanding pointing gestures in a face-to-face shared 3D space, improving mutual understanding and awareness. Our approach distorts the remote user's gestures to correctly reflect them in the local user's reference space when face-to-face. To measure what two users perceive in common when using pointing gestures in a shared 3D space, we introduce a novel metric called pointing agreement. Results from a user study suggest that MAGIC significantly improves pointing agreement in face-to-face collaboration settings, improving co-presence and awareness of interactions performed in the shared space. We believe that MAGIC improves remote collaboration by enabling simpler communication mechanisms and better mutual awareness.

Standalone Virtual Reality (VR) headsets can be used when travelling in cars, trains and planes. However, the constrained spaces around transport seating can leave users with little physical space in which to interact using their hands or controllers, and can increase the risk of invading other passengers' personal space or hitting nearby objects and surfaces. This hinders transport VR users from using most commercial VR applications, which are designed for unobstructed 1-2m 360° home spaces. In this paper, we investigated whether three at-a-distance interaction techniques from the literature could be adapted to support common commercial VR movement inputs and so equalise the interaction capabilities of at-home and on-transport users: Linear Gain, Gaze-Supported Remote Hand, and AlphaCursor. First, we analysed commercial VR experiences to identify the most common movement inputs so that we could create gamified tasks based on them. We then investigated how well each technique could support these inputs from a constrained $50\mathrm{x}50\text{cm}$ space (representative of an economy plane seat) through a user study $(\mathrm{N}=16)$, where participants played all three games with each technique. We measured task performance, unsafe movements (play boundary violations, total arm movement) and subjective experience and compared results to a control 'at-home' condition (with unconstrained movement) to determine how similar performance and experience were. Results showed that Linear Gain was the best technique, with similar performance and user experience to the 'at-home' condition, albeit at the expense of a high number of boundary violations and large arm movements. In contrast, AlphaCursor kept users within bounds and minimised arm movement, but suffered from poorer performance and experience. Based on the results, we provide eight guidelines for the use of, and research into, at-a-distance techniques and constrained spaces.

We present PassengXR, an open-source toolkit for creating passenger eXtended Reality (XR) experiences in Unity. XR allows travellers to move beyond the physical limitations of in-vehicle displays, rendering immersive virtual content based on - or ignoring - vehicle motion. There are considerable technical challenges to using headsets in moving environments: maintaining the forward bearing of IMU-based headsets; conflicts between optical and inertial tracking of inside-out headsets; obtaining vehicle telemetry; and the high cost of design given the necessity of testing in-car. As a consequence, existing vehicular XR research typically relies on controlled, simple routes to compensate. PassengXR is a cost-effective open-source in-car passenger XR solution. We provide a reference set of COTS hardware that enables the broadcasting of vehicle telemetry to multiple headsets. Our software toolkit then provides support to correct vehicle-headset alignment, and then create a variety of passenger XR experiences, including: vehicle-locked content; motion- and location-based content; and co-located multi-passenger applications. PassengXR also supports the recording and playback of vehicle telemetry, assisting offline design without resorting to costly in-car testing. Through an evaluation-by-demonstration, we show how our platform can assist practitioners in producing novel, multi-user passenger XR experiences.

Chronic pain is a continuous ailment lasting for long periods after the initial injury or disease has healed. Chronic pain is challenging to treat and affects the daily lives of patients. Distraction therapy is a proven method of relieving patients’ discomfort by taking their attention away from the pain. Virtual reality (VR) is a platform for distraction therapy by immersing the user in a virtual world detached from reality. However, there is little research on how physical interactions in VR affect pain management. We present a study to evaluate the effectiveness of physically active, mentally active, and passive interventions in VR using games with chronic pain patients. Our results indicate that physical and mental activities in VR are equally effective at reducing pain. Furthermore, These actively engage patients, while the effects of observing relaxing content persist outside VR. These findings can help inform the design of future VR games targeted at chronic pain management.

This article discusses the Keyboard Augmentation Toolkit (KAT), which supports the creation of virtual keyboards that can be used both for standalone input (e.g., for mid-air text entry) and to augment physically tracked keyboards/surfaces in mixed reality. In a user study, we firstly examine the impact and pitfalls of visualising shortcuts on a tracked physical keyboard, exploring the utility of virtual per-keycap displays. Supported by this and other recent developments in XR keyboard research, we then describe the design, development, and evaluation-by-demonstration of KAT. KAT simplifies the creation of virtual keyboards (optionally bound to a tracked physical keyboard) that support enhanced display —2D/3D per-key content that conforms to the virtual key bounds; enhanced interactivity —supporting extensible per-key states such as tap, dwell, touch, swipe; flexible keyboard mappings that can encapsulate groups of interaction and display elements, e.g., enabling application-dependent interactions; and flexible layouts —allowing the virtual keyboard to merge with and augment a physical keyboard, or switch to an alternate layout (e.g., mid-air) based on need. Through these features, KAT will assist researchers in the prototyping, creation and replication of XR keyboard experiences, fundamentally altering the keyboard’s form and function.

Passengers spend considerable periods of time in shared transit spaces, relying on smartphones and laptops for work. However, these displays are limited in size and ergonomics compared to typical multi-monitor setups used in the office, impairing productivity. Augmented Reality (AR) headsets could provide large, flexible virtual workspaces during travel, enabling passengers to work more efficiently. This paper investigates the factors affecting how passengers choose to layout virtual displays in car, train, subway and plane environments, studying the affordances of each mode of transport and the presence of others. Results from our experiment showed: significant usage of the physical environment to align displays; strong social effects meant avoiding placing displays over other passengers or their belongings; and use of displays for shielding oneself from others. Our findings show the unique challenges posed by the mode of transport and presence of others on the use of AR for mobile productivity in the future.

Augmented / Mixed Reality headsets will in-time see adoption and use in a variety of mobility and transit contexts, allowing users to view and interact with virtual content and displays for productivity and entertainment. However, little is known regarding how multi-display virtual workspaces should be presented in a transit context, nor to what extent the unique affordances of transit environments (e.g. the social presence of others) might influence passenger perception of virtual display layouts. Using a simulated VR passenger airplane environment, we evaluated three different AR-driven virtual display configurations (Horizontal, Vertical, and Focus main display with smaller secondary windows) at two different depths, exploring their usability, user preferences, and the underlying factors that influenced those preferences. We found that the perception of invading other’s personal space significantly influenced preferred layouts in transit contexts. Based on our findings, we reflect on the unique challenges posed by passenger contexts, provide recommendations regarding virtual display layout in the confined airplane environment, and expand on the significant benefits that AR offers over physical displays in said environments.

Head-Mounted Virtual reality (VR) systems provide full-immersive experiences to users and completely isolate them from the outside world, placing them in unsafe situations. Existing research proposed different alert-based solutions to address this. Our work builds on these studies on notification systems for VR environments from a different perspective. We focus on: (i) exploring alert systems to notify VR users about non-immersed bystanders' in socially related, non-critical interaction contexts; (ii) understanding how best to provide awareness of non-immersed bystanders while maintaining presence and immersion within the Virtual Environment(VE). To this end, we developed single and combined alert cues - leveraging proxemics, perception channels, and push/pull approaches and evaluated those via two user studies. Our findings indicate a strong preference towards maintaining immersion and combining audio and visual cues, push and pull notification techniques that evolve dynamically based on proximity.

In this paper, we present a system that allows a user with a head-mounted display (HMD) to communicate and collaborate with spectators outside of the headset. We evaluate its impact on task performance, immersion, and collaborative interaction. Our solution targets scenarios like live presentations or multi-user collaborative systems, where it is not convenient to develop a VR multiplayer experience and supply each user (and spectator) with an HMD. The spectator views the virtual world on a large-scale tiled video wall and is given the ability to control the orientation of their own virtual camera. This allows spectators to stay focused on the immersed user's point of view or freely look around the environment. To improve collaboration between users, we implemented a pointing system where a spectator can point at objects on the screen, which maps an indicator directly onto the objects in the virtual world. We conducted a user study to investigate the influence of rotational camera decoupling and pointing gestures in the context of HMD-immersed and non-immersed users utilizing a large-scale display. Our results indicate that camera decoupling and pointing positively impacts collaboration. A decoupled view is preferable in situations where both users need to indicate objects of interest in the scene, such as presentations and joint-task scenarios, as it requires a shared reference space. A coupled view, on the other hand, is preferable in synchronous interactions such as remote-assistant scenarios.

Corresponding lighting and reflectance between real and virtual objects is important for spatial presence in augmented and mixed reality (AR and MR) applications. We present a method to reconstruct real-world environmental lighting, encoded as a reflection map (RM), from a conventional photograph. To achieve this, we propose a stacked convolutional neural network (SCNN) that predicts high dynamic range (HDR) 360° RMs with varying roughness from a limited field of view, low dynamic range photograph. The SCNN is progressively trained from high to low roughness to predict RMs at varying roughness levels, where each roughness level corresponds to a virtual object's roughness (from diffuse to glossy) for rendering. The predicted RM provides high-fidelity rendering of virtual objects to match with the background photograph. We illustrate the use of our method with indoor and outdoor scenes trained on separate indoor/outdoor SCNNs showing plausible rendering and composition of virtual objects in AR/MR. We show that our method has improved quality over previous methods with a comparative user study and error metrics.

Locomotion in virtual environments is currently a difficult and unnatural task to perform. Normally, researchers tend to devise ground- or floor-based metaphors to constrain the degrees of freedom (DoFs) during motion. These restrictions enable interactions that accurately emulate the human gait to provide high interaction fidelity. However, flying allows users to reach specific locations in a virtual scene more expeditiously. Our experience suggests that even though flying is not innate to humans, high-interaction-fidelity techniques may also improve the flying experience since flying requires simultaneously controlling additional DoFs. We present the Magic Carpet, an approach to flying that combines a floor proxy with a full-body representation to avoid balance and cybersickness issues. This design space enables DoF separation by treating direction indication and speed control as two separate phases of travel, thereby enabling techniques with higher interaction fidelity. To validate our design space, we conducted two complementary studies, one for each of the travel phases. In this paper, we present the results of both studies and report the best techniques for use within the Magic Carpet design space. To this end, we use both objective and subjective measures to evaluate the efficiency, embodiment effect, and side effects, such as physical fatigue and cybersickness, of the tested techniques in our design space. Our results show that the proposed approach enables high-interaction-fidelity techniques while improving the user experience.

Telecollaboration involves the teleportation of a remote collaborator to another real-world environment where their partner is located. The fidelity of the environment plays an important role for allowing corresponding spatial references in remote collaboration. We present a novel asymmetric platform, Augmented Virtual Teleportation (AVT), which provides high-fidelity telepresence of a remote VR user (VR-Traveler) into a real-world collaboration space to interact with a local AR user (AR-Host). AVT uses a 360° video camera (360-camera) that captures and live-streams the omni-directional scenes over a network. The remote VR-Traveler watching the video in a VR headset experiences live presence and co-presence in the real-world collaboration space. The VR-Traveler's movements are captured and transmitted to a 3D avatar overlaid onto the 360-camera which can be seen in the AR-Host's display. The visual and audio cues for each collaborator are synchronized in the Mixed Reality Collaboration space (MRC-space), where they can interactively edit virtual objects and collaborate in the real environment using the real objects as a reference. High fidelity, real-time rendering of virtual objects and seamless blending into the real scene allows for unique mixed reality use-case scenarios. Our working prototype has been tested with a user study to evaluate spatial presence, co-presence, and user satisfaction during telecollaboration. Possible applications of AVT are identified and proposed to guide future usage.

Modern volumetric projection-based telepresence approaches are capable of providing realistic full-size virtual representations of remote people. Interacting with full-size people may not be desirable due to the spatial constraints of the physical environment, application context, or display technology. However, the miniaturization of remote people is known to create an eye gaze matching problem. Eye-contact is essential to communication as it allows for people to use natural nonverbal cues and improves the sense of “being there”. In this paper we discuss the design space for interacting with volumetric representations of people and present an approach for dynamically manipulating scale, orientation and the position of holograms which guarantees eye-contact. We created a working augmented reality-based prototype and validated it with 14 participants.

A clear and well-documented LaTeX document is presented as an article formatted for publication by ACM in a conference proceedings or journal publication. Based on the “acmart” document class, this article presents and explains many of the common variations, as well as many of the formatting elements an author may use in the preparation of the documentation of their work.

In this paper, we present OORT, a cooling system for head-mounted displays (HMDs) that improves wearing comfort by decreasing skin temperatures of the facial areas covered by the headset. The integrated cooling system consists of an electronically controlled fan blower. The fan compartment is integrated into an hmd padding element with custom-designed air flow channels that provide cool air circulation around the covered facial regions. We report on the design and implementation of OORT as a viable way to provide thermal comfort during long-term virtual reality experiences.

Head-Mounted Displays are useful to place users in virtual reality (VR). They do this by totally occluding the physical world, including users' bodies. This can make self-awareness problematic. Indeed, researchers have shown that users' feeling of presence and spatial awareness are highly influenced by their virtual representations, and that self-embodied representations (avatars) of their anatomy can make the experience more engaging. On the other hand, recent user studies show a penchant towards a third-person view of one's own body to seemingly improve spatial awareness. However, due to its unnaturality, we argue that a third-person perspective is not as effective or convenient as a first-person view for task execution in VR. In this paper, we investigate, through a user evaluation, how these perspectives affect task performance and embodiment, focusing on navigation tasks, namely walking while avoiding obstacles. For each perspective, we also compare three different levels of realism for users' representation, specifically a stylized abstract avatar, a mesh-based generic human, and a real-time point-cloud rendering of the users' own body. Our results show that only when a third-person perspective is coupled with a realistic representation, a similar sense of embodiment and spatial awareness is felt. In all other cases, a first-person perspective is still better suited for navigation tasks, regardless of representation.

Context-aware pervasive applications can improve user experiences by tracking people in their surroundings. Such systems use multiple sensors to gather information regarding people and devices. However, when developing novel user experiences, researchers are left to building foundation code to support multiple network-connected sensors, a major hurdle to rapidly developing and testing new ideas.

We present an intelligent user interface that allows people to perform rehabilitation exercises by themselves under the offline supervision of a therapist. Every year, many people suffer injuries that require rehabilitation. This entails considerable time overheads since it requires people to perform specified exercises under the direct supervision of a therapist. Therefore it is desirable that patients continue performing exercises outside the clinic (for instance at home, thus without direct supervision), to complement in-clinic physical therapy. However, to perform rehabilitation tasks accurately, patients need appropriate feedback, as otherwise provided by a physical therapist, to ensure that these unsupervised exercises are correctly executed. Different approaches address this problem, providing feedback mechanisms to aid rehabilitation. Unfortunately, test subjects frequently report having trouble to completely understand the feedback thus provided, which makes it hard to correctly execute the prescribed movements. Worse, injuries may occur due to incorrect performance of the prescribed exercises, which severely hinders recovery. SleeveAR is a novel approach to provide real-time, active feedback, using multiple projection surfaces to provide effective visualizations. Empirical evaluation shows the effectiveness of our approach as compared to traditional video-based feedback. Our experimental results show that our intelligent UI can successfully guide subjects through an exercise prescribed (and demonstrated) by a physical therapist, with performance improvements between consecutive executions, a desirable goal to successful rehabilitation.

Travel on Virtual Environments is the simple action where a user moves from a starting point A to a target point B. Choosing an incorrect type of technique could compromise the Virtual Reality experience and cause side effects such as spatial disorientation, fatigue and cybersickness. The design of effective travelling techniques demands to be as natural as possible, thus real walking techniques presents better results, despite their physical limitations. Approaches to surpass these limitations employ techniques that provide an indirect travel metaphor such as point-steering and target-based. In fact, target-based techniques evince a reduction in fatigue and cybersickness against the point-steering techniques, even though providing less control. In this paper we investigate further effects of speed and transition on target-based techniques on factors such as comfort and cybersickness using a Head-Mounted Display setup.

Head-Mounted Displays (HMDs) and similar 3D visualization devices are becoming ubiquitous. Going a step forward, HMD see-through systems bring virtual objects to real world settings, allowing augmented reality to be used in complex engineering scenarios. Of these, optical and video see-through systems differ on how the real world is captured by the device. To provide a seamless integration of real and virtual imagery, the absolute depth and size of both virtual and real objects should match appropriately. However, these technologies are still in their early stages, each featuring different strengths and weaknesses which affect the user experience. In this work we compare optical to video see-through systems, focusing on depth perception via exocentric and egocentric methods. Our study pairs Meta Glasses, an off-the-shelf optical see-through, to a modified Oculus Rift setup with attached video-cameras, for video see-through. Results show that, with the current hardware available, the video see-through configuration provides better overall results. These experiments and our results can help interaction designers for both virtual and augmented reality conditions.