Although immersive technologies hold enormous potential for enhancing sports training, existing systems encounter notable limitations. While fully immersive virtual reality (VR) systems can address constraints of physical training setups such as limited space or equipment, full VR environments often lack rich, realistic, and multisensory feedback often required for effective sports training. To address this gap, this work investigates skill-based sport training, particularly, golf putting, across the reality-virtuality continuum. We implemented a mixed reality (MR) putting simulator that combines real equipment (putter and golf ball) with a virtual training environment. Three training conditions were realized: (i) real-world putting, constrained by limited physical space; (ii) a VR putting, in which a real putter and ball are used but represented virtually within the virtual environment (VE); (iii) an augmented virtuality (AV) putting, which reveals the real putter, ball, and users' limbs through a localized video-passthrough window, aiming to preserve users' sense of embodiment and presence while enabling simulation of an unlimited putting area. We conducted a pilot within-subjects experiment with 36 participants and found that the AV condition improved immediate putting performance, reduced task load, reduced perceived motion sickness, and increased user preference compared to the VR condition, while user's sense of presence remained comparable to VR training. These findings indicate that AV offers a pragmatic "sweet spot" on the virtuality continuum, balancing the tradeoff between sensory fidelity and training flexibility, offering valuable design insights for immersive sports training systems.

The development of radiance fields (RF), such as 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF), has revolutionized interactive photorealistic view synthesis and presents enormous opportunities for XR research and applications. However, despite the exponential growth of RF research, RF-related contributions to the XR community remain sparse. To better understand this research gap, we performed a systematic survey of current RF literature to analyze (i) how RF is envisioned for XR applications, (ii) how they have already been implemented, and (iii) the remaining research gaps. We collected 365 RF contributions related to XR from computer vision, computer graphics, robotics, multimedia, human-computer interaction, and XR communities, seeking to answer the above research questions. Among the 365 papers, we performed an analysis of 66 papers that already addressed a detailed aspect of RF research for XR. With this survey, we extended and positioned XR-specific RF research topics in the broader RF research field and provide a helpful resource for the XR community to navigate within the rapid development of RF research.

We introduce Reality Fusion, a novel robot teleoperation system that localizes, streams, projects, and merges a typical onboard depth sensor with a photorealistic, high resolution, high framerate, and wide FoV rendering of the complex remote environment represented as 3D Gaussian splats (3DGS). Our framework enables robust egocentric and exocentric robot teleoperation in immersive VR, with the 3DGS effectively extending spatial information of a depth sensor with limited FoV and balancing the trade-off between data streaming costs and data visual quality. We evaluated our framework through a user study with 24 participants, which revealed that Reality Fusion leads to significantly better user performance, situation awareness, and user preferences. To support further research and development, we provide an open-source implementation with an easy-to-replicate custom-made telepresence robot, a high-performance virtual reality 3DGS renderer, and an immersive robot control package.1

Recent advancements in Neural Radiance Fields (NeRF) provide enormous potential for a wide range of Mixed Reality (MR) applications. However, the applicability of NeRF to real-time MR systems is still largely limited by the rendering performance of NeRF. In this paper, we present a novel approach for Variable Rate Shading for Neural Radiance Fields (VRS-NeRF). In contrast to previous techniques, our approach does not require training multiple neural networks or re-training of already existing ones, but instead utilizes the raytracing properties of NeRF. This is achieved by merging rays depending on a variable shading rate, which reduces the overall number of queries to the neural network. We demonstrate the generalizability of our approach by implementing three alternative functions for the determination of the shading rate. The first method uses the gaze of users to effectively implement a foveated rendering technique in NeRF. For the other two techniques, we utilize shading rates based on edges and saliency. Based on a psychophysical experiment and multiple image-based metrics, we suggest a set of parameters for each technique, yielding an optimal tradeoff between rendering performance gain and perceived visual quality.

We present mixed reality (MR) tunneling, a novel method to balance the trade-off between limited render performance and high visual quality of video see-through head-mounted displays (VST-HMDs) through fusing images of two types of camera sensors with different resolutions and frame rates. By merging a color video stream from an external stereoscopic camera with the grayscale VST commonly integrated into today’s standalone virtual reality (VR) headsets, we create a perceptually high-resolution and wide field of view VSTHMD prototype. The external high-resolution VST displayed at the central foveal to the para-peripheral region of the human visual field complements the low-resolution, low-latency grayscale VST at the far peripheral region, producing a tunneling effect, which simulates the human foveal and peripheral vision, with the potential to reduce cybersickness as in the tunneling effect in immersive VR. We propose two extensions to the MR tunneling method. The first one accommodates the user’s head movement speed by fading out the external VST when fast head movements are detected, thus potentially compensating for video streaming latency. The second one is a foveated MR tunneling effect, which displays the center of the external VST based on the tracked user eye movements. We evaluated the three MR tunneling methods in a within-subject study with 24 participants. The user study demonstrates the potential of our prototype and techniques based on the example of an assembly task that requires hand-eye coordination, untethered locomotion, and fine motor skills. The results demonstrate that, although not significant, the MR tunneling effects lead to higher overall usability, less perceived motion sickness, and a better sense of presence.1

Nowadays, high-power and multi-spectral lasers are used in many scientific experiments and industrial processes. Those laser sources can rapidly cause permanent damage to human eyes. Research and development work with those laser sources requires typically wearing personal protective equipment (PPE), such as laser safety goggles as eye protectors. Currently, laser safety goggles are based on optical spectral filters, which block spectral bands where hazardous laser radiation is emitted. Such laser safety goggles can filter up to 99% of the visible spectrum, rendering researchers working in hazardous and complex laboratory environments visually impaired. Video see-through head-mounted displays (VST-HMD) could be used as eye protectors without reducing users’ visibility of the environment since they can be constructed such that all laser and ambient light is blocked from the human eye. To date, this application domain is still largely unexplored in the MR community. To our best knowledge, there has been no comprehensive work that investigates the human factors of such an eye protection method at an advanced optics laboratory. In this work, we present the results of an empirical study where we evaluate the usability, perceived safety, advantages, and limitations of using VST-HMDs as laser safety goggles. We use a stereoscopic VST-HMD developed through a human-centered design approach at one of the most advanced optics laboratories in the world. 18 participants including 14 laser experts evaluated the current prototype. Our user evaluation and field studies confirm that the complex and hazardous working conditions at high-energy laser laboratories could be significantly improved with MR technology.