In multilingual conferences, translation support should not compromise non‑verbal cues or social interaction. Prior work on eXtended Reality (XR) subtitles aids comprehension but rarely examines translation latency. We conducted a VR-simulated conference, testing latencies of 0, 1.5, 3, 4.5, and 6 seconds to measure overall comprehension and attribution of verbal and non‑verbal information. Results showed that latencies beyond 3 seconds significantly increased subjective difficulty and affected accuracy, while shorter latencies showed no significant effects. Furthermore, participants noted that very low delay drew attention to subtitles, reducing opportunities to observe the speaker. Guided by these insights, we designed and evaluated four VR subtitle interfaces, including one traditional and three novel designs. Across delay conditions, Merged Subtitles improved opportunities to observe the speaker and resulted in better emotion attribution and user experience than other designs. We also proposed design guidelines for XR subtitle interfaces based on different levels of translation latency.

The recent emergence and popularity of consumer-grade augmented reality (AR) glasses from major technology companies highlight their potential to become the next daily computing platform. A dominant design trend in this context is the integration of a front-facing camera to deliver a first-person perspective. While this approach is intuitive, there is limited evidence that it is optimal (or sufficient) for supporting users in daily tasks. This paper explores a more effective camera interaction technique for AR glasses, which we term "FlexiCamAR". This novel method aims to enhance both efficiency and the range of applications for AR glasses by offering flexible and comfortable secondary camera viewpoints. To investigate the applicability and usability of this approach, we developed a ring camera prototype that can be attached to users’ fingers. We then conducted a user study with 12 participants, comparing FlexiCamAR against the baseline, a traditional front-facing AR camera setup, across two common tasks: taking photos and scanning QR codes. Our findings show that FlexiCamAR significantly reduces physical load. We also explore potential scenarios where the additional viewpoint afforded by FlexiCamAR proves valuable, such as capturing low-angle perspectives or navigating confined spaces. Participant feedback further suggests strong potential for additional applications, including selfie taking, video conferencing, and object scanning. Overall, FlexiCamAR presents a novel interaction approach that can serve as a powerful supplement or alternative to the first-person perspective, significantly improving the adaptability of AR glasses for everyday use.

The Video See-Through (VST) technology in Extended Reality Head-Mounted Displays (XR HMDs) allows users to perceive the real-world environment while viewing seamlessly integrated virtual content in passthrough mode. This technology has demonstrated significant potential across domains such as industry, healthcare, and intelligent driving. However, the camera of current VST technologies often struggles with visual fidelity, especially in low-light environments. To address this challenge, we introduce NoctuaXR, a lightweight, low-cost, environment-aware enhancement that dynamically adjusts the pixel-level brightness and saturation of the VST view in real time based on ambient light conditions, significantly improving visual quality in low-light scenarios. In a 10 lx environment, which is a very dim level of light, our tests show that NoctuaXR improves the color accuracy of the Baseline by 68.59% and outperforms the current leading, professional XR HMD, the Varjo XR-4 Focal Edition, by 64.26%. Furthermore, a comprehensive evaluation, including a perceptual quality evaluation and a daily task performance evaluation, confirms NoctuaXR's effectiveness. Our method provides users with a significantly improved visual experience, elevating their task efficiency to a level comparable to the Varjo XR-4. We also present compelling real-world use cases-from in-flight entertainment to navigating a dark room without disturbing others-that highlight NoctuaXR's tangible value in making XR technology safer, more convenient, and truly practical for daily life.

High latency is often introduced due to limited computational capabilities and high hardware demands. It has proven to significantly impair user performance in target selection, a fundamental interaction task. Existing research has established that latency negatively impacts selection times and success rates in 2D interactive systems; however, the underlying behavioral mechanisms remain unclear. This article investigates the effects of latency on selection times, success rates, and endpoint distributions in Virtual Reality (VR) with controller-based raycasting and bare-hand direct touch-the two most common selection methods. Our results from a user study (N = 31) revealed distinct patterns between the two methods, leading to two novel mathematical models that account for latency, target width, and movement amplitude. These two models were validated via a new dataset collected from a second user study (N = 16) and were demonstrated to outperform the existing models. Our findings provide actionable recommendations to mitigate the negative impacts of latency and improve user experience in VR interface design.

Current voice assistants present messages in a predefined format without considering users’ mental states. This paper presents an optimization-based approach to alleviate this issue which adjusts the level of details and speech speed of the voice messages according to the estimated cognitive load of the user. In the first user study (N = 12), we investigated the impact of cognitive load on user performance. The findings reveal significant differences in preferred message formats across five cognitive load levels, substantiating the need for voice message adaptation. We then implemented AdaptiveVoice, an algorithm based on combinatorial optimization to generate adaptive voice messages in real time. In the second user study (N = 30) conducted in a VR-simulated driving environment, we compare AdaptiveVoice with a fixed format baseline, with and without visual guidance on the Heads-up display (HUD). Results indicate that users benefit from AdaptiveVoice with reduced response time and improved driving performance, particularly when it is augmented with HUD.

Users' perceived image quality of virtual reality head-mounted displays (VR HMDs) is determined by multiple factors, including the HMD's structure, optical system, display and render resolution, and users' visual acuity (VA). Existing metrics such as pixels per degree (PPD) have limitations that prevent accurate comparison of different VR HMDs. One of the main limitations is that not all VR HMD manufacturers released the official PPD or details of their HMDs' optical systems. Without these details, developers and users cannot know the precise PPD or calculate it for a given HMD. The other issue is that the visual clarity varies with the VR environment. Our work has identified a gap in having a feasible metric that can measure the visual clarity of VR HMDs. To address this gap, we present an end-to-end and user-centric visual clarity metric, omnidirectional virtual visual acuity (OVVA), for VR HMDs. OVVA extends the physical visual acuity chart into a virtual format to measure the virtual visual acuity of an HMD's central focal area and its degradation in its noncentral area. OVVA provides a new perspective to measure visual clarity and can serve as an intuitive and accurate reference for VR applications sensitive to visual accuracy. Our results show that OVVA is a simple yet effective metric for comparing VR HMDs and environments.

Virtual reality (VR) technology has progressed rapidly and is used in various domains, particularly games. Simulator sickness (SS) still represents a significant problem for its wider adoption. The most common way to detect SS is using the simulator sickness questionnaire (SSQ). SSQ is a subjective measurement and is inadequate for real-time applications such as VR games. This research aims to develop a model to predict SS in real time using in-game characters’ movement and users’ eye motion data during gameplay in VR games. To achieve this, we designed an experiment to collect such data with three types of games. We trained a long short-term memory neural network with the eye-tracking and character movement data to predict SS. Our model can predict SS in real time with an accuracy of 83.4% for players who suffer from severe sensitivity to SS. Our results indicate that, in VR games, our model is an accurate and efficient method to predict SS in real time.

The refresh rate of virtual reality (VR) head-mounted displays (HMDs) has been growing rapidly in recent years because of the demand to provide higher frame rate content as it is often linked with a better experience. Today's HMDs come with different refresh rates ranging from 20Hz to 180Hz, which determines the actual maximum frame rate perceived by users' naked eyes. VR users and content developers often face a choice because having high frame rate content and the hardware that supports it comes with higher costs and other trade-offs (such as heavier and bulkier HMDs). Both VR users and developers can choose a suitable frame rate if they are aware of the benefits of different frame rates in user experience, performance, and simulator sickness (SS). To our knowledge, limited research on frame rate in VR HMDs is available. In this paper, we aim to fill this gap and report a study with two VR application scenarios that compared four of the most common and highest frame rates currently available (60, 90, 120, and 180 frames per second (fps)) to explore their effect on users' experience, performance, and SS symptoms. Our results show that 120fps is an important threshold for VR. After 120fps, users tend to feel lower SS symptoms without a significant negative effect on their experience. Higher frame rates (e.g., 120 and 180fps) can ensure better user performance than lower rates. Interestingly, we also found that at 60fps and when users are faced with fast-moving objects, they tend to adopt a strategy to compensate for the lack of visual details by predicting or filling the gaps to try to meet the performance needs. At higher fps, users do not need to follow this compensatory strategy to meet the fast response performance requirements.

Virtual reality (VR) head-mounted displays (HMD) have recently been used to provide an immersive, first-person vision/view in real-time for manipulating remotely-controlled unmanned ground vehicles (UGV). The teleoperation of UGV can be challenging for operators when it is done in real time. One big challenge is for operators to perceive quickly and rapidly the distance of objects that are around the UGV while it is moving. In this research, we explore the use of monoscopic and stereoscopic views and display types (immersive and non-immersive VR) for operating vehicles remotely. We conducted two user studies to explore their feasibility and advantages. Results show a significantly better performance when using an immersive display with stereoscopic view for dynamic, real-time navigation tasks that require avoiding both moving and static obstacles. The use of stereoscopic view in an immersive display in particular improved user performance and led to better usability.

The amount of interest in Virtual Reality (VR) research has significantly increased over the past few years, both in academia and industry. The release of commercial VR Head-Mounted Displays (HMDs) has been a major contributing factor. However, there is still much to be learned, especially how views and input techniques, as well as their interaction, affect the VR experience. There is little work done on First-Person Shooter (FPS) games in VR, and those few studies have focused on a single aspect of VR FPS. They either focused on the view, e.g., comparing VR to a typical 2D display or on the controller types. To the best of our knowledge, there are no studies investigating variations of 2D/3D views in HMDs, controller types, and their interactions. As such, it is challenging to distinguish findings related to the controller type from those related to the view. If a study does not control for the input method and finds that 2D displays lead to higher performance than VR, we cannot generalize the results because of the confounding variables. To understand their interaction, we propose to analyze in more depth, whether it is the view (2D vs. 3D) or the way it is controlled that gives the platforms their respective advantages. To study the effects of the 2D/3D views, we created a 2D visual technique, PlaneFrame, that was applied inside the VR headset. Our results show that the controller type can have a significant positive impact on performance, immersion, and simulator sickness when associated with a 2D view. They further our understanding of the interactions that controllers and views have and demonstrate that comparisons are highly dependent on how both factors go together. Further, through a series of three experiments, we developed a technique that can lead to a substantial performance, a good level of immersion, and can minimize the level of simulator sickness.