External Human-Machine Interfaces (eHMIs) have been proposed to enhance communication between automated vehicles (AVs) and pedestrians, with growing interest in multi-modal designs such as audio-visual eHMIs. Just as poor lighting can impair visual cues, a loud background noise may mask the auditory stimuli. However, its effects within these systems have not been examined, and little is known about how pedestrians — particularly Deaf and Hard-of-Hearing (DHH) people — perceive different types of auditory stimuli. We conducted a virtual reality study (Hearing N=25, DHH N=11) to examine the effects of background noise (quiet and loud) on auditory stimuli (baseline, bell, speech) within an audio-visual eHMI. Results revealed that: (1) Crossing experiences of DHH pedestrians significantly differ from Hearing pedestrians. (2) Loud background noise adversely affects pedestrians’ crossing experiences. (3) Providing an additional auditory eHMI (bell/speech) improves crossing experiences. We outlined four practical implications for future eHMI design and research.

External Human-Machine Interfaces (eHMIs) have been proposed to facilitate communication between Automated Vehicles (AVs) and pedestrians. However, no attention was given to Deaf and Hard-of-Hearing (DHH) people. We conducted a formative study through focus groups with 6 DHH people and 6 key stakeholders (including researchers, assistive technologists, and automotive interface designers) to compare proposed eHMIs and extract key design requirements. Subsequently, we investigated the effects of visual and auditory eHMI in a virtual reality user study with 32 participants (16 DHH). Results from our scenario suggesting that (1) DHH participants spent more time looking at the AV; (2) both visual and auditory eHMIs enhanced trust, usefulness, and perceived safety; and (3) only visual eHMIs reduced the time to step into the road, time looking at the AV, gaze time, and percentage looking at active visual eHMI components. Lastly, we provided five practical implications for making eHMI inclusive of DHH people.

Sedentary behavior is becoming increasingly prevalent in daily work and study environments. VR exergaming has emerged as a promising solution in these places of work and study. However, private spaces in these environments are not easy, and engaging in VR exergaming in public settings presents its own set of challenges (e.g., safety, social acceptance, isolation, and privacy protection). The recent development of Passthrough functionality in VR headsets allows users to maintain awareness of their surroundings, enhancing safety and convenience. Despite its potential benefits, little is known about how Passthrough could affect user performance and experience and solve the challenges of playing VR exergames in real-world public environments. To our knowledge, this work is the first to conduct a field study in an underground passageway on a university campus to explore the use of Passthrough in a realworld public environment, with a disturbance-free closed room as a baseline. Results indicate that enabling Passthrough in a public environment improves performance without compromising presence. Moreover, Passthrough can increase social acceptance, especially among individuals with higher levels of self-consciousness. These findings highlight Passthrough’s potential to encourage VR exergaming adoption in public environments, with promising implications for overall health and well-being.

Virtual reality (VR) not only allows head-mounted display (HMD) users to immerse themselves in virtual worlds but also to share them with others. When designed correctly, this shared experience can be enjoyable. However, in typical scenarios, HMD users are isolated by their devices, and non-HMD observers lack connection with the virtual world. To address this, our research investigates visually representing observers on both HMD and 2D screens to enhance shared experiences. The study, including five representation conditions, reveals that incorporating observer representation positively impacts both HMD users and observers. For how to design and represent them, our work shows that HMD users prefer methods displaying real-world visuals, while observers exhibit diverse preferences regarding being represented with real or virtual images. We provide design guidelines tailored to both displays, offering valuable insights to enhance co-located shared VR experiences for HMD users and non-HMD observers.

VR exergames offer an engaging solution to combat sedentary behavior and promote physical activity. However, challenges emerge when playing these games in shared spaces, particularly due to the presence of bystanders. VR's passthrough functionality enables players to maintain awareness of their surrounding environment while immersed in VR gaming, rendering it a promising solution to improve users' awareness of the environment. This study investigates the passthrough's impact on player performance and experiences in shared spaces, involving an experiment with 24 participants that examines Space (Office vs. Corridor) and Passthrough Function (With vs. Without). Results reveal that Passthrough enhances game performance and environmental awareness while reducing immersion. Players prefer an open area to an enclosed room, whether with or without Passthrough, finding it more socially acceptable. Additionally, Passthrough appears to encourage participation among players with higher self-consciousness, potentially alleviating their concerns about being observed by bystanders. Our findings provide valuable insights for designing VR experiences in shared spaces, underscoring the potential of VR's passthrough to enhance user experiences and promote VR adoption in these environments.

Familiarity with audiences plays a significant role in shaping individual performance and experience across various activities in everyday life. This study delves into the impact of familiarity with non-playable character (NPC) audiences on player performance and experience in virtual reality (VR) exergames. By manipulating of NPC appearance (face and body shape) and voice familiarity, we explored their effect on game performance, experience, and exertion. The findings reveal that familiar NPC audiences have a positive impact on performance, creating a more enjoyable gaming experience, and leading players to perceive less exertion. Moreover, individuals with higher levels of self-consciousness exhibit heightened sensitivity to the familiarity with NPC audiences. Our results shed light on the role of familiar NPC audiences in enhancing player experiences and provide insights for designing more engaging and personalized VR exergame environments.

It is well documented that exergames are enjoyable to play and can significantly improve older adults’ health and well-being. However, there is limited research on exploring factors affecting these users’ acceptance of such games, especially in virtual reality (VR), a relatively newer technology. This study proposes an extended version of the Technology Acceptance Model (TAM). We use variables from TAM related to older Chinese adults and specific to VR exergames to explore and confirm critical factors that could influence these users’ acceptance of such games in VR. We tested the proposed model with 51 older Chinese adults (aged 65 and above) after playing three commercial VR exergames (Beat Saber, FitXR, Dance Central). Results show that these older adults who are younger and retired and have a higher education, better financial means, and a good health condition have a more positive view of VR exergames. In addition, Perceived Usefulness, Perceived Ease of Use, and Perceived Enjoyment positively affect the intention to play VR exergames. Self-Satisfaction has a positive impact on Perceived Ease of Use and Perceived Usefulness. However, unlike previous studies, our results suggest that Facilitating Conditions have a negative effect on Perceived Ease of Use. Finally, we discuss the theoretical and practical implications of our results.

Virtual reality (VR) technology has progressed rapidly and is used in various domains, particularly games. Simulator sickness (SS) still represents a significant problem for its wider adoption. The most common way to detect SS is using the simulator sickness questionnaire (SSQ). SSQ is a subjective measurement and is inadequate for real-time applications such as VR games. This research aims to develop a model to predict SS in real time using in-game characters’ movement and users’ eye motion data during gameplay in VR games. To achieve this, we designed an experiment to collect such data with three types of games. We trained a long short-term memory neural network with the eye-tracking and character movement data to predict SS. Our model can predict SS in real time with an accuracy of 83.4% for players who suffer from severe sensitivity to SS. Our results indicate that, in VR games, our model is an accurate and efficient method to predict SS in real time.

Hand-based interaction, such as using a handheld controller or making hand gestures, has been widely adopted as the primary method for interacting with both virtual reality (VR) and augmented reality (AR) head-mounted displays (HMDs). In contrast, hands-free interaction avoids the need for users’ hands and although it can afford additional benefits, there has been limited research in exploring and evaluating hands-free techniques for these HMDs. As VR HMDs become ubiquitous, people will need to do text editing, which requires selecting text segments. Similar to hands-free interaction, text selection is underexplored. This research focuses on both, text selection via hands-free interaction. Our exploration involves a user study with 24 participants to investigate the performance, user experience, and workload of three hands-free selection mechanisms (Dwell, Blink, Voice) to complement head-based pointing. Results indicate that Blink outperforms Dwell and Voice in completion time. Users’ subjective feedback also shows that Blink is the preferred technique for text selection. This work is the first to explore handsfree interaction for text selection in VR HMDs. Our results provide a solid platform for further research in this important area.

Text selection is an essential activity in interactive systems, including virtual reality (VR) head-mounted displays (HMDs). It is useful for: sharing information across apps or platforms, highlighting and making notes while reading articles, and text editing tasks. Despite its usefulness, the space of text selection interaction is underexplored in VR HMDs. In this research, we performed a user study with 24 participants to investigate the performance and user preference of six text selection techniques (Controller+Dwell, Controller+Click, Head+Dwell, Head+Click, Hand+Dwell, Hand+Pinch). Results reveal that Head+Click is ranked first since it has excellent speedaccuracy performance (2nd fastest task completion speed with 3rd lowest total error rate), provides the best user experience, and produces a very low workload—followed by Controller+Click, which has the fastest speed and comparable experience with Head+Click, but much higher total error rate. Other methods can also be useful depending on the goals of the system or the users. As a first systematic evaluation of pointing $\times$ selection techniques for text selection in VR, the results of this work provide a strong foundation for further research in this area of growing importance to the future of VR to help it become a more ubiquitous and pervasive platform.

Uncertainty is widely acknowledged as an engaging gameplay element but rarely used in exergames. In this research, we explore the role of uncertainty in exergames and introduce three uncertain elements (false-attacks, misses, and critical hits) to an exergame. We conducted a study under two conditions (uncertain and certain), with two display types (virtual reality and large display) and across young and middle-aged adults to measure their effect on game performance, experience, and exertion. Results show that (1) our designed uncertain elements are instrumental in increasing exertion levels; (2) when playing a motion-based first-person perspective exergame, virtual reality can improve performance, while maintaining the same motion sickness level as a large display; and (3) exergames for middle-aged adults should be designed with age-related declines in mind, similar to designing for elderly adults. We also framed two design guidelines for exergames that have similar features to the game used in this research.

With rapid advances in virtual reality (VR) technology, the use of haptics has become important to allow users to feel the physical properties of virtual objects. Current research has focused mainly on either weight variation or changing the center of gravity, which limits the simulation potential and may affect the feeling of immersion. This research explores the design and development of a device that can simulate both weight and center of gravity using low-cost components. Through an iterative design process and continuous testing with users, we arrived at a final prototype, FluidWeight, a device that can be attached to a typical VR handheld controller. FluidWeight uses fluid, which is transported from a central storage to a receptacle attached to the controller. A final experiment shows that users enjoyed using it because it could help increase the sense of realism in VR applications.

Text entry is a common activity in virtual reality (VR) systems. There is a limited number of available hands-free techniques, which allow users to carry out text entry when users’ hands are busy such as holding items or hand-based devices are not available. The most used hands-free text entry technique is DwellType, where a user selects a letter by dwelling over it for a specific period. However, its performance is limited due to the fixed dwell time for each character selection. In this paper, we explore two other hands-free text entry mechanisms in VR: BlinkType and NeckType, which leverage users’ eye blinks and neck’s forward and backward movements to select letters. With a user study, we compare the performance of the two techniques with DwellType. Results show that users can achieve an average text entry rate of 13.47, 11.18 and 11.65 words per minute with BlinkType, NeckType, and DwellType, respectively. Users’ subjective feedback shows BlinkType as the preferred technique for text entry in VR.

Mid-air hand interaction has long been proposed as a ‘natural’ input method for Augmented Reality (AR) systems. Current AR Head-Mounted Displays (HMDs) have a limited area for hand-based interactions. Because of this, users may easily move their hand(s) outside this tracked area during interaction, especially in dynamic tasks (e.g., when translating an object). Compared to common midair interaction issues, such as gesture recognition, arm/hand fatigue, and unnatural ways of interacting with virtual objects (e.g., selecting a distant object), boundary awareness issues in AR devices have received little attention. In this research, we explore visual techniques for boundary awareness in AR HMDs, focusing on object translation tasks. Through a systematic formative study, we first identify the challenges that users might face when interacting with AR HMDs without any boundary awareness information (i.e., how current systems work). Based on the findings, we then propose four methods (i.e., static surfaces, dynamic surface(s), static coordinated lines, and dynamic coordinate line(s)) and evaluate them against the benchmark (i.e., baseline condition without boundary awareness) to make users aware of the tracked interaction area. Our results show that visual methods for boundary awareness can help with dynamic mid-air hand interactions in AR HMDs, but their effectiveness and application are user-dependent.

We present DMove, directional motion-based interaction for Augmented Reality (AR) Head-Mounted Displays (HMDs) that is both hands- and device-free. It uses directional walk-ing as a way to interact with virtual objects. To use DMove, a user needs to perform directional motions such as mov-ing one foot forward or backward. In this research, we first investigate the recognition accuracy of the motion direc-tions of our method and the social acceptance of this type of interactions together with users' comfort rating for each direction. We then optimize its design and conduct a sec-ond study to compare DMove in task performance and user preferences (workload, motion sickness, user experience), with two approaches-Hand interaction (Meta 2-like) and Head+Hand interaction (HoloLens-like) for menu selection tasks. Based on the results of these two studies, we provide a set of guidelines for DMove and further demonstrate two applications that utilize directional motions.

Augmented reality (AR) is on the rise with consumer-level head-mounted displays (HMDs) becoming available in recent years. Text entry is an essential activity for AR systems, but it is still relatively underexplored. Although it is possible to use a physical keyboard to enter text in AR systems, it is not the most optimal and ideal way because it confines the uses to a stationary position and within indoor environments. Instead, a virtual keyboard seems more suitable. Text entry via virtual keyboards requires a pointing method and a selection mechanism. Although there exist various combinations of pointing+selection mechanisms, it is not well understood how well suited each combination is to support fast text entry speed with low error rates and positive usability (regarding workload, user experience, motion sickness, and immersion). In this research, we perform an empirical study to investigate user preference and text entry performance of four pointing methods (Controller, Head, Hand, and Hybrid) in combination with two input mechanisms (Swype and Tap). Our research represents a first systematic investigation of these eight possible combinations. Our results show that Controller outperforms all the other device-free methods in both text entry performance and user experience. However, device-free pointing methods can be usable depending on task requirements and users' preferences and physical condition.

Head-based interactions are very handy for virtual reality (VR) head-worn display (HWD) systems. A useful head-based interaction technique could help users to interact with VR environments in a hands-free manner (i.e., without the need of a hand-held de-vice). Moreover, it can sometimes be seamlessly integrated with other input modalities to provide richer interaction possibilities. This paper explores the potential of a new approach that we call DepthMove to allow interactions that are based on head motions along the depth dimension. With DepthMove, a user can interact with a VR system proactively by moving the head perpendicular to the VR HWD forward or backward. We use two user studies to investigate, model, and optimize DepthMove by taking into con-sideration user performance, subjective response, and social ac-ceptability. The results allow us to determine the optimal and comfortable DepthMove range. We also distill recommendations that can be used to guide the design interfaces that use DepthMove for efficient and accurate interaction in VR HWD systems. A third study is conducted to demonstrate the usefulness of DepthMove relative to other techniques in four application scenarios.

Text entry is a common activity in virtual reality (VR) systems. However, there is a limited number of approaches available for mobile VR systems, where it might be inconvenient for users to carry an input device. We propose a novel hands-free text entry technique we call DepthText which leverages the acceleration sensing abilities of built-in IMU sensors of mobile VR systems. Users are able to enter text by moving their head forward. The results of a 5-day study indicate that users can achieve an average of 10.76 words per minute (wpm) on the last day with low errors. This performance is comparable to the dwell-based technique which is the most common way of entering text that is hands-free. One advantage of DepthText over the dwell-based technique is that users can have more control of the pace of selecting characters, rather than being pushed by a pre-set dwell time.

Hands-free interactions are very handy for virtual reality (VR) head-worn display (HWD) systems because they allow users to interact with VR environments without the need for a hand-held device. This paper explores the potential of a new approach that we call DepthMove to allow hands-free interactions that are based on head motions towards the depth dimension. With DepthMove, a user can interact in a VR system proactively by moving towards the depth dimension with an HWD. We present the concept and implementation of DepthMove in VR HWD systems and demonstrate its potential applications. We further discuss the advantages and limitations of using DepthMove.

In this paper, we present a case for text entry using a circular keyboard layout for mobile head-mounted displays (HMDs) that is dwell-free and does not require users to hold a dedicated input device for letter selection. To support the case, we have implemented RingText whose design is based on a circular layout with two concentric circles. The outer circle is subdivided into regions containing letters. Selection is made by using a virtual cursor controlled by the user's head movements-entering a letter region triggers a selection and moving back into the inner circle resets the selection. The design of RingText follows an iterative process, where we initially conduct one first study to investigate the optimal number of letters per region, inner circle size, and alphabet starting location. We then optimize its design by selecting the most suitable features from the first study: one letter per region, narrowing the trigger area to lower error rates, and creating candidate regions that incorporate two suggested words to appear next to the current letter region (close to the cursor) using a dynamic (rather than fixed) approach. Our second study compares text entry performance of RingText with four other hands-free techniques and the results show that RingText outperforms them. Finally, we run a third study lasting four consecutive days with 10 participants (5 novice users and 5 expert users) doing two daily sessions and the results show that RingText is quite efficient and yields a low error rate. At the end of the eighth session, the novice users can achieve a text entry speed of 11.30 WPM after 60 minutes of training while the expert (more experienced) users can reach an average text entry speed of 13.24 WPM after 90 minutes of training.

Presents corrections to author information for the paper, “RingText: Dwell-Free and Hands-Free Text Entry for Mobile Head-Mounted Displays Using Head Motions,” (Xu, W., et al), IEEE Trans. Vis. Comput. Graph., vol. 25, no. 5, pp. 1991–2001, May 2019.

Abstract Many virtual reality (VR) games are based on a first‐person perspective (1PP). There are, however, advantages in using another perspective, such as the third‐person perspective (3PP). Although there has been some research evaluating the effect of 1PP and 3PP in gameplay experiences, it is largely unexplored for VR games played via the new generation of commercial head‐mounted display systems, such as the Oculus Rift. In this research we want to shed some light on the relationship between the different perspectives, when games are played using head‐mounted display VR, and simulator sickness, enjoyment, and presence. To do so, we perform an experiment using two different perspectives (1PP and 3PP) and displays (VR and a conventional display) with a popular game. Our findings indicate that 3PP‐VR is less likely to make people have simulator sickness when compared with 1PP‐VR. However, the former is not perceived as immersive, but this might not be a problem because our data also show that presence is not mandatory for enjoyment. Also, the data suggest that there is no clear preference between 1PP‐VR and 3PP‐VR for gameplay.

We present PizzaText, a circular keyboard layout technique for text entry in virtual reality (VR) environments that uses the dual thumbsticks of a hand-held game controller. Text entry is a common activity in VR environments but remains challenging with existing techniques and keyboard layouts that is largely based on QWERTY. Our technique makes text entry simple, easy, and efficient, even for novice users. The technique uses a hand-held controller because it is still an important input device for users to interact with VR environments. To allow rapid search of characters, PizzaText divides a circle into slices and each slice contains 4 characters. To enable fast selection, the user uses the right thumbstick for traversing the slices, and the left thumbstick for choosing the letters. The design of PizzaText is based on three criteria: efficiency, learnability, and ease-of-use. In our first study, six potential layouts are considered and evaluated. The results lead to a design with 7 slices and 4 letters per slice. The final design is evaluated in a five-day study with 10 participants. The results show that novice users can achieve an average of 8.59 Words per Minute (WPM), while expert users are able to reach 15.85 WPM, with just two hours of training.