Attention deficit disorder is a frequently observed symptom in individuals with autism spectrum disorder (ASD). This condition can present significant obstacles for those affected, manifesting in challenges such as sustained focus, task completion, and the management of distractions. These issues can impede learning, social interactions, and daily functioning. This complexity of symptoms underscores the need for tailored approaches in both educational and therapeutic settings to support individuals with ASD effectively. In this study, we have expanded upon our initial virtual reality (VR) prototype, originally created for attention therapy, to conduct a detailed statistical analysis. Our objective was to precisely identify and measure any significant differences in attention-related outcomes between sessions and groups. Our study found that heart rate (HR) and electrodermal activity (EDA) were more responsive to attention shifts than temperature. The ‘Noise’ and ‘Score’ strategies significantly affected eye openness, with the ASD group showing more responsiveness. The control group had smaller pupil sizes, and the ASD group’s pupil size increased notably when switching strategies in Session 1. Distraction log data showed that both ‘Noise’ and ‘Object Opacity’ strategies influenced attention patterns, with the ‘Red Vignette’ strategy showing a significant effect only in the ASD group. The responsiveness of HR and EDA to attention shifts and the changes in pupil size could serve as valuable physiological markers to monitor and guide these interventions. These findings further support evidence that VR has positive implications for helping those with ASD, allowing for more tailored personalized interventions with meaningful impact.

It remains unknown whether the virtual agents’ implicit behaviors (e.g., the agent’s eye-gaze shifting) would also cause such an observable effect on the users’ experiences even when the users are not consciously aware of the implicit behaviors of the agents. More importantly, it is unclear whether the effect of the implicit behaviors of agents can be amplified as the agents’ group size increases. To answer these questions, we asked two participant groups ($\mathrm{N}=12$ each) to deliver a public speech in front of the virtual agents showing either an explicit (head movement) or implicit (eye movement) behavior as we manipulated the agents’ group sizes (small, medium, and large). We measured the users’ perceptions, emotions, and behavior (head and eye movements) in the study. A posterior psychophysical experiment confirmed the implicit nature of the eye-movement behavior of the virtual agents (i.e., participants’ detection rate of the eye movements was less than the chance level). The results showed that the explicit behavior of the virtual agents caused the expected participant behavior changes along with negative emotions. In contrast, the implicit behavior of the agents caused largely positive emotions, yet, without significant behavior changes of the participants. Most importantly, as the group size of the virtual agents increased, the degree of their effect on users increased in both the explicit head-movement condition and the implicit eye-movement condition.

Due to the level of immersion and differences in the user interface, there can be a very large discrepancy in the user experience between users in immersive systems and non-immersive systems when playing games together. To investigate the impact of the cross-reality experience, which refers to the asymmetric use of eXtended Reality, we aim to understand the different affordances and experiences in an asymmetric setup, where one participant uses a desktop setup with a mouse and keyboard, and one uses a virtual reality (VR) headset and controller in two different task modes, Competition or Collaboration. In our research, a pair of participants played a game in real-time, using either the VR setup or the desktop setup. In Competition mode, the two participants were asked to defeat each other. In Collaboration mode, the pair of participants played as a team and were asked to defeat a pair of AI enemies. Our results show the VR group reported a better gaming experience and perceptual responses compared to the desktop group regardless of game mode, but that the desktop group showed superior gaming performance compared to the VR group in Competition mode.

This paper investigates the influence of floor-vibration tactile feedback on immersed users. Under symmetric and asymmetric tactile sensory cue conditions, we explore how multi-user Virtual Reality (VR) experiences are impacted by these cues in terms of illusion and coherence. Based on the reported positive impact of tactile cues in solo VR experiences, we posit that if context-matched perceptual tactile feedback is exchanged between users, they will report a significantly enhanced VR experience compared to not receiving the sensory stimuli, even within the same immersive VR experience. With our custom-built, computer-controlled vibration floor, we implemented a cannonball shooting game for two physically-separated players. In the VR game, the two players shoot cannonballs to destroy their opponent’s protective wall and cannon, while the programmed floor platform generates vertical vibrations depending on the experimental condition. We used a mixed-factorial design with four conditions for each pair of participants: 1) both A and B had vibration, and 2) neither A nor B had vibration (the Symmetric group), or 3) A had vibration, but B did not, and 4) B had vibration, but A did not (the Asymmetric group). We collected subjective and objective data for variables previously shown to be related to levels of illusion, coherence, and usability, including Presence, Co-Presence, Social Presence, Plausibility Illusion, Engagement, Embodiment, Coherence, Gaming Performance, and Overall Preference. A total of 39 pairs of participants were involved in the study. We found statistically significant differences for the vibration conditions on Co-Presence, Social Presence, Engagement, and Coherence, and for the symmetric conditions on the Plausibility Illusion and Coherence, but only with trivial or small effect sizes. The results indicate that vibration provided to a pair of game players in immersive VR can significantly enhance the VR experience, but sensory symmetry does not guarantee improved gaming performance.

Physical face-to-face collaboration with someone gives a higher-quality experience compared to mediated communication options, such as a phone- or video-based chat. Participants can share rich sensory cues to multiple human senses in a physical space. Also, the perceptual sensing of the surrounding environment including other peoples' reactions can influence human communication and emotion, and thus collaborative performance. Shared spaces in virtual environments provide degraded sensory experiences because most commercial virtual reality systems typically provide only visual and audio feedback. The impact of richer, multi-sensory feedback on joint decision-making tasks in VR is still an open area of research. Two independent studies exploring this topic are presented in this paper. We implemented a multi-sensory system that delivers vision, audio, tactile, and smell feedback, and we compared the system to a typical VR system. The scenario placed two users in a virtual theme-park safari ride with a number of non-player character (NPC) passengers to simulate realistic scenarios compared to the real-world and we varied the type and complexity of NPCs reactions to participants. In Experiment 1, we provided both users with either multi-sensory or typical sensory feedback symmetrically as a between-subjects factor, and used NPC reaction type as a within-subjects factor. In Experiment 2, we provided sensory feedback asymmetrically to each user (i.e., one had multi-sensory cues and the other had typical sensory cues) as a between-subjects factor, and used NPC reaction type as a within-subjects factor. We found that the number of sensory channels and NPC reactions did not influence user perception significantly under either symmetric or asymmetric sensory feedback conditions. However, after accounting for individual personality traits (e.g., assertive, passive), as well as any existing relationship between the pairs, we found that increasing the number of sensory channels can significantly improve subjective responses.

This work addresses cybersickness, a major barrier to successful long-exposure immersive virtual reality (VR) experiences since user discomfort frequently leads to prematurely ending such experiences. Starting from sensory conflict theory, we posit that if a vibrating floor delivers vestibular stimuli that minimally match the vibration characteristics of a scenario, the size of the conflict between the visual and vestibular senses will be reduced and, thus, the incidence and/or severity of cybersickness will also be reduced. We integrated a custom-built, computer-controlled vibrating floor in our VR system. To evaluate the system, we implemented a realistic off-road vehicle driving simulator in which participants rode multiple laps as passengers on an off-road course. We programmed the floor to generate vertical vibrations similar to those experienced in real off-road vehicle travel. The scenario and driving conditions were designed to be cybersickness-inducing for users in both the Vibration and No-vibration conditions. We collected subjective and objective data for variables previously shown to be related to levels of cybersickness or presence. These included presence and simulator sickness questionnaires (SSQ), self-rated discomfort levels, and the physiological signals of heart rate, galvanic skin response (GSR), and pupil size. Comparing data between participants in the Vibration group (N=11) to the No-Vibration group (N=11), we found that Delta-SSQ Oculomotor response and the GSR physiological signal, both known to be positively correlated with cybersickness, were significantly lower (with large effect sizes) for the Vibration group. Other variables differed between groups in the same direction, but with trivial or small effect sizes. The results indicate that the floor vibration significantly reduced some measures of cybersickness.

Supporting perceptual-cognitive tasks is an important part of our daily lives. We use rich, multi-sensory feedback through sight, sound, touch, smell, and taste to support better perceptual-cognitive things we do, such as sports, cooking, and searching for a location, and to increase our confidence in performing those tasks in daily life. Same with real life, the demand for perceptual-cognitive tasks exists in serious VR simulations such as surgical or safety training systems. However, in contrast to real life, VR simulations are typically limited to visual and auditory cues, while sometimes adding simple tactile feedback. This could make it difficult to make confident decisions in VR.In this paper, we investigate the effects of multi-sensory stimuli, namely visuals, audio, two types of tactile (floor vibration and wind), and smell in terms of the confidence levels on a location-matching task which requires a combination of perceptual and cognitive work inside a virtual environment. We also measured the level of presence when participants visited virtual places with different combinations of sensory feedback. Our results show that our multi-sensory VR system was superior to a typical VR system (vision and audio) in terms of the sense of presence and user preference. However, the subjective confidence levels were higher in the typical VR system.

Redirected walking (RDW) techniques provide a way to explore a virtual space that is larger than the available physical space by imperceptibly manipulating the virtual world view or motions. These manipulations may introduce conflicts between real and virtual cues (e.g., visual-vestibular conflicts), which can be disturbing when detectable by users. The empirically established detection thresholds of rotation manipulation for RDW still require a large physical tracking space and are therefore impractical for general-purpose Virtual Reality (VR) applications. We investigate Redirected Jumping (RDJ) as a new locomotion metaphor for redirection to partially address this limitation, and because jumping is a common interaction for environments like games. We investigated the detection rates for different curvature gains during RDJ. The probability of users detecting RDJ appears substantially lower than that of RDW, meaning designers can get away with greater manipulations with RDJ than with RDW. We postulate that the substantial vertical (up/down) movement present when jumping introduces increased vestibular noise compared to normal walking, thereby supporting greater rotational manipulations. Our study suggests that the potential combination of metaphors (e.g., walking and jumping) could further reduce the required physical space for locomotion in VR. We also summarize some differences in user jumping approaches and provide motion sickness measures in our study.

The decisions made by an Air Attack Supervisor (AAS) helicopter co-pilots in aerial firefighting have critical and immediate impacts. It is difficult to always make fast, high quality decisions due to the mental and physical stress being experienced. Real world training exercises have limitations such as safety, cost, time and difficulty in reproducing events, making frequent training infeasible. Virtual Reality (VR) offers new training opportunities, but it is challenging to create a virtual environment with the analogous level of stress experienced in the real-world. In this paper, we investigate the use of a multi-user, collaborative, multi-sensory (vision, audio, tactile) VR system to produce a realistic training environment for practising aerial firefighting training scenarios. We focus on a comparison between our VR training system, an equivalent real-world field training and an existing radio-only exercise currently in use, where we compare Heart-Rate Variability (HRV) and self reported stress using the Short Stress State Questionnaire (SSSQ). We conducted the study with real trainee AAS firefighters to determine the effectiveness of the system. Our results show that there were no significant differences between the VR training exercise and the real-world exercise in terms of the level of stress, measured by HRV, and no significant difference between VR and radio-only exercises, as reported by the SSSQ.

We present a study of the usability and induced workload of a new head-mounted display (HMD) which allows the wearer to more easily switch between tasks in the virtual and real world using multiple optical channels dynamically at the press of a button. Most HMDs provide full immersion by occluding the real world from the view of the user, and replacing it with a computer-generated virtual environment. One trade-off, however, is that such HMDs make it difficult to interact with real-world objects and people, or to react to events while being immersed. Tasks that are simple in the unmediated real world, such as taking a drink or responding to a text, become nearly impossible while wearing an occlusive HMD, and normally require the user to take off the HMD. To address this limitation, we propose an HMD which provides optical channels to view the near-field real world in the periphery of the HMD while still wearing the HMD by pressing a button. We call our approach Multi-channel Dynamic Immersion (MDI). We embedded transparency controllable LCD panels around the periphery of an HMD and installed a fish-eye lens in front of its forward-facing camera. We then compared MDI to three other methods of switching from working in VR to real world activities: (a) a front camera video-see-through method, (b) using dynamic LCDs only, and (c) taking off the HMD. Participants had to interrupt their work in a VR office to carry out typical real world daily activities, such as picking up a mug, responding to a phone call, or replying to a text message. Our results show no significant differences in task execution time between the four conditions. However, we observed a tendency favouring MDI as the easiest to use among the HMDs. In addition, we found a significantly higher rated ease of use for the two HMDs with controllable LCDs compared to the two occlusive HMDs. Our data also underscores that a substantial amount of time is lost when switching between work contexts in occlusive HMDs and the real world.

This study explores ways to increase comfort in Virtual Reality by minimizing cybersickness. Cybersickness is related to classical motion sickness and causes unwanted symptoms when using immersive technologies. We developed a dial interface to accurately capture momentary user cybersickness and feed this information back to the user. Using a seated VR roller coaster environment, we found that the dial is significantly positively correlated with post-immersion questionnaires and is a valid tool compared to verbal rating approaches.

A key challenge to effective storytelling using Virtual Reality (VR), such as with 360-degree videos, is how to direct user attention to important content without taking away user agency for free exploration. In this paper, we introduce the notion of an Action Unit system, composed of social cues such as head and arm movements, as a way of directing users to focus on content important for the given narrative. We applied this idea to a 360-degree VR tour, and evaluated its effects on memory, engagement, enjoyment, and cyber-sickness. The results indicate that the levels of engagement and enjoyment increased when these Action Units were applied. Users also preferred the Action Units for their diegetic aspects.

To interact as fully-tracked avatars with rich hand gestures in Virtual Reality (VR), we often need to wear a tracking suit or attach extra sensors on our bodies. User experience and performance may be impacted by the cumbersome devices and low fidelity behavior representations, especially in social scenarios where good communication is required. In this paper, we use multiple depth sensors and focus on increasing the behavioral fidelity of a participant’s virtual body representation. To investigate the impact of the depth-sensor-based avatar system (full-body tracking with hand gestures), we compared it against a controller-based avatar system (partial-body tracking with limited hand gestures). We designed a VR interview simulation for a single user to measure the effects on presence, virtual body ownership, workload, usability, and perceived self-performance. Specifically, the interview process was recorded in VR, together with all the verbal and non-verbal cues. Subjects then took a third-person view to evaluate their previous performance. Our results show that the depth-sensor-based avatar control system increased virtual body ownership and also improved the user experience. In addition, users rated their non-verbal behavior performance higher in the full-body depth-sensor-based avatar system.

We present a study of the relative effects of gradual versus instantaneous transition between one's own body and a virtual surrogate body, and between one's real-world environment and a virtual environment. The approach uses a stereo camera attached to an HMD to provide the illusions of virtual body ownership and spatial presence in VR. We conducted the study in a static environment which is similar to the traditional rubber hand experiment platform. Since our transition method is a blending scheme between real and virtual contexts, our study investigates the direct use of real-world information during the transition to increase the dominant visual illusion in a virtual space. We also investigate the use of a conceptual stage, called Limbo, which is a transition phase that evokes anticipation of the virtual world, providing a psychological link between the real and virtual before we enter a totally virtual space. Our study of the transition effect shows that the Limbo state has a significant influence in one's illusions of virtual body ownership (VBOI) and presence.