The Wizard of Oz (WoZ) method is a common and popular approach for simulating interactive systems in Human-Computer Interaction (HCI). Running such studies is demanding for researchers because the human wizard must manage human–agent interactions in real time while keeping participants safe and the interaction natural. Many WoZ systems struggle to reproduce complex agent behaviours without minimal delays or heavy workload for the moderator. We introduce AI of Oz, a framework that uses large language models to support researchers by monitoring ongoing interactions, detecting sensitive moments, and suggesting contextually appropriate responses. In a study with 20 HCI-related researchers, the system improved participants’ ability to manage interactions and maintain control compared to a version without AI support. We outline implications for WoZ research and note current limitations and future directions.

The emergence of generative artificial intelligence (GenAI), combined with immersive virtual reality (VR), enables the rapid creation of personalised virtual content from simple text prompts, holding potential for emotional support. However, most current VR systems rely on pre-authored content and limit user agency in designing emotionally meaningful experiences. We introduce OasisMind, an AI-assisted VR system that empowers users to co-create 360° environments, corresponding ambient soundscapes, and context-aware digital companions through natural language prompts. In a user study (N=24), we observed how participants constructed virtual worlds for emotionally meaningful use cases and compared their creations to validated, pre-defined VR scenes recommended by previous research. Our results indicate a subjective preference for self-created environments, while no significant differences were observed in perceived satisfaction or presence between conditions. These findings suggest that user agency contributes to the emotional resonance of virtual experiences and inform the design of future personalised companion systems.

Virtual reality (VR) is increasingly adopted for collaborative teaching and learning, enabling immersive and interactive experiences. As Artificial Intelligence (AI) tutors begin to take on roles alongside human instructors, it becomes crucial to understand how their integration influences interaction dynamics and role perception in these settings. This study investigates the role of Embodied Virtual Agents (EVAs) substituting human instructors in virtual training, specifically addressing the previously underexplored issue of EVA appearance consistency with instructors in Human-Agent Teaming (HAT). We recruited 21 participants to compare three conditions: No Agent, Shared-Appearance (SA) EVA, and Unique-Appearance (UA) EVA, where an EVA substitutes for the instructor during temporary absences. We evaluated collaboration efficiency, user perception/preference, and HAT dynamics. Our findings confirm that EVAs significantly enhance task efficiency compared to no support and reveal a key trade-off regarding appearance: SA fosters perceived continuity and trust but risks ambiguity and uncanny effects, while UA provides transparency and role clarity but may disrupt experiential coherence. These results have implications for designing dynamic HAT systems where control may shift. We discuss the benefits and limitations of each approach and offer design recommendations for future mixed-agency interfaces.

Abstract Passive haptics have long been used to enhance the user’s experience in virtual reality (VR). However, creating props to be used in a virtual environment can be a complicated and lengthy process. Current research looks to create passive haptic props based on the layout of, or objects in, the user’s real environment. However, we identify three key limitations of current research. Firstly, procedural generation introduces many unknown variables into the design process, which complicates applying such techniques to scenarios requiring knowledge of the virtual environment’s layout ahead of time. Furthermore, such techniques limit the size and dimensions of the virtual space to that of the real space. Lastly, current research necessitates pre-scanning or real-time scanning of the user’s real environment, often requiring specialist equipment and expertise, thus limiting its generalisability. This research proposes P rop O riented W orld R otation , a technique that attempts to answer the aforementioned limitations and simplify the process of adding haptic feedback to VR applications. We implemented this technique in a demonstration game and give an overview of the steps taken to apply the technique in a real context. We analysed the demonstration system’s performance and conducted an initial user evaluation in three different physical environments. While our stress test of the system’s performance highlights the necessity for certain optimisations in complex environments, our initial user feedback suggests that users experienced a stronger sense of presence and feelings of safety in our passive haptics-enhanced environment. Hence, we conclude that our proposal has the potential to enhance experiences in VR with haptic feedback.

No abstract available.

In this paper, we present a system that allows developers to add passive haptic feedback into their virtual reality applications by making use of existing physical objects in the user’s real environment. Our approach has minimal dependence on procedural generation and does not limit the virtual space to the dimensions of the physical play-area.

Supporting perceptual-cognitive tasks is an important part of our daily lives. We use rich, multi-sensory feedback through sight, sound, touch, smell, and taste to support better perceptual-cognitive things we do, such as sports, cooking, and searching for a location, and to increase our confidence in performing those tasks in daily life. Same with real life, the demand for perceptual-cognitive tasks exists in serious VR simulations such as surgical or safety training systems. However, in contrast to real life, VR simulations are typically limited to visual and auditory cues, while sometimes adding simple tactile feedback. This could make it difficult to make confident decisions in VR.In this paper, we investigate the effects of multi-sensory stimuli, namely visuals, audio, two types of tactile (floor vibration and wind), and smell in terms of the confidence levels on a location-matching task which requires a combination of perceptual and cognitive work inside a virtual environment. We also measured the level of presence when participants visited virtual places with different combinations of sensory feedback. Our results show that our multi-sensory VR system was superior to a typical VR system (vision and audio) in terms of the sense of presence and user preference. However, the subjective confidence levels were higher in the typical VR system.

Redirected walking (RDW) techniques provide a way to explore a virtual space that is larger than the available physical space by imperceptibly manipulating the virtual world view or motions. These manipulations may introduce conflicts between real and virtual cues (e.g., visual-vestibular conflicts), which can be disturbing when detectable by users. The empirically established detection thresholds of rotation manipulation for RDW still require a large physical tracking space and are therefore impractical for general-purpose Virtual Reality (VR) applications. We investigate Redirected Jumping (RDJ) as a new locomotion metaphor for redirection to partially address this limitation, and because jumping is a common interaction for environments like games. We investigated the detection rates for different curvature gains during RDJ. The probability of users detecting RDJ appears substantially lower than that of RDW, meaning designers can get away with greater manipulations with RDJ than with RDW. We postulate that the substantial vertical (up/down) movement present when jumping introduces increased vestibular noise compared to normal walking, thereby supporting greater rotational manipulations. Our study suggests that the potential combination of metaphors (e.g., walking and jumping) could further reduce the required physical space for locomotion in VR. We also summarize some differences in user jumping approaches and provide motion sickness measures in our study.

This study explores ways to increase comfort in Virtual Reality by minimizing cybersickness. Cybersickness is related to classical motion sickness and causes unwanted symptoms when using immersive technologies. We developed a dial interface to accurately capture momentary user cybersickness and feed this information back to the user. Using a seated VR roller coaster environment, we found that the dial is significantly positively correlated with post-immersion questionnaires and is a valid tool compared to verbal rating approaches.

To interact as fully-tracked avatars with rich hand gestures in Virtual Reality (VR), we often need to wear a tracking suit or attach extra sensors on our bodies. User experience and performance may be impacted by the cumbersome devices and low fidelity behavior representations, especially in social scenarios where good communication is required. In this paper, we use multiple depth sensors and focus on increasing the behavioral fidelity of a participant’s virtual body representation. To investigate the impact of the depth-sensor-based avatar system (full-body tracking with hand gestures), we compared it against a controller-based avatar system (partial-body tracking with limited hand gestures). We designed a VR interview simulation for a single user to measure the effects on presence, virtual body ownership, workload, usability, and perceived self-performance. Specifically, the interview process was recorded in VR, together with all the verbal and non-verbal cues. Subjects then took a third-person view to evaluate their previous performance. Our results show that the depth-sensor-based avatar control system increased virtual body ownership and also improved the user experience. In addition, users rated their non-verbal behavior performance higher in the full-body depth-sensor-based avatar system.

Vehicle manufacturers are increasingly using touchscreens to support driver access to controls. However, input mechanisms displayed on touchscreens lack the tactile sensations of physical controls, creating risks of greater demand for visual attention. These risks can potentially be mitigated by restoring some degree of tactile feedback to touchscreen interaction. This paper describes a study that examines whether touchscreen target selection during simulated driving is improved by overlaying the touchscreen with a see-through 3D printed stencil that allows underlying touchscreen controls to be located or guided by feel. Results showed that touchscreen targets were selected more quickly and with shorter periods of visual attention towards the touchscreen when the stencil was present than when it was absent. Subjective preferences also favoured the stencil condition. The work demonstrates the value of adding tactile feedback to touchscreen interaction, and shows that stencils are a simple and effective way to reduce attentional demands.

Situation Awareness (SA) is an essential skill in Air Attack Supervision (AAS) for aerial based wildfire firefighting. The display types used for Virtual Reality Training Systems (VRTS) afford different visual SA depending on the Field of View (FoV) as well as the sense of presence users can obtain in the virtual environment. We conducted a study with 36 participants to evaluate SA acquisition in three display types: a high-definition TV (HDTV), an Oculus Rift Head-Mounted Display (HMD) and a 270° cylindrical simulation projection display called the SimPit. We found a significant difference between the HMD and the HDTV, as well as with the SimPit and the HDTV for the three levels of SA.

Previous work on combining Virtual Reality (VR) and visual cue-exposure therapy has shown promising results that suggest its potential as a tool to support treatments for eating disorders. Visual food cues can elicit cravings in people no matter where they get their exposure from (e.g., photograph, real world, or virtual world). However, there is little work on the influence of olfactory stimuli in VR. Consequently, we investigated the effects of olfactory stimuli and VR on food cravings. In particular, we examined the hypothesis that olfactory and interaction with food in VR can further increase food cravings. The results of this study show that VR can elicit similar effects as exposure to traditional cues when compared to a neutral baseline. Furthermore, the added olfactory cues increased food cravings and the urge to eat the presented food, which also increased when participants were allowed to interact with the virtual food. In conclusion, VR was shown to have considerable potential to be a valid alternative to traditional food-exposure interventions.

The experience of Virtual Reality (VR) can lead to unwanted or wanted psychological stress reactions. Highly immersive VR games for instance utilise extreme, life-threatening, or dangerous situations to achieve those responses from their players. There is also sufficient evidence that in clinical settings and specific situations, such as fear of heights or post-traumatic stress, virtual stimuli can lead to perceived stress for clients. However, there is a gap in research targeting everyday, mild emotional stimuli, which are neither extreme nor specific and which are not presented in an immersive system. To what extent can common stimuli in a non-immersive virtual environment elicit actual stress reactions for its users? We developed a desktop VR system and evaluated it in a study with 54 participants. We could show that virtual stimuli in a common, domestic family environment led to a significant increase in perceived stress as measured by quantitative (self-reports) and qualitative (semi-structured interviews analysed with a General Inductive Approach (GIA)) responses. The results also showed that the introduction of virtual stimuli induced significantly higher levels of perceived workload and sense of presence and led to different physiological reactions. These findings have implications for the design and implementation of non-immersive VR systems.

Collaborative Virtual Environments (CVE) with co-located or remote video communication functionality require a continuous experience of social presence. If, at any stage during the experience the communication interrupts presence, then the CVE experience as a whole is affected - spatial presence is then decoupled from social presence. We present a solution to this problem by introducing the concept of a virtualized version of Google Glass™ called Virtual Glass. Virtual Glass is integrated into the CVE as a real-world metaphor for a communication device, one particularly suited for collaborative instructor-performer systems. In a study with 65 participants we demonstrated that the concept of Virtual Glass is effective, that it supports a high level of social presence and that the social presence is rated higher than a standard picture-in-picture videoconferencing approach for certain tasks.

Augmented reality (AR) has the potential to change the way therapy and rehabilitation is understood and administered. It can be used to manipulate the experience of reality, resulting in novel rehabilitative applications including but not limited to augmented mirror box (AMB) manipulations. We present a conceptual framework for the effective use of AR in a therapeutic context developed around the aspects of belief, interactivity, predictability, and decoupling. This framework is based on previous work in perception and emotion manipulation and is derived from and illustrated with a number of empirical studies. In particular, we describe how our augmented reflection technology (ART) system is able to manipulate the experience of reality in an effective way and how this demonstrates the potential of augmented environments to improve health and wellbeing.

Mixed reality rehabilitation systems and games are demonstrating potential as innovative adjunctive therapies for health professionals in their treatment of various hand and upper limb motor impairments. Unilateral motor deficits of the arm, for example, are commonly experienced post stroke. Our TheraMem system provides an augmented reality game environment that contributes to this increasingly rich area of research. We present a prototype system which “fools the brain” by visually amplifying users' hand movements — small actual hand movements lead to perceived larger movements. We validate the usability of our system in an empirical study with forty-five non-clinical participants. In addition, we present early qualitative evidence for the utility of our approach and system for stroke recovery and motor rehabilitation. Future uses of the system are considered by way of conclusion.