Perspective-taking supports empathy, bias reduction, and social cognition, and virtual reality (VR) promises to facilitate it by immersing users into simulated perspectives. Yet, how VR qualities, such as spatial presence and agency, influence – and potentially enhance – perspective-taking remains unclear. We conducted two controlled experiments (N=23 and N=25) using a VR paradigm that manipulated spatial presence (2D vs. 3D) and avatar agency (synchronous vs. delayed motion, and with vs. without a task). We measured cognitive perspective-taking using an anchoring task and self-avatar overlap as an indicator of affective perspective-taking. Results show that increased spatial presence and agency both significantly enhanced affective perspective-taking. However, neither significantly affected cognitive perspective-taking. These findings suggest that while VR qualities can enhance how close a person feels to an avatar, they may not strongly affect cognitive perspective-taking. By studying the influence of two VR qualities, we offer guidance on building more effective perspective-taking experiences in immersive environments.

Despite decades of research, behavior change technologies remain limited in effectiveness because they mainly focus on offering information. Immersive technologies, such as extended, virtual, and augmented reality, promise to address this limitation by shifting from information-centric to experience-centric interventions. However, we do not know which technical and psychological components of immersive technologies make interventions effective. This scoping review of 53 articles analyzes how immersive technology components are linked to theory-informed behavior change techniques, summarizes their effectiveness, and synthesizes their unique advantages for experience-based immersive behavior change interventions. We offer both a formalization of the impact of immersive technology components on behavior change, and summarize practical suggestions for designing and systematically evaluating immersive behavior change interventions in a framework toward theory-driven Extended Reality behavior change interventions.

Sensor-based methods such as eye tracking are promising for user modeling in adaptive educational systems, as they enable continuous, fine-grained assessment of learners’ learning processes. However, implementing gaze-based user models requires selecting from a large set of available gaze measures, posing a challenge for adaptive-system designers. To inform this design process, we conduct an exploratory analysis of the associations between gaze measures, learner engagement, and learning outcomes in a video-based learning context. An analysis of data collected from 18 learners indicates that fixation dispersion and fixation rate on active areas of interest are indicators of engagement, while pupil diameter is an indicator of learning outcomes. To support further research on gaze-based user modeling and adaptation, we introduce EnGazement, a dataset comprising over 1.3 million processed eye-tracking data points, enriched with subjective and objective measures of engagement and learning.

In virtual reality (VR), users typically control one virtual body — their avatar. Previous works enable users to act through multiple avatars simultaneously. These works, though, lack a systematic account of why users want to interact with multiple avatars and do not explain how users can manipulate and generate avatars. To address this, we run six workshops with 12 VR experts and develop a design space that captures four fundamental dimensions for acting through multiple avatars in VR: Appearance , Context , User-Avatar Mapping , and Motion Control . Researchers can use the design space to generate novel interaction opportunities involving multiple avatars or analyze existing work. We then run a usability study with 17 participants to understand the practicalities of an interface that integrates parts of the design space, which reveals conceptual and technical challenges that we address through design recommendations. • Our design space explains how users can interact through multiple avatars in VR. • Users want multiple avatars to stand in for themselves, other persons, or things. • Users use multiple avatars to perceive and act in multiple contexts simultaneously. • Challenges with multiple avatars include clutter, syncing, and switching between them.

Generative AI in Virtual Reality offers the potential for collaborative object-building, yet challenges remain in aligning AI contributions with user expectations. In particular, users often struggle to understand and collaborate with AI when its actions are not transparently represented. This paper thus explores the co-creative object-building process through a Wizard-of-Oz study, focusing on how AI can effectively convey its intent to users during object customization in Virtual Reality. Inspired by human-to-human collaboration, we focus on three representation modes: the presence of an embodied avatar, whether the AI's contributions are visualized immediately or incrementally, and whether the areas modified are highlighted in advance. The findings provide insights into how these factors affect user perception and interaction with object-generating AI tools in Virtual Reality as well as satisfaction and ownership of the created objects. The results offer design implications for co-creative world-building systems, aiming to foster more effective and satisfying collaborations between humans and AI in Virtual Reality.

Augmented Reality (AR) excels at altering what we see but non-visual sensations are difficult to augment. To augment non-visual sensations in AR, we draw on the visual language of comic books. Synthesizing comic studies, we create a design space describing how to use comic elements (e.g., onomatopoeia) to depict non-visual sensations (e.g., hearing). To demonstrate this design space, we built eight demos, such as speed lines to make a user think they are faster and smell lines to make a scent seem stronger. We evaluate these elements in a qualitative user study (N=20) where participants performed everyday tasks with comic elements added as augmentations. All participants stated feeling a change in perception for at least one sensation, with perceived changes detected by between four participants (touch) and 15 participants (hearing). The elements also had positive effects on emotion and user experience, even when participants did not feel changes in perception.

Gaze interaction is a promising interaction method to increase variety, challenge, and fun in games.We present “Shed Some Fear”, a 2D platform game including numerous eye-gaze-based interactions. “Shed Some Fear” includes control with eye-gaze and traditional keyboard input. The eye-gaze interactions are partially based on eye exercises reducing digital eye strain but also on employing peripheral vision. By employing eye-gaze as a necessary input mechanism, we explore the effects on and tradeoffs between user enjoyment and digital eye strain in a five-day longitudinal between-subject study (N=17) compared to interaction with a traditional mouse. We found that perceived competence was significantly higher with eye gaze interaction and significantly higher internal eye strain. With this work, we contribute to the not straightforward inclusion of eye tracking as a useful and fun input method for games.

Walking-based locomotion techniques in virtual reality (VR) can use redirection to enable walking in a virtual environment larger than the physical one. This results in a mismatch between the perceived virtual and physical movement, which is known to cause VR sickness. However, it is unclear if different types of walking techniques (e.g., resetting, reorientation, or self-overlapping spaces) affect VR sickness differently. To address this, we conducted a systematic review and meta-analysis of 96 papers published in 2016–2022 that measure VR sickness in walking-based locomotion. We find different VR sickness effects between types of redirection and between normal walking and redirection. However, we also identified several problems with the use and reporting of VR sickness measures. We discuss the challenges in understanding VR sickness differences between walking techniques and present guidelines for measuring VR sickness in locomotion studies.

Environments can affect mood, motivation, and productivity. Green spaces, for example, are known to have calming effects on people’s moods. In virtual reality (VR), we could take advantage of these effects, as we have full visual control over the environment. In this paper, we explore how such potential effects caused by the environment impact performance and user experience (UX) when playing an exergame. We created four environments differing in their level of detail and visual realism.: (1) a white room, (2) outer space, (3) an abstract space, and (4) a forest environment. In a user study (N=26) in which participants played an exergame in all four environments, we found evidence that VEs influence enjoyment and performance. The simulation of green spaces or abstract VEs with enjoyable background sounds has a particularly positive impact. We discuss how environmental features impact performance and UX and present promising avenues for future work investigating specific parts of environmental features.

Systematic reviews (SRs) are vital to gathering and structuring knowledge, yet descriptions of their procedures are often inadequate. In human–computer interaction (HCI), SRs are still uncommon but gaining momentum, which prompted us to explore how SRs are reported at CHI—the flagship HCI conference venue. To assess the reporting quality of CHI reviews that aim for a systematic approach, we conducted an umbrella review and applied reporting guidelines for SRs (PRISMA and ENTREQ) to our corpus. We contribute the first exploration of how well SRs at CHI meet guidelines for reporting quality, showcasing strategies for improvement in reporting and conducting SRs especially in the domains of appraisal, synthesis, and documentation (i.e., protocol development). Finally, we present guiding questions for HCI researchers and practitioners for reporting SRs, as well as suggestions for best practices.

Research on Extended Reality (XR) and Artificial Intelligence (AI) is booming, which has led to an emerging body of literature in their intersection. However, the main topics in this intersection are unclear, as are the benefits of combining XR and AI. This paper presents a scoping review that highlights how XR is applied in AI research and vice versa. We screened 2619 publications from 203 international venues published between 2017 and 2021, followed by an in-depth review of 311 papers. Based on our review, we identify five main topics at the intersection of XR and AI, showing how research at the intersection can benefit each other. Furthermore, we present a list of commonly used datasets, software, libraries, and models to help researchers interested in this intersection. Finally, we present 13 research opportunities and recommendations for future work in XR and AI research.

Tile-based locomotion (TBL) is a popular locomotion technique for computer, console, and board games. However, despite its simplicity and unconventional movement, the transfer of TBL to virtual reality (VR) as a game platform remains unexplored. To fill this gap, we introduce TBL for VR on the example of two techniques: a controller and a feet-based one. In a first user study, we evaluated the usability and acceptance of the techniques compared to teleportation and touchpad locomotion. In a second exploratory user study, we evaluated the user experience of both TBL techniques in a maze and a museum scenario. The findings show that both techniques provide enjoyment and acceptable usability by creating either a relaxing (controller-based) or a physically active (feet-based) solution. Finally, our results highlight that TBL techniques work particularly well for small, constrained spaces that allow users to focus on exploring details in the nearby environment (important for games) in contrast to large open spaces that require faster locomotion, like teleportation.

Interacting with a group of people requires to direct the attention of the whole group, thus requires feedback about the crowd’s attention. In face-to-face interactions, head and eye movements serve as indicator for crowd attention. However, when interacting online, such indicators are not available. To substitute this information, gaze visualizations were adapted for a crowd scenario. We developed, implemented, and evaluated four types of visualizations of crowd attention in an online study with 72 participants using lecture videos enriched with audience’s gazes. All participants reported increased connectedness to the audience, especially for visualizations depicting the whole distribution of gaze including spatial information. Visualizations avoiding spatial overlay by depicting only the variability were regarded as less helpful, for real-time as well as for retrospective analyses of lectures. Improving our visualizations of crowd attention has the potential for a broad variety of applications, in all kinds of social interaction and communication in groups.

Webcam-based eye-tracking promises easy and quick data collection without the need for specific or additional eye-tracking hardware. This makes it especially attractive for educational research, in particular for modern formats, such as MOOCs. However, in order to fulfill its promises, webcam-based eye tracking has to overcome several challenges, most importantly, varying spatial and temporal resolutions. Another challenge that the educational domain faces especially, is that typically individual students are of interest in contrast to average values. In this paper, we explore whether an attention measure that is based on eye movement synchronicity of a group of students can be applied with unreliably-sampled data. Doing so we aim to reproduce earlier work that showed that, on average, eye movement synchronicity can predict performance in a comprehension quiz. We were not able to reproduce the findings with unreliably-sampled data, which highlights the challenges that lie ahead of webcam-based eye tracking in practice.

Digital eye strain (DES), caused by prolonged exposure to digital screens, stresses the visual system and negatively affects users’ well-being and productivity. While DES is well-studied in computer displays, its impact on users of virtual reality (VR) head-mounted displays (HMDs) is largely unexplored—despite that some of their key properties (e.g., the vergence-accommodation conflict) make VR-HMDs particularly prone. This work provides the first comprehensive investigation into DES in VR HMDs. We present results from a survey with 68 experienced users to understand DES symptoms in VR-HMDs. To help address DES, we investigate eye exercises resulting from survey answers and blue light filtering in three user studies (N = 71). Results demonstrate that eye exercises, but not blue light filtering, can effectively reduce DES. We conclude with an extensive analysis of the user studies and condense our findings in 10 key challenges that guide future work in this emerging research area.

Based on a systematic literature review of more than 300 papers published over the last 10 years, we provide indicators that the simulator sickness questionnaire (SSQ) is extensively used and widely accepted as a general discomfort measure in virtual reality (VR) research – although it actually only accounts for one category of symptoms. This results in important other categories (digital eye strain (DES) and ergonomics) being largely neglected. To contribute to a more comprehensive picture of discomfort in VR head-mounted displays, we further conducted an online study (N=352) on the severity and relevance of all three symptom categories. Most importantly, our results reveal that symptoms of simulator sickness are significantly less severe and of lower prevalence than those of DES and ergonomics. In light of these findings, we critically discuss the current use of SSQ as the only discomfort measure and propose a more comprehensive factor model that also includes DES and ergonomics.

Augmented Reality (AR) glasses equip users with the tools to modify the visual appearance of their surrounding environment. This might severely impact interpersonal communication, as the conversational partners will no longer share the same visual perception of reality. Grounded in color-in-context theory, we present a potential AR application scenario in which users can modify the color of the environment to achieve subconscious benefits. In a consecutive online survey (N=64), we measured the user’s comfort, acceptance of altering and being altered, and how it is impacted by being able to perceive or not perceive the alteration. We identified significant differences depending on (1) who or what is the target of the alteration, (2) which body part is altered, and (3) which relationship the conversational partners share. In light of our quantitative and qualitative findings, we discuss ethical and practical implications for future devices and applications that employ visual alterations.

Display-based interfaces pose high demands on users’ eyes that can cause severe vision and eye problems, also known as digital eye strain (DES). Although these problems can become even more severe if the eyes are actively used for interaction, prior work on gaze-based interfaces has largely neglected these risks. We offer the first comprehensive account of DES in gaze-based interactive systems that is specifically geared to gaze interaction designers. Through an extensive survey of more than 400 papers published over the last 46 years, we first discuss the current role of DES in interactive systems. One key finding is that DES is only rarely considered when evaluating novel gaze interfaces and neglected in discussions of usability. We identify the main causes and solutions to DES and derive recommendations for interaction designers on how to guide future research on evaluating and alleviating DES.

Augmented and virtual reality (AR/VR) has entered the mass market and, with it, will soon eye tracking as a core technology for next generation head-mounted displays (HMDs). In contrast to existing gaze interfaces, the 3D nature of AR and VR requires estimating a user's gaze in 3D. While first applications, such as foveated rendering, hint at the compelling potential of combining HMDs and gaze, a systematic analysis is missing. To fill this gap, we present the first design space for gaze interaction on HMDs. Our design space covers human depth perception and technical requirements in two dimensions aiming to identify challenges and opportunities for interaction design. As such, our design space provides a comprehensive overview and serves as an important guideline for researchers and practitioners working on gaze interaction on HMDs. We further demonstrate how our design space is used in practice by presenting two interactive applications: EyeHealth and XRay-Vision.

Current approaches for locomotion in virtual reality are either creating a visual-vestibular conflict, which is assumed to cause simulator sickness, or use metaphors such as teleportation to travel longer distances, lacking the perception of self motion. We propose VRSpinning, a seated locomotion approach based around stimulating the user's vestibular system using a rotational impulse to induce the perception of linear self-motion. In a first study we explored the approach of oscillating the chair in different frequencies during visual forward motion and collected user preferences on applying these feedback types. In a second user study we used short bursts of rotational acceleration to match the visual forward acceleration. We found that this rotational stimulus significantly reduced simulator sickness and increased the perception of self-motion in comparison to no physical motion.

Redirected walking (RDW) allows virtual reality (VR) users to walk infinitely while staying inside a finite physical space through subtle shifts (gains) of the scene to redirect them back inside the volume. All prior approaches measure the feasibility of RDW techniques based on if the user perceives the manipulation, leading to rather small applicable gains. However, we treat RDW as an interaction technique and therefore use visually perceivable gains instead of using the perception of manipulation. We revisited prior experiments with focus on applied gains and additionally tested higher gains on the basis of applicability in a user study. We found that users accept curvature gains up to 20°/m, which reduces the necessary physical volume down to approximately 6x6m for virtually walking infinitely straight ahead. Our findings strife to rethink the usage of redirection from being unperceived to being applicable and natural.