Inspired by the concepts of diminishing reality and ad-blocking in browsers, this study investigates the perceived benefits and concerns of blocking physical, real-world content, particularly ads, through Extended Reality (XR). To understand how users perceive this concept, we first conducted a user study (n=18) with an adblocking prototype to gather initial insights. The results revealed a mixed willingness to adopt XR blockers, with participants appreciating aspects such as customizability, convenience, and privacy. Expected benefits included enhanced focus and reduced stress, while concerns centered on missing important information and increased feelings of isolation. Hence, we investigated the user acceptance of different ad-blocking visualizations through a follow-up online survey (n=120), comparing six concepts based on related work. The results indicated that the XR ad-blocker visualizations play a significant role in how and for what kinds of advertisements such a concept might be used, paving the path for future feedback-driven prototyping.

As winter sports continue to grow in popularity, the need for effective avalanche rescue training becomes increasingly critical. While effective, traditional training methods face challenges such as logistical constraints and limited accessibility. This study investigates the potential of virtual reality (VR) as an alternative approach to enhance avalanche rescue training. We conducted a comparative analysis between traditional ($\mathrm{N}=21$) and VR ($\mathrm{N}=22$) training methods, focusing on participants' knowledge retention, procedural adherence, and confidence in handling real-life scenarios and equipment. Our findings indicate that VR training offers comparable learning outcomes to traditional methods, with the added benefits of increased engagement and accessibility. However, challenges in handling equipment suggest that VR should complement, rather than replace, hands-on training. This research contributes valuable insights into integrating immersive technologies in safety education and the potential for VR to improve training effectiveness in highrisk environments.

For individuals with Social Anxiety (SA), interacting with others can be a challenging experience, a concern that extends into the virtual world. While technology has made significant strides in creating more realistic virtual human agents (VHA), the interplay of gaze and interpersonal distance when interacting with VHAs is often neglected. This paper investigates the effect of dynamic and static Gaze animations in VHAs on interpersonal distance and their relation to SA. A Bayesian analysis shows that static centered and dynamic centering gaze led participants to stand closer to VHAs than static averted and dynamic averting gaze, respectively. In the static gaze conditions, this pattern was found to be reversed in SA: participants with higher SA kept larger distances for static-centered gaze than for averted gaze VHAs. These findings update theory, elucidate how nuanced interactions with VHAs must be designed, and offer renewed guidelines for pleasant VHA interaction design.

Research on Extended Reality (XR) and Artificial Intelligence (AI) is booming, which has led to an emerging body of literature in their intersection. However, the main topics in this intersection are unclear, as are the benefits of combining XR and AI. This paper presents a scoping review that highlights how XR is applied in AI research and vice versa. We screened 2619 publications from 203 international venues published between 2017 and 2021, followed by an in-depth review of 311 papers. Based on our review, we identify five main topics at the intersection of XR and AI, showing how research at the intersection can benefit each other. Furthermore, we present a list of commonly used datasets, software, libraries, and models to help researchers interested in this intersection. Finally, we present 13 research opportunities and recommendations for future work in XR and AI research.

We propose a novel method for seamlessly identifying users by combining thermal and visible feet features. While it is known that users’ feet have unique characteristics, these have so far been underutilized for biometric identification, as observing those features often requires the removal of shoes and socks. As thermal cameras are becoming ubiquitous, we foresee a new form of identification, using feet features and heat traces to reconstruct the footprint even while wearing shoes or socks. We collected a dataset of users’ feet (N = 21), wearing three types of footwear (personal shoes, standard shoes, and socks) on three floor types (carpet, laminate, and linoleum). By combining visual and thermal features, an AUC between 91.1% and 98.9%, depending on floor type and shoe type can be achieved, with personal shoes on linoleum floor performing best. Our findings demonstrate the potential of thermal imaging for continuous and unobtrusive user identification.

Existing virtual reality (VR) authentication schemes are either slow or prone to observation attacks. We propose CueVR, a cue-based authentication scheme that is resilient against observation attacks by design since vital cues are randomly generated and only visible to the user experiencing the VR environment. We investigate three different input modalities through an in-depth usability study (N=20) and show that while authentication using CueVR is slower than the less secure baseline, it is faster than existing observation resilient cue-based schemes and VR schemes (4.151 s – 7.025 s to enter a 4-digit PIN). Our results also indicate that using the controllers’ trackpad significantly outperforms input using mid-air gestures. We conclude by discussing how visual cues can enhance the security of VR authentication while maintaining high usability. Furthermore, we show how existing real-world authentication schemes combined with VR’s unique characteristics can advance future VR authentication procedures.

In this work, we explore attacker behavior during shoulder surfing. As such behavior is often opportunistic and difficult to observe in real world settings, we leverage the capabilities of virtual reality (VR). We recruited 24 participants and observed their behavior in two virtual waiting scenarios: at a bus stop and in an open office space. In both scenarios, participants shoulder surfed private screens displaying different types of content. From the results we derive an understanding of factors influencing shoulder surfing behavior, reveal common attack patterns, and sketch a behavioral shoulder surfing model. Our work suggests directions for future research on shoulder surfing and can serve as a basis for creating novel approaches to mitigate shoulder surfing.

Augmented Reality (AR) embeds virtual content in physical spaces, including virtual agents that are known to exert a social presence on users. Existing design guidelines for AR rarely consider the social implications of an agent’s personal space (PS) and that it can impact user behavior and arousal. We report an experiment (N=54) where participants interacted with agents in an AR art gallery scenario. When participants approached six virtual agents (i.e., two males, two females, a humanoid robot, and a pillar) to ask for directions, we found that participants respected the agents’ PS and modulated interpersonal distances according to the human-like agents’ perceived gender. When participants were instructed to walk through the agents, we observed heightened skin-conductance levels that indicate physiological arousal. These results are discussed in terms of proxemic theory that result in design recommendations for implementing pervasive AR experiences with virtual agents.

The availability of consumer-grade virtual reality (VR) devices allows user studies to be conducted remotely, that is in users’ homes. In this way, diverse populations can be reached and studies using virtual reality can be conducted in settings characterized by high ecologic validity. In this study (N=21) we investigate challenges participants face as they are required to set up and calibrate a virtual reality system in their home without assistance from experimenters. This allowed us to identify key reasons why participants struggle with this task. Our findings suggest that providing illustrative instructions and additional assistance on request can notably increase the success rate of setting up a VR environment for participating in a remote study. Interestingly, we also find that it is harder to recruit participants who do not have prior VR experience to participate in remote VR studies. We derive suggestions on how to support this task based on our findings.

Existing software-based smart home privacy mechanisms are frequently indirect and cumbersome to use. We developed PriKey, a tangible privacy mechanism for smart homes that offers intuitive, device-independent, sensor-based, and user-centric privacy control. To render our concept comprehensible, we implemented a demonstration consisting of Wizard-of-Oz prototypes that show the envisioned form factor, size, and portability of our system, as well as a larger functional prototype of PriKey, which enables control of privacy-invasive sensors integrated into two exemplary smart devices, i.e., a smart speaker and a tablet.

Augmented Reality (AR) and wearable sensors offer new possibilities to expand our senses and change how we interact with the world. Sensory augmentation can be integrated into everyday activities, but controls remain a challenge for user experience. In this paper, we investigate how users can control a futuristic interface that enables in-situ magnification. We designed an interactive system to enable users to zoom in on objects up to a microscopic level and implemented a prototype using the Microsoft Hololens. In a user-study, we compared full-screen to windowed visualizations and four interaction techniques for zooming: a clicker, two types of gestures, and voice. Our results indicate that the clicker enabled users to zoom at the fastest rate and lowered cognitive load. We also found a preference for windowed views. With our work, we provide insights for future augmented vision systems.

Human vision underlies several constraints in its spatial, temporal and spectral resolution. One limitation is the visual sensation of temperatures, which can be a useful addition for human sight in education or work scenarios. This demo presents how mixed reality applications can virtually superimpose objects’ thermal properties with the physical world into a coherent environment. We present a prototype that utilizes either a smartphone or a head-mounted display to mediate thermal flux. We show the feasibility of our prototype in the context of a physics lab experiment that fosters the understanding of heat conduction for different materials. We envision thermal vision as an addition to the human sight that can be accomplished with already available technologies. Finally, we discuss how our prototype can be designed as a context-aware in-situ interface that toggles between regular and thermal vision.

Augmented Reality (AR) is gaining increasing importance in science, education, and entertainment. A fundamental characteristic of AR is blending the virtual and physical world into a coherent environment. In this paper, we examine the effect of substituting the physical components of lab experiments with tangible replicas and virtual representations. We conducted a user study with thirty participants who carried out the experiment in three different abstraction levels (original lab equipment, non-functional tangible props, virtual representation). We compared the users’ performance regarding setup time, experienced workload, quality of measurements, and concept comprehension of the learning task. We found no effect on comprehension but significant differences in setup time and quality of measures. The results indicate that substitution reduces the experiment setup duration without affecting knowledge transfer. These results help to shape future AR learning environments, and we offer insights for creating complex mixed reality learning materials.

Virtual Reality (VR) is gaining increasing importance in science, education, and entertainment. A fundamental characteristic of VR is creating presence, the experience of 'being' or 'acting', when physically situated in another place. Measuring presence is vital for VR research and development. It is typically repeatedly assessed through questionnaires completed after leaving a VR scene. Requiring participants to leave and re-enter the VR costs time and can cause disorientation. In this paper, we investigate the effect of completing presence questionnaires directly in VR. Thirty-six participants experienced two immersion levels and filled three standardized presence questionnaires in the real world or VR. We found no effect on the questionnaires' mean scores; however, we found that the variance of those measures significantly depends on the realism of the virtual scene and if the subjects had left the VR. The results indicate that, besides reducing a study's duration and reducing disorientation, completing questionnaires in VR does not change the measured presence but can increase the consistency of the variance.

While Augmented Reality (AR) technologies are becoming increasingly available, our understanding of AR is primarily limited to controlled experiments which address use at work or for entertainment. Little is known about how it could enhance everyday interaction from a user's perspective. Personal use of AR at home may improve how users' interface with information on a daily basis. Through an online survey, we investigated attitudes towards domestic AR. We further explored the opportunities for AR at home in a technology probe. We first introduced the users to AR by offering an AR experience presented through mixed reality smart glasses. We then used a tailor-made tablet application to elicit photos illustrating how users imagine future AR experiences. Finally, we conducted semi-structured interviews based on elicited photos. Our results show that users are eager to benefit from on-demand information, assistance, enhanced sensory perception, and play offered by AR across many locations at home. We contribute insights for future AR systems designed for domestic environments.

Recent work demonstrated the opportunities of thermal imaging in the development of novel interactive systems. However, the exploration is limited to controlled lab setups. Hence, little we know about how thermal imaging could be useful for a broader range of daily applications by novice users. To investigate the potential of domestication of thermal imaging, we conducted an exploration with a technology-cultural probe. Ten households (26 individuals) used a mobile thermal camera in their daily life. We collected thermal photos taken by the participants and conducted interviews after using the camera. We found that the users were excited about using thermal cameras in their everyday lives and found many practical uses for them. Our study provides insights into how novice users wish to use thermal imaging technology to augment their vision in daily setups, as well as identifying and classifying common thermal imaging use cases. Our work contributes implications for designing thermal imaging devices targeted towards novice users.

Every day people rely on navigation systems when exploring unknown urban areas. Many navigation systems use multimodal feedback like visual, auditory or tactile cues. Although other systems exist, users mostly rely on a visual navigation using their smartphone. However, a problem with visual navigation systems is that the users have to shift their attention to the navigation system and then map the instructions to the real world. We suggest using in-situ navigation instructions that are presented directly in the environment by augmenting the reality using a projector-quadcopter. Through a user study with 16 participants, we show that using in-situ instructions for navigation leads to a significantly higher ability to observe real-world points of interest. Further, the participants enjoyed following the projected navigation cues.

Entering text is one of the most common tasks when interacting with computing systems. Virtual Reality (VR) presents a challenge as neither the user's hands nor the physical input devices are directly visible. Hence, conventional desktop peripherals are very slow, imprecise, and cumbersome. We developed a apparatus that tracks the user's hands, and a physical keyboard, and visualize them in VR. In a text input study with 32 participants, we investigated the achievable text entry speed and the effect of hand representations and transparency on typing performance, workload, and presence. With our apparatus, experienced typists benefited from seeing their hands, and reach almost outside-VR performance. Inexperienced typists profited from semi-transparent hands, which enabled them to type just 5.6 WPM slower than with a regular desktop setup. We conclude that optimizing the visualization of hands in VR is important, especially for inexperienced typists, to enable a high typing performance.

We present LYRA, a modular in-flight system that enhances service and assists flight attendants during their work. LYRA enables passengers to browse and order services from their smartphones. Smart glasses and a smart shoe-clip with RFID reader module provides flight attendants with situated information. We gained first insights into how flight attendants and passengers use of the system during a long distance flight from Frankfurt to Houston.

Human vision is limited to a small band of the electromagnetic spectrum. It has been shown that extending visual perception can be beneficial, but it is unclear if this is useful for a broader range of applications. In this paper, we explore user reactions to extended visual perception. We describe the design and implementation of TriSight, a head-mounted display that allows the user to perceive the environment in the visual spectrum, the thermal spectrum, and through depth maps. In a study, we asked participants to perform everyday tasks in a created home, kitchen, office and basement with TriSight. Through analyzing videos, interviews, questionnaires and logs we chart the users' feedback towards the augmented visual perception. Our findings, imply the positive impact of extended vision, as well as the potential of thermal vision as alternative perception mode in performing daily tasks. Finally, we discuss the implications for designing devices that extend human's visual perception beyond the visual spectrum.

We present VRHapticDrones, a system utilizing quadcopters as levitating haptic feedback proxy. A touchable surface is attached to the side of the quadcopters to provide unintrusive, flexible, and programmable haptic feedback in virtual reality. Since the users' sense of presence in virtual reality is a crucial factor for the overall user experience, our system simulates haptic feedback of virtual objects. Quadcopters are dynamically positioned to provide haptic feedback relative to the physical interaction space of the user. In a first user study, we demonstrate that haptic feedback provided by VRHapticDrones significantly increases users' sense of presence compared to vibrotactile controllers and interactions without additional haptic feedback. In a second user study, we explored the quality of induced feedback regarding the expected feeling of different objects. Results show that VRHapticDrones is best suited to simulate objects that are expected to feel either light-weight or have yielding surfaces. With VRHapticDrones we contribute a solution to provide unintrusive and flexible feedback as well as insights for future VR haptic feedback systems.

Recent advances in technology and miniaturization allow the building of self-levitating tangible interfaces. This includes flying tangibles, which extend the mid-air interaction space from 2D to 3D. While a number of theoretical concepts about interaction with levitating tangibles were previously investigated by various researchers, a user-centered evaluation of the presented interaction modalities has attracted only minor attention from prior research. We present Flyables, a system adjusting flying tangibles in 3D space to enable interaction between users and levitating tangibles. Interaction concepts were evaluated in a user study(N=17), using quadcopters as operable levitating tangibles. Three different interaction modalities are evaluated to collect quantitative data and qualitative feedback. Our findings show preferred user interaction modalities using Flyables. We conclude our work with a discussion and future research within the domain of human-drone interaction.

Rendering the user's body in virtual reality increases immersion and presence the illusion of "being there". Recent technology enables determining the pose and position of the hands to render them accordingly while interacting within the virtual environment. Virtual reality applications often use realistic male or female hands, mimic robotic hands, or cartoon hands. However, it is unclear how users perceive different hand styles. We conducted a study with 14 male and 14 female participants in virtual reality to investigate the effect of gender on the perception of six different hands. Quantitative and qualitative results show that women perceive lower levels of presence while using male avatar hands and male perceive lower levels of presence using non-human avatar hands. While women dislike male hands, men accept and feel presence with avatar hands of both genders. Our results highlight the importance of considering the users' diversity when designing virtual reality experiences.

Even though, three-dimensional representations of architectural models exist, experiencing these models like one would experience a fully constructed building is still a major challenge. With Virtual Reality (VR) it is now possible to experience a number of scenarios in a virtual environment. Also prototyping interactive architecture elements, which might be very expensive, becomes possible. Thus, researchers and designers can already start to define user interfaces for interactive architectural elements, before they were even built. However, it is still an open question how exactly VR technologies can support experiencing interactive architecture. To answer this question, we compared experiencing three-dimensional architectural models and interactive architectural elements through a 2D screen & mouse+keyboard navigation, an head mounted display (HMD) & keyboard navigation, and an HMD & walking for navigation. The results of our study show challenges and opportunities regarding the immersion of these experiences.

Flying drones have the potential to act as navigation guides for pedestrians, providing more direct guidance than the use of handheld devices. Rather than equipping a drone with a display or indicators, we explore the potential for the drone's movements to communicate the route to the walker. For example, should the drone maintain a constant distance a few meters in front of the pedestrian, or should it position itself further along the navigation route, acting as a beacon to walk towards? We created a set of flying drone gestures and evaluated them in an online survey (n = 100) and an in-the-wild user test (n = 10) where participants were guided on a walking route by a flying drone. As a result, we propose an initial set of drone gestures for pedestrian navigation and provide further design recommendations.

Searching for misplaced keys, phones, or wallets is a common nuisance. Find My Stuff (FiMS) provides search support for physical objects inside furniture, on room level, and in multiple locations, e.g., home and office. Stuff tags make objects searchable while all other localization components are integrated into furniture. FiMS requires minimal configuration and automatically adapts to the user's furniture arrangement. Object search is supported with relative position cues, such as "phone is inside top drawer" or "the wallet is between couch and table," which do not require exact object localization. Functional evaluation of our prototype shows the approach's practicality with sufficient accuracy in realistic environments and low energy consumption. We also conducted two user studies, which showed that objects can be retrieved significantly faster with FiMS than manual search and that our relative position cues provide better support than map-based cues. Combined with audiovisual feedback, FiMS also outperforms spotlight-based cues.

Searching for lost keys, wallets or mobile phones is a common nuisance. Compared to digital information, search support for physical objects is very limited. We propose Find My Stuff (FiMS) as a search engine for physical objects. We built a fully functional Arduino-based prototype. FiMS offers the users a simple search interface to locate tagged physical items in different indoor environments. A hierarchical search process ensures energy efficient and effective searches. Instead of a fixed search infrastructure, the localization system is based on SmartFurniture equipped with RFID readers and ZigBee modules. Search results provide intuitive search cues based on relative positioning to support users in the physical retrieval of their lost objects. The system requires no manual calibration and is robust against rearrangement of SmartFurniture. Safety mechanisms prevent abuse of the system and protect user privacy.