Contemporary museum experiences often incorporate digital media through modern technologies, such as handheld augmented reality (AR). However, these often fall short of providing a holistic visitor experience, as exhibits are still thought, designed, and experienced in isolation and fail to consider the user’s contexts (e. g., physical, social, and personal). To address this issue, we investigate leveraging a digital twin for designing and evaluating interactive exhibitions in large connected spaces through a case study: our publicly available handheld AR-based exhibition “Stayin’ Alive”. During this exhibition, we gained insights from the interaction data of 1303 visitors, post-visit interviews, as well as rich experiences and observations, based on which we identify four opportunity-challenge pairs that contribute design process insights for practitioners and a road map for future research.

This paper investigates avatar self-views in virtual reality (VR) and their impact on multi-user communication scenarios. Self-views in video conferencing have been shown to impact our self-perception and mental load, so we explore whether similar effects occur in VR, as personal and professional gatherings progressively move to virtual spaces with 3D avatars. We identify the following key design dimensions for self-representations in VR: spatiality, anchoring and size, and self-visibility. Based on these, we designed three variants (Remote Perspective View, Personal Mirror, and Miniature Avatar), which we compare to a baseline (No Additional Self-View) in a user study. Our analysis of sixteen dyads playing a word-guessing game requiring verbal and non-verbal communication (i.e., explaining and charades) in VR confirms that self-views are beneficial for communication scenarios that require expressive body language or facial expressions, as this allows monitoring the own avatar's animations. Further, the Miniature Avatar was overall preferred due to its spatiality, world-anchoring, and full self-visibility. Based on our results, we offer design recommendations for self-views covering the key design dimensions in VR.

Studies show that evoking empathy can positively affect the host society's acceptance of refugees, whose numbers are increasing due to political instability and environmental disasters. To examine how commercially available serious games designed to foster empathy with refugees are perceived by refugees and residents, we invited both groups to play two games: Path Out and Bury Me, My Love. Following the gaming session, we used semi-structured interviews, creating a safe space, with the refugees (n=6) to ask about their personal experiences. Residents (n=5) joined a focus group to facilitate a broader discussion. Our results show that the employed game mechanics, such as decisions with unclear consequences and narratives, generally evoked empathy. Yet, the residents missed a more extensive background on the circumstances to create a deeper connection with the games’ protagonists, and both groups wished for more agency, leaving them with a feeling of passivity. These aspects are crucial to take into account when designing new games to foster empathy for marginalized groups.

Rhythm is essential in early music education. To acquire rhythm abilities, the synchronisation between body and rhythm patterns is fundamental. Extensive research has shown the link between the auditory-motor system and rhythm acquisition; however, other sensorimotor systems also improve rhythm skills. Rhythm acquisition can be trained using new technologies, such as video games; however, most of these use mainly auditory stimuli to improve it. Therefore, in this work, we explore tactile stimuli to improve rhythm skills using a video game and how visual elements affect rhythm perception using different stimuli. We evaluate and compare the synchronisation of 27 participants following either vibrotactile or auditory stimuli emitted through a rhythm video game in two difficulty levels and we evaluated the user experience. Consistent with the literature, our results indicate that participants synchronised slightly better using auditory than vibrotactile stimuli and players’ performance was better when they had visual guides. A fatigue effect persisted and increased over playtime in both conditions. In the current study, usability measures indicate no significant differences. These findings suggest that vibrotactile stimuli are also suitable to train rhythm skills in a video game environment and the game-elements play an important role in players' performance.

Supporting remote collaboration through augmented reality facilitates the experience of co-presence by presenting the collaborator’s avatar in the user’s physical environment. While visual user representations are continuously researched and advanced, the audio configuration – especially in combination with different visualizations – is rarely considered. In a user study (n = 48, 24 dyads), we evaluate the combination of two visual (Simple vs. Rich Avatar) with two auditory (Mono vs. Spatial Audio) user representations, to investigate their impact on user’s overall experience, performance, and perceived social presence during collaboration. Our results show a preference for rich auditory and visual user representations, as Spatial Audio supports completion of parallel tasks and the Rich Avatar positively influence user experience. However, the Simple Avatar draws less attention, which potentially benefits task efficiency, advocating for simpler visualizations in performance-oriented settings. Our findings contribute to a deeper understanding of how visual and auditory user representation combinations impact remote collaboration in augmented reality.

Providing haptic feedback in virtual reality to make the experience more realistic has become a strong focus of research in recent years. The resulting haptic feedback systems differ greatly in their technologies, feedback possibilities, and overall realism making it challenging to compare different systems. We propose the Haptic Fidelity Framework providing the means to describe, understand and compare haptic feedback systems. The framework locates a system in the spectrum of providing realistic or abstract haptic feedback using the Haptic Fidelity dimension. It comprises 14 criteria that either describe foundational or limiting factors. A second Versatility dimension captures the current trade-off between highly realistic but application-specific and more abstract but widely applicable feedback. To validate the framework, we compared the Haptic Fidelity score to the perceived feedback realism of evaluations from 38 papers and found a strong correlation suggesting the framework accurately describes the realism of haptic feedback.

Informed by modern imaging techniques, current medical navigation systems support physicians during a variety of interventions, such as needle-based operations. During these, an abundance of information is often displayed on monitors placed in positions that are uncomfortable for the operator to view. In this article, we address these issues with the concept and prototype of a customized flexible display that is placed on the patient’s body to provide the essential information at just the right location. We present an empirical evaluation comparing the flexible display against a control condition using a standard interventional monitor setup and an additional condition that combines both. Our results show that the flexible display significantly reduces task load while improving overall usability. Furthermore, we found indications that the flexible display reduces task completion time while also observing a negative effect on accuracy, which needs to be balanced carefully.

Professionals in domains like film, theater, or architecture often rely on physical models to visualize spaces. With virtual reality (VR) new tools are available providing immersive experiences with correct perceptions of depth and scale. However, these lack the tangibility of physical models. Using tangible objects in VR can close this gap but creates the challenges of producing suitable objects and interacting with them with only the virtual objects visible. This work addresses these challenges by evaluating tangibles with three haptic fidelities: equal disc-shaped tangibles for all virtual objects, Lego-built tangibles, and 3D-printed tangibles resembling the virtual shapes. We present results from a comparative study on immersion, performance, and intuitive interaction and interviews with domain experts. The results show that 3D-printed objects perform best, but Lego offers a good trade-off between fast creation of tangibles and sufficient fidelity. The experts rate our approach as useful and would use all three versions.

Needle-guiding templates are used for a variety of minimally invasive medical interventions. While physically supporting needle placement with a grid of holes, they lack integrated information where needles need to be inserted. Physicians must manually determine the correct holes based on the output of planning software - a workflow that is error-prone and lengthy. We address these issues by embedding a display into the template using electroluminescence (EL) screen printing. The EL display is connected to planning software and illuminates the correct hole. In an empirical evaluation with physicians and researchers from the medical domain, we compare the illuminated against the conventional template as used in magnetic resonance imaging (MRI) guided prostate biopsies. Our results show that the EL display significantly improves task completion time by 51%, task load by 47% and usability by 30%.