As virtual reality (VR) headsets become more comfortable and accessible, their growing use in high-stakes, time-critical settings raises concerns about fatigue. Fatigue impairs perceptual and cognitive functioning. It reduces oculomotor accuracy and visual focus, and can lead to an early decline in depth estimation performance. By augmenting the visual presentation with depth information cues, adaptive designs can reduce fatigue-related depth perception errors, enhancing safety and task effectiveness. Explicit cues present depth information directly through text or color, while subtle cues adjust scene properties, such as depth-dependent blur, to convey depth information implicitly without drawing overt attention. We examined how fatigue interacts with different cues in a 27-hour within-subject protocol. Across six overnight sessions (20:00– 07:00), twenty-three participants completed a VR depth perception task at varying fatigue levels and under four cue conditions: no cue (baseline), text, color, and blur. Over the night, vigilance declined, sleepiness and mental effort increased, and simulator sickness rose before stabilizing, independent of cue condition. All augmented cues reduced depth estimation error relative to baseline. Text yielded the largest and most consistent accuracy gains, especially for farther targets and later sessions. At peak fatigue, response probability dipped for text but remained stable for blur and color, indicating an accuracy versus responsiveness trade-off. These results support mixed adaptive designs that default to subtle cues to preserve responsiveness at low alertness and introduce explicit overlays when precise metric information is needed.

Augmented reality displays are becoming more powerful and simultaneously more mobile. Although mobile AR is gaining popularity, it remains difficult to get an insight into users' cognitive load, despite its relevance for many mobile-based tasks. Usually, cognitive load is measured via subjective, task-disruptive self-reports such as NASA TLX. While biosensors such as galvanic skin response, heart rate variability, or pulse have been used to obtain more objective measures, these are highly susceptible to motion-induced noise. More robust techniques like EEG offer higher reliability but are impractical for mobile, real-world use. In this paper, we report on a non-contact multi-sensor approach to assess cognitive load in mobile AR. Our approach combines pupillometry, facial expression tracking, and thermal imaging for respiratory rate analysis. Within the frame of our study, we analysed the aptness of the methods, comparing load assessment for low and high cognitive load tasks under both stationary and mobile conditions. Using an XGBoost classifier, our model achieved 86.11% accuracy for binary cognitive load assessment (low vs. high cognitive load) and 84.24% accuracy for four-way classification (cognitive load$\times$mobility). Feature importance analysis revealed that robust predictors included gaze dynamics (e.g., fixation, pursuit, and saccade durations), pupil diameter metrics (such as FFT band power and variability measures), and facial and respiratory features (including brow lowering and nostril temperature quantiles) for assessing cognitive load in mobile AR.

Awe is a multifaceted emotion often associated with the perception of vastness, that challenges existing mental frameworks. Despite its growing relevance in affective computing and psychological research, awe remains difficult to elicit and measure. This raises the research questions of how awe can be effectively elicited, which factors are associated with the experience of awe, and whether it can reliably be measured using biosensors. For this study, we designed 10 immersive Virtual Reality (VR) scenes with dynamic transitions from narrow to vast environments. These scenes were used to explore how awe relates to environmental features (abstract, human-made, nature), personality traits, and country of origin. We collected skin conductance, respiration, self-reported awe and presence data from participants from Germany, Japan, and Jordan. Our results indicate that self-reported awe varies significantly across countries and scene types. In particular, a scene depicting outer space elicited the strongest awe. Scenes that elicited high selfreported awe also induced a stronger sense of presence. However, we found no evidence that awe ratings are correlated with physiological responses. These findings challenge the assumption that awe is reliably reflected in autonomic arousal and underscore the importance of cultural and perceptual context. Our study offers new insights into how immersive VR can be designed to elicit awe, and suggests that subjective reports - rather than physiological signals - remain the most consistent indicators of emotional impact.

This paper presents a comprehensive study of virtual 3D object manipulation along 4DoF on real surfaces in mixed reality (MR), using hand-based and tangible interactions. A custom cylindrical tangible proxy leverages affordances of physical knobs and tabletop support for stable input. We evaluate both modalities across isolated tasks (2DoF translation, 1DoF rotation/scaling), semi-combined (3DoF translation+rotation), and full 4DoF compound manipulation. We offer analyses of hand interactions, tangible interactions, and their comparison in MR tasks. For hand interactions, compound tasks required repetitive corrections, increasing completion times—yet surprisingly, rotation errors were smaller in compound tasks than in rotation-only tasks. Tangible interactions exhibited significantly larger errors in translation, rotation, and scaling during compound tasks compared to isolated tasks. Crucially, tangible interactions outperformed hand interactions in precision, likely due to tabletop support and constrained 4DoF design. These findings inform designers opting for hand-only interaction (highlighting trade-offs in compound tasks) and those leveraging tangibles (emphasizing precision gains despite compound-task challenges).

Change detection (CD) is critical in everyday tasks. While current algorithmic approaches for CD are improving, they remain imprecise, often requiring human intervention. Cognitive science research focuses on understanding CD mechanisms, especially through change blindness studies. However, these do not address the primary requirement in real-life CD - detecting changes as effectively as possible. Such a requirement is directly relevant to the visual comparison field - studying visualisation techniques to compare data and identify differences or changes effectively. Recent studies have used Virtual Reality (VR) to improve visual comparison by providing an immersive platform where users can interact with 3D data at a real-life scale, enhancing spatial reasoning. We believe VR could also improve CD performance accordingly. Particularly, VR offers stereoscopic depth perception over traditional displays, potentially enhancing the detection of spatial change. In this paper, we develop and analyse three 3D visual comparison techniques for CD in VR: Sliding Window, 3D Slider, and Switch Back. These techniques are evaluated under synthetic but realistic environments and frequently occurring Perceptual Challenges, including different Changed Object Size, Lighting Variation, and Scene Drift conditions. Experimental results reveal significant differences between the techniques in detection time measures and subjective user experience.

Awe is a complex emotion described as a perception of vastness and a need for accommodation to integrate new, overwhelming experiences. Virtual Reality (VR) has recently gained attention as a convenient means to facilitate experiences of awe. In VR, a first-person perspective might increase awe due to its immersive nature, while a third-person perspective might enhance the perception of vastness. However, the impact of VR perspectives on experiencing awe has not been thoroughly examined. We created two types of VR scenes: one with elements designed to induce high awe, such as a snowy mountain, and a low awe scene without such elements. We compared first-person and third-person perspectives in each scene. Forty-two participants explored the VR scenes, with their physiological responses captured by electrocardiogram (ECG) and face tracking (FT). Subsequently, participants self-reported their experience of awe (AWE-S) and presence (IPQ) within VR. The results revealed that the first-person perspective induced stronger feelings of awe and presence than the third-person perspective. The findings of this study provide useful guidelines for designing VR content that enhances emotional experiences.

Understanding an environment relies on human sensory systems, with visual perception as the primary source for defining spatial relationships and estimating distances. The visual system uses natural cues, each offering partial information that can lead to bias and conflicts, especially when ambiguous. Virtual Reality (VR) environments challenge these natural depth cues with discrepancies in perspective and variations in light and shadow depiction, leading to potential confusion in depth perception. However, VR also allows for the isolation and study of these cues. This paper introduces artificial subtle cues (texture blur) to enhance natural depth information in VR. Our results show that augmenting natural depth cues with artificial ones improves depth prediction accuracy and spatial relationship awareness. Subtle blur cues enhance depth estimation without participants’ subjective awareness of the augmentation, suggesting that such subtle cueing can effectively enhance depth perception.

Augmented Reality (AR) applications increasingly rely on eye and head gaze tracking for user interaction, with their efficacy influenced by environmental factors such as spatial arrangements and lighting conditions. This paper presents two studies that examine how these variables affect the performance of eye and head gaze tracking in AR environments. While eye tracking partially delivered faster results, its performance exhibited greater variability, especially under dynamic lighting conditions. Conversely, head gaze tracking, while providing more consistent results, showed a notable reduction in accuracy in environments with fluctuating light levels. Furthermore, the spatial properties of the environment had notable implications on both tracking methods. Our research demonstrates that both spatial properties and lighting conditions are key determinants in the choice of a tracking method, underscoring the need for AR systems that can dynamically adapt to these environmental variables.

Abstract While humans can effortlessly pick a view from multiple streams, automatically choosing the best view is a challenge. Choosing the best view from multi-camera streams poses a problem regarding which objective metrics should be considered. Existing works on view selection lack consensus about which metrics should be considered to select the best view. The literature on view selection describes diverse possible metrics. And strategies such as information-theoretic, instructional design, or aesthetics-motivated fail to incorporate all approaches. In this work, we postulate a strategy incorporating information-theoretic and instructional design-based objective metrics to select the best view from a set of views. Traditionally, information-theoretic measures have been used to find the goodness of a view, such as in 3D rendering. We adapted a similar measure known as the viewpoint entropy for real-world 2D images. Additionally, we incorporated similarity penalization to get a more accurate measure of the entropy of a view, which is one of the metrics for the best view selection. Since the choice of the best view is domain-dependent, we chose demonstration-based training scenarios as our use case. The limitation of our chosen scenarios is that they do not include collaborative training and solely feature a single trainer. To incorporate instructional design considerations, we included the trainer’s body pose, face, face when instructing, and hands visibility as metrics. To incorporate domain knowledge we included predetermined regions’ visibility as another metric. All of those metrics are taken into account to produce a parameterized view recommendation approach for demonstration-based training. An online study using recorded multi-camera video streams from a simulation environment was used to validate those metrics. Furthermore, the responses from the online study were used to optimize the view recommendation performance with a normalized discounted cumulative gain (NDCG) value of 0.912, which shows good performance with respect to matching user choices.

Physical walking is often considered the gold standard for VR travel whenever feasible. However, limited free-space walking areas in the real-world do not allow exploring larger-scale virtual environments by actual walking. Therefore, users often require handheld controllers for navigation, which can reduce believability, interfere with simultaneous interaction tasks, and exacerbate adverse effects such as motion sickness and disorientation. To investigate alternative locomotion options, we compared handheld Controller (thumbstick-based) and physical walking versus a seated (HeadJoystick) and standing/stepping (NaviBoard) leaning-based locomotion interface, where seated/standing users travel by moving their head toward the target direction. Rotations were always physically performed. To compare these interfaces, we designed a novel simultaneous locomotion and object interaction task, where users needed to keep touching the center of upward moving target balloons with their virtual lightsaber, while simultaneously staying inside a horizontally moving enclosure. Walking resulted in the best locomotion, interaction, and combined performances while the controller performed worst. Leaning-based interfaces improved user experience and performance compared to Controller, especially when standing/stepping using NaviBoard, but did not reach walking performance. That is, leaning-based interfaces HeadJoystick (sitting) and NaviBoard (standing) that provided additional physical self-motion cues compared to controller improved enjoyment, preference, spatial presence, vection intensity, motion sickness, as well as performance for locomotion, object interaction, and combined locomotion and object interaction. Our results also showed that less embodied interfaces (and in particular the controller) caused a more pronounced performance deterioration when increasing locomotion speed. Moreover, observed differences between our interfaces were not affected by repeated interface usage.

Handheld virtual reality (VR) controllers enable users to manipulate virtual objects in VR but do not convey a virtual object's weight. This hinders users from effectively experiencing lighter and heavier objects. While previous work explored either hardware-based interfaces or software-based pseudo-haptics, in this paper, we combine two techniques to improve the virtual weight perception in VR. By adapting the trigger resistance of the VR controller when grasping a virtual object and manipulating the control-display (C/D) ratio during lifting, we create a continuous weight sensation. In a psychophysical study (N=29), we compared our combined approach against the individual rendering techniques. Our results show that participants were significantly more sensitive towards smaller weight differences in the combined weight simulations compared to the individual methods. Additionally, participants were also able to determine weight differences significantly faster with both cues present compared to the single pseudo-haptic technique. While all three techniques were generally valued to be effective, the combined approach was favoured the most. Our findings demonstrate the meaningful benefit of combining physical and virtual techniques for virtual weight rendering over previously proposed methods.

Continuous locomotion in VR provides uninterrupted optical flow, which mimics real-world locomotion and supports path integration . However, optical flow limits the maximum speed and acceleration that can be effectively used without inducing cybersickness. In contrast, teleportation provides neither optical flow nor acceleration cues, and users can jump to any length without increasing cybersickness. However, teleportation cannot support continuous spatial updating and can increase disorientation. Thus, we designed 'HyperJump' in an attempt to merge benefits from continuous locomotion and teleportation. HyperJump adds iterative jumps every half a second on top of the continuous movement and was hypothesized to facilitate faster travel without compromising spatial awareness/orientation. In a user study, Participants travelled around a naturalistic virtual city with and without HyperJump (equivalent maximum speed). They followed waypoints to new landmarks, stopped near them and pointed back to all previously visited landmarks in random order. HyperJump was added to two continuous locomotion interfaces (controller- and leaning-based). Participants had better spatial awareness/orientation with leaning-based interfaces compared to controller-based (assessed via rapid pointing). With HyperJump, participants travelled significantly faster, while staying on the desired course without impairing their spatial knowledge. This provides evidence that optical flow can be effectively limited such that it facilitates faster travel without compromising spatial orientation. In future design iterations, we plan to utilize audio-visual effects to support jumping metaphors that help users better anticipate and interpret jumps, and use much larger virtual environments requiring faster speeds, where cybersickness will become increasingly prevalent and thus teleporting will become more important.

Using standard handheld interfaces for VR locomotion may not provide a believable self-motion experience and can contribute to unwanted side effects such as motion sickness, disorientation, or increased cognitive load. This paper demonstrates how using a seated leaning-based locomotion interface -HeadJoystick- in VR ground-based navigation affects user experience, usability, and performance. In three within-subject studies, we compared controller (touchpad/thumbstick) with a more embodied interface ("HeadJoystick") where users moved their head and/or leaned in the direction of desired locomotion. In both conditions, users sat on a regular office chair and used it to control virtual rotations. In the first study, 24 participants used HeadJoystick versus Controller in three complementary tasks including reach-the-target, follow-the-path, and racing (dynamic obstacle avoidance). In the second study, 18 participants repeatedly used HeadJoystick versus Controller (8 one-minute trials each) in a reach-the-target task. To evaluate potential benefits of different brake mechanisms, in the third study 18 participants were asked to stop within each target area for one second. All three studies consistently showed advantages of HeadJoystick over Controller: we observed improved performance in all tasks, as well as higher user ratings for enjoyment, spatial presence, immersion, vection intensity, usability, ease of learning, ease of use, and rated potential for daily and long-term use, while reducing motion sickness and task load. Overall, our results suggest that leaning-based interfaces such as HeadJoystick provide an interesting and more embodied alternative to handheld interfaces in driving, reach-the-target, and follow-the-path tasks, and potentially a wider range of scenarios.

Flying in virtual reality (VR) using standard handheld controllers can be cumbersome and contribute to unwanted side effects such as motion sickness and disorientation. This article investigates a novel hands-free flying interface-HeadJoystick, where the user moves their head similar to a joystick handle toward the target direction to control virtual translation velocity. The user sits on a regular office swivel chair and rotates it physically to control virtual rotation using 1:1 mapping. We evaluated short-term (Study 1) and extended usage effects through repeated usage (Study 2) of the HeadJoystick versus handheld interfaces in two within-subject studies, where participants flew through a sequence of increasingly difficult tunnels in the sky. Using the HeadJoystick instead of handheld interfaces improved both user experience and performance, in terms of accuracy, precision, ease of learning, ease of use, usability, long-term use, presence, immersion, sensation of self-motion, workload, and enjoyment in both studies. These findings demonstrate the benefits of using leaning-based interfaces for VR flying and potentially similar telepresence applications such as remote flight with quadcopter drones. From a theoretical perspective, we also show how leaning-based motion cueing interacts with full physical rotation to improve user experience and performance compared to the gamepad.

Augmented reality applications allow users to enrich their real surroundings with additional digital content. However, due to the limited field of view of augmented reality devices, it can sometimes be difficult to become aware of newly emerging information inside or outside the field of view. Typical visual conflicts like clutter and occlusion of augmentations occur and can be further aggravated especially in the context of dense information spaces. In this article, we evaluate how multisensory cue combinations can improve the awareness for moving out-of-view objects in narrow field of view augmented reality displays. We distinguish between proximity and transition cues in either visual, auditory or tactile manner. Proximity cues are intended to enhance spatial awareness of approaching out-of-view objects while transition cues inform the user that the object just entered the field of view. In study 1, user preference was determined for 6 different cue combinations via forced-choice decisions. In study 2, the 3 most preferred modes were then evaluated with respect to performance and awareness measures in a divided attention reaction task. Both studies were conducted under varying noise levels. We show that on average the Visual-Tactile combination leads to 63% and Audio-Tactile to 65% faster reactions to incoming out-of-view augmentations than their Visual-Audio counterpart, indicating a high usefulness of tactile transition cues. We further show a detrimental effect of visual and audio noise on performance when feedback included visual proximity cues. Based on these results, we make recommendations to determine which cue combination is appropriate for which application.

Walking has always been considered as the gold standard for navigation in Virtual Reality research. Though full rotation is no longer a technical challenge, physical translation is still restricted through limited tracked areas. While rotational information has been shown to be important, the benefit of the translational component is still unclear with mixed results in previous work. To address this gap, we conducted a mixed-method experiment to compare four levels of translational cues and control: none (using the trackpad of the HTC Vive controller to translate), upper-body leaning (sitting on a "NaviChair", leaning the upper-body to locomote), whole-body leaning/stepping (standing on a platform called NaviBoard, leaning the whole body or stepping one foot off the center to navigate), and full translation (physically walking). Results showed that translational cues and control had significant effects on various measures including task performance, task load, and simulator sickness. While participants performed significantly worse when they used a controller with no embodied translational cues, there was no significant difference between the NaviChair, NaviBoard, and actual walking. These results suggest that translational body-based motion cues and control from a low-cost leaning/stepping interface might provide enough sensory information for supporting spatial updating, spatial awareness, and efficient locomotion in VR, although future work will need to investigate how these results might or might not generalize to other tasks and scenarios.

This paper introduces FaceHaptics, a novel haptic display based on a robot arm attached to a head-mounted virtual reality display. It provides localized, multi-directional and movable haptic cues in the form of wind, warmth, moving and single-point touch events and water spray to dedicated parts of the face not covered by the head-mounted display.The easily extensible system, however, can principally mount any type of compact haptic actuator or object. User study 1 showed that users appreciate the directional resolution of cues, and can judge wind direction well, especially when they move their head and wind direction is adjusted dynamically to compensate for head rotations. Study 2 showed that adding FaceHaptics cues to a VR walkthrough can significantly improve user experience, presence, and emotional responses.

Telepresence robots allow people to participate in remote spaces, yet they can be difficult to manoeuvre with people and obstacles around. We designed a haptic-feedback system called "FeetBack," which users place their feet in when driving a telepresence robot. When the robot approaches people or obstacles, haptic proximity and collision feedback are provided on the respective sides of the feet, helping inform users about events that are hard to notice through the robot's camera views. We conducted two studies: one to explore the usage of FeetBack in virtual environments, another focused on real environments. We found that FeetBack can increase spatial presence in simple virtual environments. Users valued the feedback to adjust their behaviour in both types of environments, though it was sometimes too frequent or unneeded for certain situations after a period of time. These results point to the value of foot-based haptic feedback for telepresence robot systems, while also the need to design context-sensitive haptic feedback.

Current augmented reality displays still have a very limited field of view compared to the human vision. In order to localize out-of-view objects, researchers have predominantly explored visual guidance approaches to visualize information in the limited (in-view) screen space. Unfortunately, visual conflicts like cluttering or occlusion of information often arise, which can lead to search performance issues and a decreased awareness about the physical environment. In this paper, we compare an innovative non-visual guidance approach based on audio-tactile cues with the state-of-the-art visual guidance technique EyeSee360 for localizing out-of-view objects in augmented reality displays with limited field of view. In our user study, we evaluate both guidance methods in terms of search performance and situation awareness. We show that although audio-tactile guidance is generally slower than the well-performing EyeSee360 in terms of search times, it is on a par regarding the hit rate. Even more so, the audio-tactile method provides a significant improvement in situation awareness compared to the visual approach.

We present a novel, multilayer interaction approach that enables state transitions between spatially above-screen and 2D on-screen feedback layers. This approach supports the exploration of haptic features that are hard to simulate using rigid 2D screens. We accomplish this by adding a haptic layer above the screen that can be actuated and interacted with (pressed on) while the user interacts with on-screen content using pen input. The haptic layer provides variable firmness and contour feedback, while its membrane functionality affords additional tactile cues like texture feedback. Through two user studies, we look at how users can use the layer in haptic exploration tasks, showing that users can discriminate well between different firmness levels, and can perceive object contour characteristics. Demonstrated also through an art application, the results show the potential of multilayer feedback to extend on-screen feedback with additional widget, tool and surface properties, and for user guidance.

Mobile navigation apps are among the most used mobile applications and are often used as a baseline to evaluate new mobile navigation technologies in field studies. As field studies often introduce external factors that are hard to control for, we investigate how pedestrian navigation methods can be evaluated in virtual reality (VR). We present a study comparing navigation methods in real life (RL) and VR to evaluate if VR environments are a viable alternative to RL environments when it comes to testing these. In a series of studies, participants navigated a real and a virtual environment using a paper map and a navigation app on a smartphone. We measured the differences in navigation performance, task load and spatial knowledge acquisition between RL and VR. From these we formulate guidelines for the improvement of pedestrian navigation systems in VR like improved legibility for small screen devices. We furthermore discuss appropriate low-cost and low-space VR-locomotion techniques and discuss more controllable locomotion techniques.

Head-worn devices with a narrow field of view are common commodity for Augmented Reality. However, their limited screen space makes view management difficult. Especially in dense information spaces this potentially leads to visual conflicts such as overlapping labels (occlusion) and visual clutter. In this paper, we look into the potential of using audio and vibrotactile feedback to guide search and information localization. Our results indicate users can be guided with high accuracy using audio-tactile feedback with maximum median deviations of only 2° on longitude, 3.6° on latitude and 0.07 meter in depth. Regarding the encoding of latitude we found a superior performance when using audio, resulting in an improvement of 61% and fastest search times. When interpreting localization cues the maximum median deviation was 9.9° on longitude and 18% of a selected distance to be encoded which could be reduced to 14% when using audio.

Many optical-see-through displays have a relatively narrow field of view. However, a limited field of view can constrain how information can be presented and searched through. To understand these constraints, we present a series of experiments that address the interrelationships between field of view, information density, and search performance. We do so by simulating various fields of view using two approaches: limiting the field of view presented on a Microsoft HoloLens optical-see-through head-worn display and dynamically changing the portion of a large tiled-display wall on which information is presented, for head-tracked users in both cases. Our results indicate a significant effect of information density and field of view on search performance, with potential search performance benefits of using a larger FOV between ca. 7-28%. Furthermore, while grids guided visual search, they did not significantly affect performance.

In Augmented Reality (AR), search performance for outdoor tasks is an important metric for evaluating the success of a large number of AR applications. Users must be able to find content quickly, labels and indicators must not be invasive but still clearly noticeable, and the user interface should maximize search performance in a variety of conditions. To address these issues, we have set up a series of experiments to test the influence of virtual characteristics such as color, size, and leader lines on the performance of search tasks and noticeability in both real and simulated environments. We evaluate two primary areas, including 1) the effects of peripheral field of view (FOV) limitations and labeling techniques on target acquisition during outdoor mobile search, and 2) the influence of local characteristics such as color, size, and motion on text labels over dynamic backgrounds. The first experiment showed that limited FOV will severely limit search performance, but that appropriate placement of labels and leaders within the periphery can alleviate this problem without interfering with walking or decreasing user comfort. In the second experiment, we found that different types of motion are more noticeable in optical versus video see-through displays, but that blue coloration is most noticeable in both. Results can aid in designing more effective view management techniques, especially for wider field of view displays.

Large, high-resolution displays demonstrated their effectiveness in lab settings for cognitively demanding tasks in single user and collaborative scenarios. The effectiveness is mostly reached through inherent displays» properties - large display real estate and high resolution - that allow for visualization of complex datasets, and support of group work and embodied interaction. To raise users» efficiency, however, more sophisticated user support in the form of advanced user interfaces might be needed. For that we need profound understanding of how large, tiled displays impact users work and behavior. We need to extract behavioral patterns for different tasks and data types. This paper reports on study results of how users, while working collaboratively, process spatially fixed items on large, tiled displays. The results revealed a recurrent pattern showing that users prefer to process documents column wise rather than row wise or erratic.

In this research demo, we show different examples for the integration of multisensory cues to enhance user engagement and trigger emotional responses in immersive environments. Our primary focus is to use this modular-designed system for the supportive treatment of different anxiety disorders.

In presence of conflicting or ambiguous visual cues in complex scenes, performing 3D selection and manipulation tasks can be challenging. To improve motor planning and coordination, we explore audio-tactile cues to inform the user about the presence of objects in hand proximity, e.g., to avoid unwanted object penetrations. We do so through a novel glove-based tactile interface, enhanced by audio cues. Through two user studies, we illustrate that proximity guidance cues improve spatial awareness, hand motions, and collision avoidance behaviors, and show how proximity cues in combination with collision and friction cues can significantly improve performance.

We present a novel forearm-and-glove tactile interface that can enhance 3D interaction by guiding hand motor planning and coordination. In particular, we aim to improve hand motion and pose actions related to selection and manipulation tasks. Through our user studies, we illustrate how tactile patterns can guide the user, by triggering hand pose and motion changes, for example to grasp (select) and manipulate (move) an object. We discuss the potential and limitations of the interface, and outline future work.

Enhancing touch screen interfaces through non-visual cues has been shown to improve performance. In this paper we report on a novel system that explores the usage of a forcesensitive motion-platform enhanced tablet interface to improve multi-modal interaction based on visuo-haptic instead of tactile feedback. Extending mobile touch screen with force-sensitive haptic feedback has potential to enhance performance interacting with GUIs and to improve perception of understanding relations. A user study was performed to determine the perceived recognition of different 3D shapes and the perception of different heights. Furthermore, two application scenarios are proposed to explore our proposed visuo-haptic system. The studies show the positive stance towards the feedback, as well as the found limitations related to perception of feedback.

We describe here a pilot user study comparing five different locomotion interfaces for virtual reality (VR) locomotion. We compared a standard non-motion cueing interface, Joystick, with four leaning-based seated motion-cueing interfaces: NaviChair, MuvMan, Head-Directed and Swivel Chair. The aim of this mixed methods study was to investigate the usability and user experience of each interface, in order to better understand relevant factors and guide the design of future ground-based VR locomotion interfaces. We asked participants to give talk-aloud feedback and simultaneously recorded their responses while they were performing a search task in VR. Afterwards, participants completed an online questionnaire. Although the Joystick was rated as more comfortable and precise than the other interfaces, the leaning-based interfaces showed a trend to provide more enjoyment and a greater sense of self-motion. There were also potential issues of using velocity-control for rotations in leaning-based interfaces when using HMDs instead of stationary displays. Developers need to focus on improving the controllability and perceived safety of these seated motion cueing interfaces.

In this course, we will take a detailed look at various breeds of spatial navigation interfaces that allow for locomotion in digital 3D environments such as games, virtual environments or even the exploration of abstract data sets. We will closely look into the basics of navigation, unraveling the psychophysics (including wayfinding) and actual navigation (travel) aspects. The theoretical foundations form the basis for the practical skillset we will develop, by providing an in-depth discussion of navigation devices and techniques, and a step-by-step discussion of multiple real-world case studies. Doing so, we will cover the full range of navigation techniques from handheld to full-body, highly engaging and partly unconventional methods and tackle spatial navigation with hands-on-experience and tips for design and validation of novel interfaces. In particular, we will be looking at affordable setups and ways to “trick” out users to enable a realistic feeling of self-motion in the explored environments. As such, the course unites the theory and practice of spatial navigation, serving as entry point to understand and improve upon currently existing methods for the application domain at hand.

Advances in computer graphics enable us to create digital images of astonishing complexity and realism. However, processing resources are still a limiting factor. Hence, many costly but desirable aspects of realism are often not accounted for, including global illumination, accurate depth of field and motion blur, spectral effects, etc. especially in real-time rendering. At the same time, there is a strong trend towards more pixels per display due to larger displays, higher pixel densities or larger fields of view. Further observable trends in current display technology include more bits per pixel (high dynamic range, wider color gamut/fidelity), increasing refresh rates (better motion depiction), and an increasing number of displayed views per pixel (stereo, multi-view, all the way to holographic or lightfield displays). These developments cause significant unsolved technical challenges due to aspects such as limited compute power and bandwidth. Fortunately, the human visual system has certain limitations, which mean that providing the highest possible visual quality is not always necessary. In this report, we present the key research and models that exploit the limitations of perception to tackle visual quality and workload alike. Moreover, we present the open problems and promising future research targeting the question of how we can minimize the effort to compute and display only the necessary pixels while still offering a user full visual experience.

In this paper, we report on novel zooming interface methods that deploy a small handheld projector. Using mobile projections to visualize object/environment-related information on real objects introduces new aspects for zooming interfaces. Different approaches are investigated that focus on maintaining a level of context while exploring detailed information. Doing so, we propose methods that provide alternative contextual cues within a single projector and deploy the potential of zoom lenses to support a multilevel zooming approach. Furthermore, we look into the correlation between pixel density, distance to the target, and projection size. Alongside these techniques, we report on multiple user studies, in which we quantified the projection limitations and validated various interactive visualization approaches. Thereby, we focused on solving issues related to pixel density, brightness, and contrast that affect the design of more effective legible zooming interfaces for handheld projectors.

Supported by their large size and high resolution, display walls suit well for different collaboration types. However, in order to foster instead of impede collaboration processes, interaction techniques need to be carefully designed, taking into regard the possibilities and limitations of the display size, and their effects on human perception and performance. In this paper we investigate the impact of visual distractors (which, for instance, might be caused by other collaborators' input) in peripheral vision on short-term memory and attention. The distractors occur frequently when multiple users collaborate in large wall display systems and may draw attention away from the main task, as such potentially affecting performance and cognitive load. Yet, the effect of these distractors is hardly understood. Gaining a better understanding thus may provide valuable input for designing more effective user interfaces. In this article, we report on two interrelated studies that investigated the effect of distractors. Depending on when the distractor is inserted in the task performance sequence, as well as the location of the distractor, user performance can be disturbed: we will show that distractors may not affect short term memory, but do have an effect on attention. We will closely look into the effects, and identify future directions to design more effective interfaces.

Abstract Head‐mounted displays with dense pixel arrays used for virtual reality applications require high frame rates and low latency rendering. This forms a challenging use case for any rendering approach. In addition to its ability of generating realistic images, ray tracing offers a number of distinct advantages, but has been held back mainly by its performance. In this paper, we present an approach that significantly improves image generation performance of ray tracing. This is done by combining foveated rendering based on eye tracking with reprojection rendering using previous frames in order to drastically reduce the number of new image samples per frame. To reproject samples a coarse geometry is reconstructed from a G‐Buffer. Possible errors introduced by this reprojection as well as parts that are critical to the perception are scheduled for resampling. Additionally, a coarse color buffer is used to provide an initial image, refined smoothly by more samples were needed. Evaluations and user tests show that our method achieves real‐time frame rates, while visual differences compared to fully rendered images are hardly perceivable. As a result, we can ray trace non‐trivial static scenes for the Oculus DK2 HMD at 1182 × 1464 per eye within the the VSync limits without perceived visual differences.

This paper introduces a novel zooming interface deploying a pico projector that, instead of a second visual display, leverages audioscapes for contextual information. The technique enhances current flashlight metaphor approaches, supporting flexible usage within the domain of spatial augmented reality to focus on object or environment-related details. Within a user study we focused on quantifying the projection limitations related to depiction of details through the pico projector and validated the interaction approach. The quantified results of the study correlate pixel density, detail and proximity, which can greatly aid to design more effective, legible zooming interfaces for pico projectors - the study can form an example testbed that can be applied well for testing aberrations with other projectors. Furthermore, users rated the zooming technique using audioscapes well, showing the validity of the approach. The studies form the foundation for extending our work by detailing out the audio-visual approach and looking more closely in the role of real-world features on interpreting projected content.

We report on two experiments that deploy low-frequency audio and strong vibrations to induce haptic-like sensations throughout the human body. Vibration is quite frequently deployed in immersive systems, for example to provide collision feedback, but its actual effects are not well understood [Kruijff & Pander 2005; Kruijff et al. 2015]. The starting point of our experiments was a study by Rasmussen [Rasmussen 1982], which found that different vibration frequencies were experienced differently throughout the body. We will show how vibrations affect sensations throughout the body and may provide some directional cues to some parts of the body, yet also illustrate the difficulties.

A wide field of view augmented reality display is a special type of head-worn device that enables users to view augmentations in the peripheral visual field. However, the actual effects of a wide field of view display on the perception of augmentations have not been widely studied. To improve our understanding of this type of display when conducting divided attention search tasks, we conducted an in depth experiment testing two view management methods, in-view and in-situ labelling. With in-view labelling, search target annotations appear on the display border with a corresponding leader line, whereas in-situ annotations appear without a leader line, as if they are affixed to the referenced objects in the environment. Results show that target discovery rates consistently drop with in-view labelling and increase with in-situ labelling as display angle approaches 100 degrees of field of view. Past this point, the performances of the two view management methods begin to converge, suggesting equivalent discovery rates at approximately 130 degrees of field of view. Results also indicate that users exhibited lower discovery rates for targets appearing in peripheral vision, and that there is little impact of field of view on response time and mental workload.

The persona method is a powerful approach to focus on needs and characteristics of target users, keeping complex user data,numbers and diagrams alive during the whole design cycle.However, the development of prosperous personas requires a considerable amount of time, effort and specific skills. This paper introduces the development of a set of 30 basic senior personas, covering a broad range of characteristics of European older adults, following a quantitative development approach. The aim of this tool is to support researchers and developers in extending empathy for their target users when developing ICT solutions for the benefit of older adults. The main innovation lies in the representativeness of the basic senior personas. The personas build on multifaceted quantitative data from a single source including micro-level information from roughly 12,500 older individuals living in different European countries. The resulting personas may be applied in their basic form but are extendable to specific contexts. Also, the suggested tool addresses the drawbacks of current existing personas describing older adults: being representative and cost-efficient. The basic senior personas, a filter tool, a manual and templates for "persona marketing" articles are available for free online under http://elderlypersonas.cure.at.

In this paper, we explore techniques that aim to improve site understanding for outdoor Augmented Reality (AR) applications. While the first person perspective in AR is a direct way of filtering and zooming on a portion of the data set, it severely narrows overview of the situation, particularly over large areas. We present two interactive techniques to overcome this problem: multi-view AR and variable perspective view. We describe in details the conceptual, visualization and interaction aspects of these techniques and their evaluation through a comparative user study. The results we have obtained strengthen the validity of our approach and the applicability of our methods to a large range of application domains.

The goal of our work is to support experts in the process of hypotheses generation concerning the roles of genes in diseases. For a deeper understanding of the complex interdependencies between genes, it is important to bring gene expressions (measurements) into context with pathways. Pathways, which are models of biological processes, are available in online databases. In these databases, large networks are decomposed into small sub-graphs for better manageability. This simplification results in a loss of context, as pathways are interconnected and genes can occur in multiple instances scattered over the network. Our main goal is therefore to present all relevant information, i.e., gene expressions, the relations between expression and pathways and between multiple pathways in a simple, yet effective way. To achieve this we employ two different multiple-view approaches. Traditional multiple views are used for large datasets or highly interactive visualizations, while a 2.5D technique is employed to support a seamless navigation of multiple pathways which simultaneously links to the expression of the contained genes. This approach facilitates the understanding of the interconnection of pathways, and enables a non-distracting relation to gene expression data. We evaluated Caleydo with a group of users from the life science community. Users were asked to perform three tasks: pathway exploration, gene expression analysis and information comparison with and without visual links, which had to be conducted in four different conditions. Evaluation results show that the system can improve the process of understanding the complex network of pathways and the individual effects of gene expression regulation considerably. Especially the quality of the available contextual information and the spatial organization was rated good for the presented 2.5D setup.

This paper provides a classification of perceptual issues in augmented reality, created with a visual processing and interpretation pipeline in mind. We organize issues into ones related to the environment, capturing, augmentation, display, and individual user differences. We also illuminate issues associated with more recent platforms such as handhelds or projector-camera systems. Throughout, we describe current approaches to addressing these problems, and suggest directions for future research.

Multi-display environments combine displays of various form factors into a common interaction space. Cross-display navigation techniques have to provide transitions to move the mouse pointer across display boundaries to reach distant display locations. A spatially consistent description of display relationships thereby supports fluid cross-display navigation. In this paper, we present two spatially consistent navigation techniques for seamless cross-display navigation in multi-user multi-display environments. These navigation techniques are automatically configured from a spatial model of the environment, which is generated in a camera-assisted calibration step. We describe the implementation in a distributed system and present results of a comparative experiment.

In this paper, we report on four generations of display-sensor platforms for handheld augmented reality. The paper is organized as a compendium of requirements that guided the design and construction of each generation of the handheld platforms. The first generation, reported in [17]), was a result of various studies on ergonomics and human factors. Thereafter, each following iteration in the design-production process was guided by experiences and evaluations that resulted in new guidelines for future versions. We describe the evolution of hardware for handheld augmented reality, the requirements and guidelines that motivated its construction.

This special section contains the best three full papers from the ACM Symposium on Virtual Reality Software and Technology (VRST) 2008, held in Bordeaux, France.

This paper focuses on the design of devices for handheld spatial interaction. In particular, it addresses the requirements and construction of a new platform for interactive AR, described from an ergonomics stance, prioritizing human factors of spatial interaction. The result is a multi-configurable platform for spatial interaction, evaluated in two AR application scenarios. The user tests validate the design with regards to grip, weight balance and control allocation, and provide new insights on the human factors involved in handheld spatial interaction.

This paper presents first results of an interaction design study performed for a novel handheld interaction system. Human factors of mid-size, self-containing wearable computer systems are explored.

This paper focuses at the usage of neuromuscular electrical stimulation (NMES) for achieving pseudo-haptic feedback. By stimulating the motor nerves, muscular contractions can be triggered that can be matched to a haptic event. Reflecting an initial user test, we will explain how this process can be realized, by investigating the physiological processes involved. Relating the triggered feedback to general haptics, its potential in future interfaces will be identified and laid out in a development roadmap.

Recent studies have shown that through a careful combination of multiple sensory channels, so called multisensory binding effects can be achieved that can be beneficial for collision detection and texture recognition feedback. During the design of a new pen-input device called Tactylus, specific focus was put on exploring multisensory effects of audiotactile cues to create a new, but effective way to interact in virtual environments with the purpose to overcome several of the problems noticed in current devices.