Designing effective navigation systems in mixed reality (MR) is a critical challenge that requires a nuanced understanding of how virtual cues interact with user characteristics and the physical environment. To address this, we investigated the impact of two fundamental design choices: the frame of reference for displaying cues (world-fixed vs. screen-fixed) and the use of explicit landmark annotations. An urban environment with local and global landmarks was designed to understand how the visibility of environmental features impacts users’ visual attention and interferes with the formation of spatial knowledge. We assessed the navigation behavior, gaze patterns, and spatial knowledge acquisition of 81 participants (40 female, 41 male). We also analyzed the role of individual differences, including spatial ability and gender. Results show that world-fixed cues aligned better with natural gaze patterns and reduced unnecessary saccades, landmark cues improved learning, particularly for participants with lower spatial ability, and the frequency of utilizing global landmarks during navigation enhances spatial memory. We further observed gender-related differences in gaze–head coordination strategies. These findings advance understanding of how MR navigation cues influence real-time behavior and outcomes in spatial tasks. They provide design recommendations for adaptive MR systems that match cue placement to environmental conditions and user characteristics.

Gaze-adaptive interfaces are increasingly employed in mixed reality (MR) to support attention-aware interaction. However, it remains unclear how the usability and effectiveness of such interfaces are influenced by individual differences in spatial ability, cognition, or prior experience. This paper addresses this gap by focusing on user spatial navigation ability and its role in modulating user interaction with gaze-adaptive cues during spatial navigation. We compare gaze-adaptive cues to proximity-triggered cues in an MR navigation task, and leverage eye tracking to examine how users' gaze patterns relate to their spatial ability and spatial knowledge acquisition under each cue type. This enables us to explore contentindependent gaze measures as a potential subjective approach to assess spatial ability and learning outcomes in MR. Our findings show that gaze-adaptive cueing impacts users differently based on their spatial abilities. In particular, when adding gaze-adaptive cues, individuals with higher spatial ability seem to engaged with cues more and exhibited longer fixations. Furthermore, higher spatial ability and better learning outcomes were linked to more effective gaze patterns defined by longer fixation durations, larger saccadic amplitudes, and reduced gaze transition entropy, with effects varying by cue type. Our findings highlight the potential of gaze transition entropy for assessing spatial ability disparities in MR spatial tasks.

Daily travel usually demands navigation on foot across a variety of different application domains, including tasks like search and rescue or commuting. Head-mounted augmented reality (AR) displays provide a preview of future navigation systems on foot, but designing them is still an open problem. In this paper, we look at two choices that such AR systems can make for navigation: 1) whether to denote landmarks with AR cues and 2) how to convey navigation instructions. Specifically, instructions can be given via a head-referenced display (screen-fixed frame of reference) or by giving directions fixed to global positions in the world (world-fixed frame of reference). Given limitations with the tracking stability, field of view, and brightness of most currently available head-mounted AR displays for lengthy routes outdoors, we decided to simulate these conditions in virtual reality. In the current study, participants navigated an urban virtual environment and their spatial knowledge acquisition was assessed. We experimented with whether or not landmarks in the environment were cued, as well as how navigation instructions were displayed (i.e., via screen-fixed or world-fixed directions). We found that the world-fixed frame of reference resulted in better spatial learning when there were no landmarks cued; adding AR landmark cues marginally improved spatial learning in the screen-fixed condition. These benefits in learning were also correlated with participants' reported sense of direction. Our findings have implications for the design of future cognition-driven navigation systems.

Today, augmented reality (AR) is most easily experienced through a mobile device such as a modern smartphone. For AR to be useful for applications such as training, it is important to understand how people perceive interactions with virtual objects presented to them via mobile AR. In this paper, we investigated two judgments of action capabilities (affordances) with virtual objects presented through smartphones: passing through an aperture and stepping over a gap. Our goals were to 1) determine if people can reliably scale these judgments to their body dimensions or capabilities and 2) explore whether cues presented in the context of the action could change their judgments. Assessments of perceived action capabilities were made in a pre/post-test design in which observers judged their affordances towards virtual objects prior to seeing an AR cue denoting their body dimension/capability, while viewing the cue, and after seeing the cue. Different patterns of results were found for the two affordances. For passing through, estimates became closer to shoulder width in the post-cue compared to the pre-cue block. For gap stepping, estimates were closer to actual stepping capability while viewing the cue, but did not persist when the cue was no longer present. Overall, our findings show that mobile smartphones can be used to assess perceived action capabilities with virtual targets and that AR cues can influence the perception of action capabilities in these devices. Our work provides a foundation for future studies investigating perception with the use of mobile AR with smartphones.

Immersive virtual environments (VEs) are most useful for training and education when viewers perceive and act accurately within them. Judgments of action capabilities within a VE provide a good measure of perceptual fidelity - the notion of how closely perception and action in the VE match that in the real world - and can also assess how perception for action may be calibrated with visual feedback based on one's own actions. In the current study we tested judgments of action capabilities within a VE for two different reaching behaviors: reaching out and reaching up. Our goal was to assess whether feedback from actual reaching improves judgments and if any recalibration due to feedback differed across reaching behaviors. We first measured participants' actual reaching out and reaching up capabilities so that feedback trials could be scaled to their actual abilities. Participants then completed blocks of alternating perceptual adjustment and feedback trials. In adjustment trials, they adjusted a virtual target to a distance perceived to be just reachable. In feedback trials, they viewed targets that were farther or closer than their actual reach, decided whether the target was reachable, and then reached out to the target to receive visual feedback from a hand-held controller. The first feedback block manipulated the target distance to be 30% over or under actual reach and subsequent blocks decreased the deviation to 20%,10% and 5% of actual reach. We found that for both reaching behaviors, reach was initially overestimated, and then perceptual estimations decreased to become more accurate over feedback blocks. Accuracy in the feedback trials themselves showed that targets just beyond reach were more difficult to judge correctly. This study establishes a straightforward methodology that can be used for calibration of actions in VEs and has implications for applications that depend on accurate reaching within VEs.