This paper investigates the feasibility of fusing two eye-centric authentication modalities—eye movements and periocular images—within a calibration-free authentication system. While each modality has independently shown promise for user authentication, their combination within a unified gaze-estimation pipeline has not been thoroughly explored at scale. In this report, we propose a multimodal authentication system and evaluate it using a large-scale in-house dataset comprising 9202 subjects with an eye tracking (ET) signal quality equivalent to a consumer-facing virtual reality (VR) device. Our results show that the multimodal approach consistently outperforms both unimodal systems across all scenarios, surpassing the FIDO benchmark. The integration of a state-of-the-art machine learning architecture contributed significantly to the overall authentication performance at scale, driven by the model’s ability to capture authentication representations and the complementary discriminative characteristics of the fused modalities.

To enable egocentric contextual AI in always-on smart glasses, it is crucial to be able to keep a record of the user's interactions with the world, including during reading. In this paper, we introduce a new task of reading recognition to determine when the user is reading. We first introduce the first-of-its-kind large-scale multimodal Reading in the Wild dataset, containing 100 hours of reading and non-reading videos in diverse and realistic scenarios. We then identify three modalities (egocentric RGB, eye gaze, head pose) that can be used to solve the task, and present a flexible transformer model that performs the task using these modalities, either individually or combined. We show that these modalities are relevant and complementary to the task, and investigate how to efficiently and effectively encode each modality. Additionally, we show the usefulness of this dataset towards classifying types of reading, extending current reading understanding studies conducted in constrained settings to larger scale, diversity and realism. Code, model, and data will be public.

We explored how interaction performance is affected by multi-depth VR targeting and button selection using two targeting methods: eye tracking with no UX modifications and feedback, or the controller with a visible cursor for targeting. Selections happened on a controller button press for both targeting modalities. Targets had a diameter of either 3, 4, or 5 degrees, placed in depths between 0.3 m-5m. When comparing conditions of a 1 m single depth vs. multi-depth environment, the eyes were less affected by depth than the controller. We found that performance decreased in multi-depth scenarios on targeting and selection for the controller as measured by Throughput (22% decrease), Movement Time (31% increase), and Misses (66% increase). Depth also affected eye tracking significantly, but to a lesser degree, for Throughput (4% decrease) and Movement Time (6% increase) but not Misses (5% increase). The eyes outperformed the controller in multi-depth scenarios, as measured by Throughput (2.86 bits/s vs. 2.56 bits/s), and were similar in Movement Time (1.10s vs. 1.10s) but had the most Misses (21% vs. 9%). Our study also shows that selecting consecutive targets that come closer to the user is more difficult than those that diverge away from the user, and that targets with larger depth distances take longer to select. Overall, this study provides further supporting evidence that eye tracking can play an important role in 3D interactions.

Can eye tracking enable VR users to target and select elements at par or better than controller or head-based targeting? We explored visual feedback methods (none, cursor, outline and resize) for gaze targeting with a button press for selection, and an ecologically valid representation of commercially available user interfaces with a body-locked, grid-based design. Forty participants interacted with a 5x5 square element grid, and elements subtended either 3-, 6- or 9-degrees of visual angle. If the participant looked out of the grid boundary, on button press, we chose to select the last targeted element, but no other algorithms to enhance performance were employed. We also assessed signal quality requirements with a fixed offset 1.5-degree accuracy degradation. Participants completed 36 blocks and in each, targeted and selected 15 successive elements. We found that gaze targeting, with appropriate feedback methods and a button press, can perform at par or better than the controller in cases intended to replicate targeting and selecting in world- or body-locked paradigms in AR/VR. We anticipate that with design improvements or algorithmic mitigations that this can improve significantly.

Virtual Reality (VR) holds great potential for psychomotor training, with existing applications using almost exclusively a ‘learning-by-doing’ active learning approach, despite the possible benefits of incorporating observational learning. We compared active learning (n=26) with different variations of observational learning in VR for a manual assembly task. For observational learning, we considered three levels of visual similarity between the demonstrator avatar and the user, dissimilar (n=25), minimally similar (n=26), or a self-avatar (n=25), as similarity has been shown to improve learning. Our results suggest observational learning can be effective in VR when combined with ‘hands-on’ practice and can lead to better far skill transfer to real-world contexts that differ from the training context. Furthermore, we found self-similarity in observational learning can be counterproductive when focusing on a manual task, and skills decay quickly without further training. We discuss these findings and derive design recommendations for future VR training.

Gaze-assisted interaction techniques enable intuitive selections without requiring manual pointing but can result in unintended selections, known as Midas touch. A confirmation trigger eliminates this issue but requires additional physical and conscious user effort. Brain-computer interfaces (BCIs), particularly passive BCIs harnessing anticipatory potentials such as the Stimulus-Preceding Negativity (SPN) - evoked when users anticipate a forthcoming stimulus - present an effortless implicit solution for selection confirmation. Within a VR context, our research uniquely demonstrates that SPN has the potential to decode intent towards the visually focused target. We reinforce the scientific understanding of its mechanism by addressing a confounding factor - we demonstrate that the SPN is driven by the user’s intent to select the target, not by the stimulus feedback itself. Furthermore, we examine the effect of familiarly placed targets, finding that SPN may be evoked quicker as users acclimatize to target locations; a key insight for everyday BCIs.

Surfaces with deformable and shape-changing properties seek to enhance and diversify tangible interactions with computing systems. However, we currently lack fundamental knowledge and user interface design principles that connect the inherent properties of deformable shapes with our human senses and cognitive associations. To address this knowledge gap, we systematically explored deformable shapes’ cross-modal correspondences (CC) with colours and emotions. In our CC study, 52 participants were presented with deformable shape stimuli that varied in stiffness and angularity. They were asked to associate these stimuli with colours and emotions under (i) visuo-tactile and; (ii) tactile-only conditions. For the first time, our findings reveal (1) how stiffness level primarily influences the CC associations and; (2) that stiffness and angularity play a significant role in CC associations over the visibility of the shapes. The results were distilled into design guidelines for future deformable, shape-changing interfaces that engage specific human senses and responses.

All-day augmented reality (AR) requires contextually-aware artificial intelligence (AI) models that excel across diverse daily contexts. Eye tracking could be a key source of information about user context and intention. However, such models using gaze sometimes struggle to outperform egocentric video-based baseline models. We propose that learning representations of scanpath history in a perceptually-relevant state space may solve this problem. However, scanpaths are often assumed to obey a Markovian assumption, i.e., only the current and previous fixation matter. In a user study (30 participants; 26.2 hours total), we analyzed scanpaths during nine everyday tasks and identified long-term temporal dependencies, with an average timescale of four fixations (2 seconds) into the past (i.e., violating the Markovian assumption). We discovered substantial task-specific variations in these dependencies. This confirms that scanpaths contain stereotyped “motifs” with context-dependent lengths/timescales. We discuss the implications for designing contextual AI models for AR applications.

Gaze-based user interfaces and interactions are becoming more prevalent in augmented and virtual reality (AR/VR). The effectiveness of eye tracking for interaction depends on its quality. Many studies discuss eye tracking as an input and interaction modality but do not provide details about eye tracking quality, making it difficult to compare findings. Here we implement a framework to degrade accuracy error with the user in the loop. We then approximate calibration error, with those degradations applied in each block to provide an “Effective Gaze Error.” Participants selected single targets (3° or 5° diameter) using an eye tracking sampling frequency and display rate of 120 Hz. Higher “Effective Gaze Error” on smaller targets resulted in decreased human performance and subjective evaluations. Our experiment framework and results provide a starting point for future studies assessing how gaze accuracy degradation impacts performance, beyond interactions tasks.

This paper performs the crucial work of establishing a baseline for gaze-driven authentication performance to begin answering fundamental research questions using a very large dataset of gaze recordings from 9202 people with a level of eye tracking (ET) signal quality equivalent to modern consumer-facing virtual reality (VR) platforms. The size of the employed dataset is at least an order-of-magnitude larger than any other dataset from previous related work. Binocular estimates of the optical and visual axes of the eyes and a minimum duration for enrollment and verification are required for our model to achieve a false rejection rate (FRR) of below 3% at a false acceptance rate (FAR) of 1 in 50,000. In terms of identification accuracy which decreases with gallery size, we estimate that our model would fall below chance-level accuracy for gallery sizes of 148,000 or more. Our major findings indicate that gaze authentication can be as accurate as required by the FIDO standard when driven by a state-of-the-art machine learning architecture and a sufficiently large training dataset.

We introduce Pic2Tac, a novel system that automatically converts photographs into tactile images. It offers an alternative way to communicate visual information that is difficult to express using braille or alternative text. Current methods for creating tactile images are either limited in representation, or require handmade artefacts. Pic2Tac employs a unique approach that avoids a literal representation of image content (e.g. contours). Instead, it detects salient semantic content within photographs and translates them into tactile images using dedicated ‘tactile words’. Foreground objects are represented using icons, and patterns are used for background regions. The resulting binary image is printed on swell paper, where black regions rise to form a tactile image. Studies involving 60 participants, both sighted and with visual impairments, demonstrate the effectiveness of these tactile images in communicating semantic meaning. Our findings show that tactile and visual descriptions of scenes matched significantly. Overall, Pic2Tac is an affordable way to create accessible tactile images, costing only 1.50 USD per sheet.

VR exergaming is a promising motivational tool to incentivise exercise. We present a novel VR exergaming method called social interactive feedforward. The player competes with an 'enhanced model' of one of their friends in a real-time VR environment, showing improved performance levels in a way the player can relate to. Social interactive feedforward was tested in a cycling-based VR exergame and players competed with enhanced models of themselves, their friend, and a stranger moving at the same enhanced pace. Results show that friend-modelled social interactive feedforward improves performance and intrinsic motivation the most. This indicates that the mere association of the enhanced model with their friend results in a rapid improvement in performance and motivation which implies that social feedforward was successfully elicited by using an enhanced friend's model. This widens the application of self-modelled feedforward to a wide range of social options which enables players to also reap the benefits of socialising in addition to feedforward benefits.

Outside of self-report surveys, there are no proven, reliable methods to quantify visual discomfort or visually induced motion sickness symptoms when using head-mounted displays. While valuable tools, self-report surveys suffer from potential biases and low sensitivity due to variability in how respondents may assess and report their experience. Consequently, extreme visual-vestibular conflicts are generally used to induce discomfort symptoms large enough to measure reliably with surveys (e.g., stationary participants riding virtual roller coasters). An emerging area of research is the prediction of discomfort survey results from physiological and behavioral markers. However, the signals derived from experimental paradigms that are explicitly designed to be uncomfortable may not generalize to more naturalistic experiences where comfort is prioritized. In this work we introduce a custom VR headset designed to introduce significant near-eye optical distortion (i.e., pupil swim) to induce visual discomfort during more typical VR experiences. We evaluate visual comfort in our headset while users play the popular VR title Job Simulator and show that eye-tracked dynamic distortion correction improves visual comfort in a multi-session, within-subjects user study. We additionally use representational similarity analysis to highlight changes in head and gaze behavior that are potentially more sensitive to visual discomfort than surveys.

Eye tracking is routinely being incorporated into virtual reality (VR) systems. Prior research has shown that eye tracking data, if exposed, can be used for re-identification attacks [14]. The state of our knowledge about currently existing privacy mechanisms is limited to privacy-utility trade-off curves based on data-centric metrics of utility, such as prediction error, and black-box threat models. We propose that for interactive VR applications, it is essential to consider user-centric notions of utility and a variety of threat models. We develop a methodology to evaluate real-time privacy mechanisms for interactive VR applications that incorporate subjective user experience and task performance metrics. We evaluate selected privacy mechanisms using this methodology and find that re-identification accuracy can be decreased to as low as 14% while maintaining a high usability score and reasonable task performance. Finally, we elucidate three threat scenarios (black-box, black-box with exemplars, and white-box) and assess how well the different privacy mechanisms hold up to these adversarial scenarios. This work advances the state of the art in VR privacy by providing a methodology for end-to-end assessment of the risk of re-identification attacks and potential mitigating solutions. f.

Virtual environments can support psychomotor learning by allowing learners to observe instructor avatars. Instructor avatars that look like the learner hold promise in enhancing learning; however, it is unclear whether this works for psychomotor tasks and how similar avatars need to be. We investigated ‘minimal’ customisation of instructor avatars, approximating a learner’s appearance by matching only key visual features: gender, skin-tone, and hair colour. These avatars can be created easily and avoid problems of highly similar avatars. Using modern dancing as a skill to learn, we compared the effects of visually similar and dissimilar avatars, considering both learning on a screen (n=59) and in VR (n=38). Our results indicate that minimal avatar customisation leads to significantly more vivid visual imagery of the dance moves than dissimilar avatars. We analyse variables affecting interindividual differences, discuss the results in relation to theory, and derive design implications for psychomotor training in virtual environments.

Presence is one of the most studied and most important variables in immersive virtual reality (VR) and it influences the effectiveness of many VR applications. Separate bodies of research indicate that presence is determined by (1) technical factors such as the visual realism of a virtual environment (VE) and the field of view (FoV), and (2) human factors such as emotions and agency. However, it remains unknown how technical and human factors may interact in the presence formation process. We conducted a user study (n=360) to investigate the effects of visual realism (high/low), FoV (high/low), emotions (focusing on fear) and agency (yes/no) on presence. Counter to previous assumptions, technical factors did not affect presence directly but were moderated through human factors. We propose TAP-Fear, a structural equation model that describes how design decisions, technical factors and human factors combine and interact in the formation of presence.

Personality characteristics can affect how much presence an individual experiences in virtual reality, and researchers have explored how it may be possible to prime users to increase their sense of presence. A personality characteristic that has yet to be explored in the VR literature is imaginative suggestibility, the ability of an individual to successfully experience an imaginary scenario as if it were real. In this paper, we explore how suggestibility and priming affect presence when consulting an ancient oracle in VR as part of an educational experience – a common VR application. We show for the first time how imaginative suggestibility is a major factor which affects presence and emotions experienced in VR, while priming cues have no effect on participants’ (n=128) user experience, contrasting results from prior work. We consider the impacts of these findings for VR design and provide guidelines based on our results.

Deformable interfaces provide unique interaction potential for force input, for example, when users physically push into a soft display surface. However, there remains limited understanding of which visual-tactile design elements signify the presence and stiffness of such deformable force-input components. In this paper, we explore how people correspond surface stiffness to colours, graphical shapes, and physical shapes. We conducted a cross-modal correspondence (CC) study, where 30 participants associated different surface stiffnesses with colours and shapes. Our findings evidence the CCs between stiffness levels for a subset of the 2D/3D shapes and colours used in the study. We distil our findings in three design recommendations: (1) lighter colours should be used to indicate soft surfaces, and darker colours should indicate stiff surfaces; (2) rounded shapes should be used to indicate soft surfaces, while less-curved shapes should be used to indicate stiffer surfaces, and; (3) longer 2D drop-shadows should be used to indicate softer surfaces, while shorter drop-shadows should be used to indicate stiffer surfaces.

In this study, we establish a much-needed baseline for evaluating eye tracking interactions using an eye tracking enabled Meta Quest 2 VR headset with 30 participants. Each participant went through 1098 targets using multiple conditions representative of AR/VR targeting and selecting tasks, including both traditional standards and those more aligned with AR/VR interactions today. We use circular white world-locked targets, and an eye tracking system with sub-1-degree mean accuracy errors running at approximately 90Hz. In a targeting and button press selection task, we, by design, compare completely unadjusted, cursor-less, eye tracking with controller and head tracking, which both had cursors. Across all inputs, we presented targets in a configuration similar to the ISO 9241-9 reciprocal selection task and another format with targets more evenly distributed near the center. Targets were laid out either flat on a plane or tangent to a sphere and rotated toward the user. Even though we intended this to be a baseline study, we see unmodified eye tracking, without any form of a cursor, or feedback, outperformed the head by 27.9% and performed comparably to the controller (5.63% decrease) in throughput. Eye tracking had improved subjective ratings relative to head in Ease of Use, Adoption, and Fatigue (66.4%, 89.8%, and 116.1 % improvements, respectively) and had similar ratings relative to the controller (reduction by 4.2%, 8.9%, and 5.2% respectively). Eye tracking had a higher miss percentage than controller and head (17.3% vs 4.7% vs 7.2% respectively). Collectively, the results of this baseline study serve as a strong indicator that eye tracking, with even minor sensible interaction design modifications, has tremendous potential in reshaping interactions in next-generation AR/VR head mounted displays.

Street harassment is a widespread problem that can constrain people’s freedom to enjoy public spaces safely, along with many other negative psychological impacts. However, very little research has looked at how immersive technology can help in addressing it. We conducted three studies to investigate the design decisions, ethical issues and efficacy of an immersive simulation of street harassment: an online design study (n=20), an interview study with experts working in the area (n=9), and a comparative lab study investigating design, ethics and efficacy (n=44). Our results deepen understanding of the design decisions that contribute to a realistic psychological experience, such as the effects of screen-based video vs passive VR vs interactive VR. They also highlight important ethical issues such as traumatisation and potential for victim blaming, and how they can be approached in an ethical manner. Finally, they provide insights into efficacy in terms of perceived usefulness, competence and empathy.

We investigate the hypothesis that gaze-signal can improve egocentric action recognition on the standard benchmark, EGTEA Gaze++ dataset. In contrast to prior work where gaze-signal was only used during training, we formulate a novel neural fusion approach, Cross-modality Attention Blocks (CMA), to leverage gaze-signal for action recognition during inference as well. CMA combines information from different modalities at different levels of abstraction to achieve state-of-the-art performance for egocentric action recognition. Specifically, fusing the video-stream with optical-flow with CMA outperforms the current state-of-the-art by 3%. However, when CMA is employed to fuse gaze-signal with video-stream data, no improvements are observed. Further investigation of this counter-intuitive finding indicates that small spatial overlap between the network’s attention-map and gaze ground-truth renders the gaze-signal uninformative for this benchmark. Based on our empirical findings, we recommend improvements to the current benchmark to develop practical systems for egocentric video understanding with gaze-signal.

Ultrasonic mid-air haptic technologies, which provide haptic feedback through airwaves produced using ultrasound, could be employed to investigate the sense of body ownership and immersion in virtual reality (VR) by inducing the virtual hand illusion (VHI). Ultrasonic mid-air haptic perception has solely been investigated for glabrous (hairless) skin, which has higher tactile sensitivity than hairy skin. In contrast, the VHI paradigm typically targets hairy skin without comparisons to glabrous skin. The aim of this article was to investigate illusory body ownership, the applicability of ultrasonic mid-air haptics, and perceived immersion in VR using the VHI. Fifty participants viewed a virtual hand being stroked by a feather synchronously and asynchronously with the ultrasonic stimulation applied to the glabrous skin on the palmar surface and the hairy skin on the dorsal surface of their hands. Questionnaire responses revealed that synchronous stimulation induced a stronger VHI than asynchronous stimulation. In synchronous conditions, the VHI was stronger for palmar stimulation than dorsal stimulation. The ultrasonic stimulation was also perceived as more intense on the palmar surface compared to the dorsal surface. Perceived immersion was not related to illusory body ownership per se but was enhanced by the provision of synchronous stimulation.

The landscape of digital games is segregated by player ability. For example, sighted players have a multitude of highly visual games at their disposal, while blind players may choose from a variety of audio games. Attempts at improving cross-ability access to any of those are often limited in the experience they provide, or disregard multiplayer experiences. We explore ability-based asymmetric roles as a design approach to create engaging and challenging mixed-ability play. Our team designed and developed two collaborative testbed games exploring asymmetric interdependent roles. In a remote study with 13 mixed-visual-ability pairs we assessed how roles affected perceptions of engagement, competence, and autonomy, using a mixed-methods approach. The games provided an engaging and challenging experience, in which differences in visual ability were not limiting. Our results underline how experiences unequal by design can give rise to an equitable joint experience.

Arguably one of the most important characteristics of virtual reality (VR) is its ability to induce higher feelings of presence. Still, research has remained inconclusive on how presence is affected by human factors such as emotion and agency. Here we adopt a novel design to investigate their effects by testing virtual environments inducing either happiness or fear, with or without user agency. Results from 121 participants showed that the dominant emotion induced by a virtual environment is positively correlated with presence. In addition, agency had a significant positive effect on presence and, furthermore, moderated the effect of emotion on presence. We show for the first time that the effects of emotion and agency on presence are not straightforward but they can be modelled by separating design factors from subjective measures. We discuss how these findings can explain seemingly conflicting results of related work and their implications for VR design.

With increasing interest in multisensory experiences in HCI there is a need to consider the potential impact of crossmodal correspondences (CCs) between sensory modalities on perception and interpretation. We investigated CCs between active haptic experiences of tangible 3D objects, visual colour and emotion using the “Bouba/Kiki” paradigm. We asked 30 participants to assign colours and emotional categories to 3D-printed objects with varying degrees of angularity and complexity. We found tendencies to associate high degrees of complexity and angularity with red colours, low brightness and high arousal levels. Less complex round shapes were associated with blue colours, high brightness and positive valence levels. These findings contrast previously reported crossmodal effects triggered by 2D shapes of similar angularity and complexity, suggesting that designers cannot simply extrapolate potential perceptual and interpretive experiences elicited by 2D shapes to seemingly similar 3D tangible objects. Instead, we propose a design space for creating tangible multisensory artefacts that can trigger specific emotional percepts and discuss implications for exploiting CCs in the design of interactive technology.

User experience estimation of VR exergame players by recognising their affective state could enable us to personalise and optimise their experience. Affect recognition based on psychophysiological measurements has been successful for moderate intensity activities. High intensity VR exergames pose challenges as the effects of exercise and VR headsets interfere with those measurements. We present two experiments that investigate the use of different sensors for affect recognition in a VR exergame. The first experiment compares the impact of physical exertion and gamification on psychophysiological measurements during rest, conventional exercise, VR exergaming, and sedentary VR gaming. The second experiment compares underwhelming, overwhelming and optimal VR exergaming scenarios. We identify gaze fixations, eye blinks, pupil diameter and skin conductivity as psychophysiological measures suitable for affect recognition in VR exergaming and analyse their utility in determining affective valence and arousal. Our findings provide guidelines for researchers of affective VR exergames.

There is an increasing interest in non-visual interfaces for HCI to take advantage of the information processing capability of the other sensory modalities. The BrainPort is a vision-to-tactile sensory substitution device that conveys information through electro-stimulation on the tongue. As the tongue is a horizontal surface, it makes for an interesting platform to study the brain's representation of space. But which way is up on the tongue? We provided participants with perceptually ambiguous stimuli and measured how often different perspectives were adopted; furthermore, whether camera orientation and gender had an effect. Additionally, we examined whether personality (trait extraversion and openness) could predict the perspective taken. We found that self-centered perspectives were predominantly adopted, and that trait openness may predict perspective. This research demonstrates how individual differences can affect the usability of sensory substitution devices, and highlights the need for flexible and customisable interfaces.

Virtual environments for gaming and simulation provide dynamic and adaptive experiences, but, despite advances in multisensory interfaces, these are still primarily visual experiences. To support real-time dynamic adaptation, interactive virtual environments could implement techniques to predict and manipulate human visual attention. One promising way of developing such techniques is to base them on psychophysical observations, an approach that requires a sound understanding of visual attention allocation. Understanding how this allocation of visual attention changes depending on a user’s task offers clear benefits in developing these techniques and improving virtual environment design. With this aim, we investigated the effect of task on visual attention in interactive virtual environments. We recorded fixation data from participants completing freeview, search, and navigation tasks in three different virtual environments. We quantified visual attention differences between conditions by identifying the predictiveness of a low-level saliency model and its corresponding color, intensity, and orientation feature-conspicuity maps, as well as measuring fixation center bias, depth, duration, and saccade amplitude. Our results show that task does affect visual attention in virtual environments. Navigation relies more than search or freeview on intensity conspicuity to allocate visual attention. Navigation also produces fixations that are more central, longer, and deeper into the scenes. Further, our results suggest that it is difficult to distinguish between freeview and search tasks. These results provide important guidance for designing virtual environments for human interaction, as well as identifying future avenues of research for developing “attention-aware” virtual worlds.

A key requirement for a sense of presence in Virtual Environments (VEs) is for a user to perceive space as naturally as possible. One critical aspect is distance perception. When judging distances, compression is a phenomenon where humans tend to underestimate the distance between themselves and target objects (termed egocentric or absolute compression), and between other objects (exocentric or relative compression). Results of studies in virtual worlds rendered through head mounted displays are striking, demonstrating significant distance compression error. Distance compression is a multisensory phenomenon, where both audio and visual stimuli are often compressed with respect to their distances from the observer. In this paper, we propose and test a method for reducing crossmodal distance compression in VEs. We report an empirical evaluation of our method via a study of 3D spatial perception within a virtual reality (VR) head mounted display. Applying our method resulted in more accurate distance perception in a VE at longer range, and suggests a modification that could adaptively compensate for distance compression at both shorter and longer ranges. Our results have a significant and intriguing implication for designers of VEs: an incongruent audiovisual display, i.e. where the audio and visual information is intentionally misaligned, may lead to better spatial perception of a virtual scene.

This paper explores the possible solutions for image annotation and retrieval by implicitly monitoring user attention via eye-tracking. Features are extracted from the gaze trajectory of users examining sets of images to provide implicit information on the target template that guides visual attention. Our Gaze Inference System (GIS) is a fuzzy logic based framework that analyzes the gaze-movement features to assign a user interest level (UIL) from 0 to 1 to every image that appeared on the screen. Because some properties of the gaze features are unique for every user, our user adaptive framework builds a new processing system for every new user to achieve higher accuracy. The generated UILs can be used for image annotation purposes; however, the output of our system is not limited as it can be used also for retrieval or other scenarios. The developed framework produces promising and reliable UILs where approximately 53% of target images in the users' minds can be identified by the machine with an error of less than 20% and the top 10% of them with no error. We show in this paper that the existing information in gaze patterns can be employed to improve the machine's judgement of image content by assessment of human interest and attention to the objects inside virtual environments.