the field toward real-time use of gaze as active or passive modality. When active, gaze can imbue interaction within the user’s egocentric peripersonal space, e.g., for selection. When passive, gaze serves as context for other interaction modalities, e.g., as cues. In both cases, system response needs to be immediate, i.e., real-time. In such scenarios, the system needs information not only from traditional gaze-related metrics such as “what is being fixated” but also from real-time dynamics of gaze just after the given fixation. Although recent advances in real-time metrics have appeared, thus far they have only been evaluated on data that was previously captured, i.e., re-streamed. What are needed are protocols designed to evaluate real-time metrics to validate their efficacy. In this position paper, we review two real-time metrics and suggest potential for future directions of their evaluation.

Traditional eye movement research has in large part been dependent on static, post-experimental analysis of aggregate first-order metrics (e.g., fixations, fixation durations, etc.). Advances in eye-tracking methodology call for dynamical evaluation of second-order metrics (e.g., \(\mathcal {K}\) or gaze entropy) from the time course of collected gaze and eventually in real time. We consider such analysis of these gaze-based indicators for their response during visual search performed by two distinct user groups: Healthy Controls (HC) or those with (Mild) Cognitive Impairment (CI). Analysis of the time course of gaze transition entropy and \(\mathcal {K}\) with Generalized Additive Models (GAMs) shows differing visual scanning strategies on two types of stimuli. On a jumbled image, the HC group adopted a more focal and less predictable strategy compared to the CI group. The effect was reversed on an image of a classical painting.

This eye-tracking study examines saccadic latency during subtitled video processing, i.e., the time to shift attention from the video image to the subtitle, based on subtitle speed (145 vs. 180 words per minute–wpm) and number of lines (one vs. two). We employed a mixed-design experiment with deaf (N=20) and hearing (N=20) participants, making use of a linear mixed-model regression for analysis. While the group difference was not statistically significant, the results revealed a significant interaction between subtitle speed and number of lines: at fast speeds (180 wpm), subtitle line count did not affect latency, yet at slow speeds (145 wpm), two-line subtitles incurred a significantly shorter latency. These findings indicate that two-line subtitles, especially at slower speeds, reduce processing latency, which may preserve more time for actual text comprehension.

The present longitudinal quasi-experimental study examined the extent to which music education is related to the development of attentional control. Control of visual attention was examined with the use of an antisaccade task in an eye tracking study. Fifty primary school children (6–7 years old), 25 from music school matched on fluid intelligence with their peers from non-music primary school, performed the antisaccade task three times, at the beginning of school education, after 12 and 24 months. Their eye movements were recorded each time. Over time, attentional control increased in both groups. Music school children performed significantly better than general school children in antisaccadic trials. In line with the prediction, all students’ correct responses in the antisaccade task were faster over the time of education, supporting growth in their ability of attentional control. Yet this growth was significantly greater in music school children. Only music school children significantly decrease the latency of saccades toward the target in the correct antisaccade trials. No such trend was significant for children without music education. Finally, fluid intelligence increased over time in both groups. The present study demonstrated a relationship between systematic music education for the development of children’s attentional control.

The number of devices embedding eye tracking (ET) capabilities, such as portable webcam-based consumer devices and wearable ones, such as headsets and smart eyeglasses, is rapidly increasing, making this technology truly pervasive. Despite the large number of papers and reviews discussing data quality and benchmarking of trackers, none of them is addressing the trade-off between power consumption, speed and accuracy. Power dissipation is typically dominated by signal processing to extract gaze information from sensors embedded in the glasses. This compromise is crucial for smart glasses, powered by miniature batteries, offering a typical power budget of a few tens of mW for ET. Here we propose a simple benchmarking flow for wearable trackers, focused on power consumption, as well as accuracy, precision and sampling rate, and based on three complementary test setups. We report the preliminary results of the experimental characterization of 6 commercial trackers in the first static setup and we show a comparison of their performance based on a single figure of merit.

ABSTRACT This study examines how subtitles and image visualizations influence gaze behavior, working alliance, and behavior change intentions in virtual health conversations with ECAs. Visualizations refer to images on a 3D model TV and text on a virtual whiteboard, both reinforcing key content conveyed by the ECA. Using a 2 2 factorial design, participants were randomly assigned to one of four conditions: no subtitles or visualizations (Control), subtitles only (SUB), visualizations only (VIS), or both subtitles and visualizations (VISSUB). Structural equation path modeling showed that SUB and VIS individually reduced gaze toward the ECA, whereas VISSUB moderated this reduction, resulting in less gaze loss than the sum of either condition alone. Gaze behavior was positively associated with working alliance, and perceptions of enjoyment and appropriateness influenced engagement, which in turn predicted behavior change intentions. VIS was negatively associated with behavior change intentions, suggesting that excessive visual input may introduce cognitive trade‐offs.

ABSTRACT This study examines how multimodal communication strategies (subtitles, visualizations, and their combination), affect memory retention, attitudes toward behavior change, and cognitive load during long conversations (+20 min) in immersive virtual reality (VR). Using embodied conversational agents to educate participants on diabetes and healthy eating, we found that all conditions effectively improved memory retention and behavior change attitudes. However, the combination of multimodal strategies increased cognitive load, suggesting a trade‐off between engagement and cognitive demands. These findings highlight the potential of long VR conversations for healthcare education, while emphasizing the importance of balancing cognitive demands and exploring personalization for diverse users.

Research on interpersonal relationships has shown that individuals often exhibit different gaze patterns when viewing faces of other races compared to those similar to their own. In the context of Virtual Humans (VHs), participatory design processes frequently include features that reflect the designer's identity. This paper explores gaze behavior towards demographically diverse VHs in the context of emerging educational technology. Using Augmented Reality (AR), 33 participants engaged in a within-subjects study where their eye movements, learning outcomes, and perceptions were compared. Results indicate that gaze (no. of fixations and durations) was influenced by demographic concordance (when race and sex matched that of the VH). Additionally, learning outcomes positively correlated with the number of fixations, suggesting that increased visual focus may enhance learning. However, participants frequently misidentified the race of the Hispanic, Asian, and Black VH, revealing a potential perceptual misalignment likely shaped by implicit social categorization biases.

Body dissatisfaction and eating disorder symptoms have previously been linked to biased attentional processes when looking at one’s own body. The tendency is to focus on body parts self-defined to be most unattractive, compared to those self-defined to be most attractive. This attention bias is assumed to contribute to the maintenance of body dissatisfaction. Therefore, there is a need to measure and analyze attentional bias efficiently in studies on etiology and treatment of body dissatisfaction and eating disorders. We validate an automated analysis pipeline for mobile eye-tracking data while looking at one’s own body in a large mirror. The pipeline uses and extends a machine-learning based body pose estimation algorithm by segmenting identified body parts into subregions defined by provided texture coordinates. Results of a pilot study include a guided-viewing validation task, in which participants were asked to look at specific body regions, and a 3-minute free-viewing task.

Various experiments have led to a better understanding of how the surrounding environment influences long-term memory and spatial cognition, including those indicating how environmental context is encoded. However, specific external environment characteristics that may impact cognitive memory and performance remain unclear. Using an immersive Virtual Reality (VR) environment, we investigated the effects of color congruency on object placement accuracy, recall speed, and recognition.

Locomotion in virtual reality (VR) can be accomplished through various techniques. Some utilize controller-based interactions, while others use more advanced hardware. The current study focuses on systems that provide naturalistic movement. By building upon prior knowledge of previous omnidirectional system evaluations, this study compares users’ performance and preferences between four systems: Infinadeck, Cyberith Virtualizer Elite 2, EKTO VR Voyager Boots, and real-walking with teleportation.

Invisible fiducial markers are introduced for localization of Areas Of Interest (AOIs) in mobile eye tracking applications. Fiducial markers are made invisible through the use of film passing Infra-Red (IR) light while blocking the visible spectrum. An IR light source is used to illuminate the markers which are then detected by an IR-sensitive camera, but which are imperceptible by the human eye. We provide the first empirical study that demonstrates such invisible markers are not distracting to a given task, as demonstrated in a flight simulator where distraction of visible and invisible markers are compared between experienced and novice pilots. Fixation frequency and subjective distraction scores showed that visible markers disrupted natural gaze behaviour, particularly in novice pilots. Our findings show that invisible markers should be used when there is a need for them to remain inconspicuous.

We are excited to provide the third issue of the Proceedings of the ACM on Human-Computer Interaction focusing on contributions from the ACM Eye Tracking Research and Applications (ETRA) community. ETRA is the premier eye-tracking conference bringing together researchers from across disciplines to present advances in eye-tracking systems and methods, oculomotor research, eye movement data analysis, gaze-based interaction, and eye-tracking applications. A total of 25 full papers were accepted for ETRA 2024 (June 4 - 7, 2024, in Glasgow, U.K.), selected from 68 submissions after a rigorous reviewing process (37% acceptance rate). Depending on the topic fit and authors' preferences, accepted papers were split into two issues. 19 accepted papers are presented in this issue, and 6 will be included in an issue of the Proceedings of the ACM on Computer Graphics and Interactive Techniques. We want to thank all members of the Editorial Board and all external reviewers for their effort and dedication, as well as all authors for their high-quality contributions.

The paper presents empirical evidence in how differently art history experts and non-experts scan images of architectural objects. The study is a work-in-progress on a ubiquitous system enhancing accessibility to architectural heritage for a broad audience with a special focus on the Blind and Visually Impaired (BVI). The system provides city navigation for BVI along with the location-based real-time information and Gaze-Led Audio Descriptions (GLAD) about architectural artefacts. Gaze visualizations of non-experts are used to guide professional audio describers in shaping more natural, gaze-led narratives.

Invisible fiducial markers are introduced for localization of Areas Of Interest (AOIs) in mobile eye tracking applications. Fiducial markers are made invisible through the use of film passing Infra-Red (IR) light while blocking the visible spectrum. An IR light source is used to illuminate the markers which are then detected by an IR-sensitive camera, but which are imperceptible by the human eye. We provide the first empirical study that demonstrates such invisible markers are not distracting to a given task, as demonstrated in a flight simulator where distraction of visible and invisible markers are compared between experienced and novice pilots. Fixation frequency and subjective distraction scores showed that visible markers disrupted natural gaze behaviour, particularly in novice pilots. Our findings show that invisible markers should be used when there is a need for them to remain inconspicuous.

Radiologists are trained professionals who use medical images to obtain clinically relevant information. However, little is known about visual search patterns and strategies radiologists employ during medical image analysis. Thus, there is a current need for guidelines to specify optimal visual search routines commonly used by radiologists. Identifying these features could improve radiologist training and assist radiologists in their work. Our study found that during the moments in which radiologists view chest X-ray images in silence before verbalizing the analysis, they exhibit unique search patterns regardless of the type of disease depicted. Our findings suggest that radiologists’ search behaviors can be identified at this stage. However, when radiologists verbally interpret the X-rays, the gaze patterns appear noisy and arbitrary. Current deep-learning approaches train their systems using this noisy and arbitrary gaze data. This may explain why previous research still needs to show the superiority of deep-learning models that use eye tracking for disease classification. Our paper investigates these patterns and attempts to uncover the eye-gaze configurations during the different analysis phases.

In mobile eye-tracking, visual attention is commonly evaluated using fixation-based measures, which can be mapped to predefined objects of interest for task-specific attention analysis. Even though attention can be directed independently from the fovea, little research can be found on the quantification of peripheral vision for attention analysis. In this work, we discuss the benefits of enhancing traditional mapping methods with near-peripheral information and expand previous research by presenting a novel machine learning-based gaze measure, the visual attention index (VAI), for the analysis of visual attention using dynamic stimuli. Results are discussed using the data of two multi-object mobile eye tracking use cases and visualized using radar graphs.

In 2022, ETRA moved its publication of full papers to a journal-based model, and we are delighted to present the second issue of the Proceedings of the ACM on Human-Computer Interaction to focus on contributions from the Eye Tracking Research and Applications (ETRA) community. ETRA is the premier eye-tracking conference that brings together researchers from across disciplines to present advances and innovations in oculomotor research, eye tracking systems, eye movement data analysis, eye tracking applications, and gaze-based interaction. This issue presents 13 full papers accepted for presentation at ETRA 2023 (May 30 - June 2, 2023, in Tübingen, Germany) selected from 37 submissions (35% acceptance rate). We are grateful to all authors for the exciting contributions they have produced and to the Editorial Board and external reviewers for their effort during the entire rigorous reviewing process which resulted in high-quality and insightful reviews for all submitted articles.

The paper contributes to scanpath bundling methods. We propose an analytical approach for statistical comparisons of aggregated scanpath visualizations by means of second-order gaze analysis metrics. The present study explores differences in attention distribution and cognitive processing over architectural objects between architects, art historians, and non-experts. The results show between-group differences in attention dynamics of the aggregated scanpaths. The aggregated scanpaths of both expert groups were focal, while non-experts' scanpaths were ambient. Experts also paid more attention to and tended to remember better the architectural details and their location. The discussion explores the scalability of the proposed approach for Human-Computer Interaction and accessibility technologies designed to enhance the experience of cultural heritage.

This article evaluates the Low/High Index of Pupillary Activity (LHIPA), a measure of cognitive effort based on pupil response, in the context of reading. At the beginning of 2nd and 3rd grade, 107 children (8-9 y.o.) from music and general primary school were asked to read 40 sentences with keywords differing in length and frequency while their eye movements were recorded. Sentences with low frequency or long keywords received more attention than sentences with high frequent or short keywords. The word frequency and length effects were more pronounced in younger children. At the 2nd grade, music children dwelt less on sentences with short frequent keywords than on sentences with long frequent keywords. As expected LHIPA decreased over sentences with low frequency short keywords suggesting more cognitive effort at earlier stages of reading ability. This finding shows the utility of LHIPA as a measure of cognitive effort in education.

We compare the measurement error and validity of webcam-based eye tracking to that of a remote eye tracker as well as software integration of both. We ran a study with n = 83 participants, consisting of a point detection task and an emotional visual search task under three between-subjects experimental conditions (webcam-based, remote, and integrated). We analyzed location-based (e.g., fixations) and process-based eye tracking metrics (ambient-focal attention dynamics). Despite higher measurement error of webcam eye tracking, our results in all three experimental conditions were in line with theoretical expectations. For example, time to first fixation toward happy faces was significantly shorter than toward sad faces (the happiness-superiority effect). As expected, we also observed the switch from ambient to focal attention depending on complexity of the visual stimuli. We conclude that webcam-based eye tracking is a viable, low-cost alternative to remote eye tracking.

Eye movement analysis in modern 3D rendering systems is reviewed and three new techniques are derived inspired by work developed in early Virtual Reality so-called 2.5D implementations, namely (a) gaze depth (i.e., vergence) estimation, (b) vergence calibration, and (c) real-time 3D event detection that considers eye- and head-coupling. The new 3D calibration shows excellent error reduction in terms of Mean Squared Error (MSE).

The Low/High Index of Pupillary Activity (LHIPA), an eye-tracked measure of pupil diameter oscillation, is redesigned and implemented to function in real-time. The novel Real-time IPA (RIPA) is shown to discriminate cognitive load in re-streamed data from earlier experiments. Rationale for the RIPA is tied to the functioning of the human autonomic nervous system yielding a hybrid measure based on the ratio of Low/High frequencies of pupil oscillation. The paper's contribution is drawn from provision of documentation of the calculation of the RIPA. As with the LHIPA, it is possible for researchers to apply this metric to their own experiments where a measure of cognitive load is of interest.

When conducting eye tracking studies, having a mechanism to collect data, build workflows, and validate results in a FAIR (i.e., findable, accessible, interoperable, and reusable) manner, facilitates automation. Given the vast landscape of vendor-specific eye tracking software, adopting FAIR metadata standards for the eye tracking domain is one step towards this. In this paper, we propose an approach to simplify the creation, execution, and validation of eye tracking studies through metadata. Using a metadata format that we developed, we first describe two eye trackers, and two datasets collected using them. Next, we use this metadata to simulate real-time data collection by replaying each dataset. From this replayed data, we analyze eye movements in real-time, and synthesize eye movement data from analytics in real-time. Based on our results, we discuss the utility of metadata in real-time eye tracking studies, and how this idea can be generalized into other applications.

A novel eye-tracked measure of pupil diameter oscillation is derived as an indicator of cognitive load. The new metric, termed the Low/High Index of Pupillary Activity (LHIPA), is able to discriminate cognitive load (vis-a-vis task difficulty) in several experiments where the Index of Pupillary Activity fails to do so. Rationale for the LHIPA is tied to the functioning of the human autonomic nervous system yielding a hybrid measure based on the ratio of Low/High frequencies of pupil oscillation. The paper's contribution is twofold. First, full documentation is provided for the calculation of the LHIPA. As with the IPA, it is possible for researchers to apply this metric to their own experiments where a measure of cognitive load is of interest. Second, robustness of the LHIPA is shown in analysis of three experiments, a restrictive fixed-gaze number counting task, a less restrictive fixed-gaze n-back task, and an applied eye-typing task.

We discuss several techniques for mapping the gaze point to fiducial markers detected in a video stream, as commonly used in Augmented Reality eye-tracking applications. Specifically, we focus on using the recovered camera rotation and translation determined by the apparent distortion of the marker. We exploit this to map Areas Of Interest (AOIs) in the plane relative to the given marker. This gives the advantage of not needing any more than a single marker. We also review how the recovered homography is transformed to the graphics world coordinates so that AOI visualization can be performed in world space instead of camera space.

The paper presents a detailed analysis of modern techniques that can be used to track gaze with a webcam. We present a practical implementation of the most popular methods for tracking gaze. Various models of deep neural networks that can be involved in the process of online gaze monitoring are reviewed. We introduce a new eye-tracking approach where the effectiveness of using a deep learning method is significantly increased. Implementation is in Python where its application is demonstrated by controlling interaction with the computer. Specifically, a dual coordinate system is given for controlling the computer with the help of a gaze. The first set of coordinates-the position of the face relative to the computer, is implemented by detecting color from the infrared LED via the OpenCV library. The second set of coordinates-giving gaze position-is obtained via the YOLO (v3) package. A method of labeling the eyes is given, in which 3 objects are used to track gaze (to the left, to the right, and in the center).

The trade-off between speed and accuracy in precision tasks is important to evaluate during user interaction with input devices. When different sensory cues are added or altered in such interactions, those cues have an effect on this trade-off, and thus, they affect overall user performance. For instance, adding cues like haptic feedback and stereoscopic viewing will result in more realistic user interaction, thus improving performance in these tasks. Also, adding a noticeable disparity between physical and virtual movements creates a mismatch between visual and proprioceptive systems, which generally has a negative effect on performance. In this study, we investigate the effects of haptic feedback, stereoscopic viewing, and visuo-proprioceptive mismatch on how quickly and accurately users complete a virtual pick-and-place task using the PHANToM OMNI. Through this experiment, we find that in the movement phase of a ring transfer, movement time and user performance are affected by haptic feedback and visuo-proprioceptive mismatch, and the main effects of stereoscopic viewing appears to be limited to the more precise step when the ring is around the target peg.

We develop an approach to using microsaccade dynamics for the measurement of task difficulty/cognitive load imposed by a visual search task of a layered surface. Previous studies provide converging evidence that task difficulty/cognitive load can influence microsaccade activity. We corroborate this notion. Specifically, we explore this relationship during visual search for features embedded in a terrain-like surface, with the eyes allowed to move freely during the task. We make two relevant contributions. First, we validate an approach to distinguishing between the ambient and focal phases of visual search. We show that this spectrum of visual behavior can be quantified by a single previously reported estimator, known as Krejtz's K coefficient. Second, we use ambient/focal segments based on K as a moderating factor for microsaccade analysis in response to task difficulty. We find that during the focal phase of visual search (a) microsaccade magnitude increases significantly, and (b) microsaccade rate decreases significantly, with increased task difficulty. We conclude that the combined use of K and microsaccade analysis may be helpful in building effective tools that provide an indication of the level of cognitive activity within a task while the task is being performed.

Research has shown that keyframed eye motions are perceived as more realistic when some noise is added to eyeball motions and to pupil size changes. We investigate whether this noise, in contrast to being motion captured, can be synthesized with standard techniques, e.g., procedural or data-driven approaches. In a two-alternative forced choice task, we compare eye animations created with four different techniques: motion captured, procedural, data-driven, and keyframed (lacking noise). Our perceptual experiment uses three character models with different levels of realism and two motions. Our results suggest that procedural and data-driven noise can be used to create animations at similar perceived naturalness to our motion captured approach. Participants’ eye movements when viewing the animations show that animations without jitter yielded fewer fixations, suggesting ease of dismissal as unnatural.

We examined the effect of rendering style and the interplay between attention and emotion in users during interaction with a virtual patient in a medical training simulator. The virtual simulation was rendered representing a sample from the photo-realistic to the non-photorealistic continuum, namely Near-Realistic, Cartoon or Pencil-Shader. In a mixed design study, we collected 45 participants’ emotional responses and gaze behavior using surveys and an eye tracker while interacting with a virtual patient who was medically deteriorating over time. We used a cross-lagged panel analysis of attention and emotion to understand their reciprocal relationship over time. We also performed a mediation analysis to compare the extent to which the virtual agent’s appearance and his affective behavior impacted users’ emotional and attentional responses. Results showed the interplay between participants’ visual attention and emotion over time and also showed that attention was a stronger variable than emotion during the interaction with the virtual human.

Gaze-Contingent Displays (GCDs) can improve visual search performance on large displays. GCDs, a Level Of Detail (LOD) management technique, discards redundant peripheral detail using various human visual perception models. Models of depth and contrast perception (e.g., depth-of-field and foveation) have often been studied to address the trade-off between the computational and perceptual benefits of GCDs. However, color perception models and combinations of multiple models have not received as much attention. In this paper, we present GeoGCD which uses individual contrast, color, and depth-perception models, and their combination to render scenes without perceptible latency. As proof-of-concept, we present a three-stage user evaluation built upon geographic image interpretation tasks. GeoGCD does not impair users' visual search performance or affect their display preferences. On the contrary, in some cases, it can significantly improve users' performance.

Automatic Area Of Interest (AOI) demarcation of facial regions is not yet commonplace in applied eye-tracking research, partially because automatic AOI labeling is prone to error. Most previous eye-tracking studies relied on manual frame-by-frame labeling of facial AOIs. We present a fully automatic approach for facial AOI labeling (i.e., eyes, nose, mouth) and gaze registration within those AOIs, based on modern computer vision techniques combined with heuristics drawn from art. We discuss details in computing gaze analytics, provide proof-of-concept, and a short validation against what we consider ground truth. Relative dwell time over expected AOIs exceeded 98% showing efficacy of the approach.

We evaluate subtle, emotionally-driven models of eye movement animation. Two models are tested, reading and face scanning, each based on recorded gaze transition probabilities. For reading, simulated emotional mood is governed by the probability density function that varies word advancement, i.e., re-fixations, forward, or backward skips. For face scanning, gaze behavior depends on task (gender or emotion discrimination) or the facial emotion portrayed. Probability density functions in both cases are derived from empirically observed transitions that significantly alter viewing behavior, captured either during mood-induced reading or during scanning faces expressing different emotions. A perceptual study shows that viewers can distinguish between reading and face scanning eye movements. However, viewers could not gauge the emotional valence of animated eye motion. For animation, our contribution shows that simulated emotionally-driven viewing behavior is too subtle to be discerned, or it needs to be exaggerated to be effective.

As large eye-tracking datasets are created, data privacy is a pressing concern for the eye-tracking community. De-identifying data does not guarantee privacy because multiple datasets can be linked for inferences. A common belief is that aggregating individuals' data into composite representations such as heatmaps protects the individual. However, we analytically examine the privacy of (noise-free) heatmaps and show that they do not guarantee privacy. We further propose two noise mechanisms that guarantee privacy and analyze their privacy-utility tradeoff. Analysis reveals that our Gaussian noise mechanism is an elegant solution to preserve privacy for heatmaps. Our results have implications for interdisciplinary research to create differentially private mechanisms for eye tracking.

A significant number of Internet users is unaware of privacy threats and they might not pay much attention to privacy notifications. It is imperative to create effective but, at the same time, non-distracting notifications about privacy policy of different web services. The article presents an eye-tracking study (N = 16) testing the effectiveness of different common types of privacy notification in capturing and retaining users' attention. Results showed that an non-intrusive notification presented at the top of the web page may be as effective as intrusive notifications in capturing attention. Results are discussed in terms of well-known effects of visual attention distribution on web pages. Present findings also offer practical conclusions for web service developers and publishers.

In this contribution we studied how different rendering styles of a virtual human impacted users' visual attention in an interactive medical training simulator. In a mixed design experiment, 78 participants interacted with a virtual human representing a sample from the non-photorealistic (NPR) to the photorealistic (PR) rendering continuity. We presented five rendering style samples scenarios, namely low fidelity all Pencil Shaded (APS), Low to Mid Fidelity Pencil Shader on virtual patient (VP) only (PS), Mid Fidelity All Cartoon Shaded (ACT), Mid to High Fidelity Cartoon Shader on VP only (CT), and relatively High Fidelity Human Like (HL) appearance, and compared how visual attention differed between groups of users. For this study, we employed an eye tracking system for collecting and analyzing users' gaze during interaction with the virtual human in a failure to rescue medical training simulation. Results suggests that users may spend more time in the simulations on the non-realistic fidelity continuum that necessarily do not involve interaction with the virtual human. However, users preferred visually attending to virtual humans in the middle and high fidelity visual appearance conditions when engaging virtual humans in simulated social face-to-face dialogue as compared to the other conditions

We present an empirical evaluation on how stereoscopic viewing and haptic feedback deferentially affects fine motor perception-action coordination in a pick-and-place task in Virtual Reality (VR). The factors considered were stereoscopic viewing, haptic feedback, sensory-motor congruence and mismatch, and calibration on perception-action coordination in near field fine motor task performance in VR. Quantitative measures of placement error, distance, collision, and time to complete trials were recorded and analyzed. Overall, we found that participants' manual dexterous task performance was enhanced in the presence of both stereoscopic viewing and haptic feedback. However, we found that time to complete task was greatly enhanced by the presence of haptic feedback, and economy and efficiency of movement of the end effector as well as the manipulated object was enhanced by the presence of both haptic feedback and stereoscopic viewing. Whereas, number of collisions and placement accuracy were greatly enhanced by the presence of stereoscopic viewing in near-field fine motor perception-action coordination. Our research additionally shows that mismatch in sensory-motor stimuli can detrimentally affect the number of collisions, and efficiency of end effector and object movements in near-field fine motor activities, and can be further negatively affected by the absence of haptic feedback and stereoscopic viewing. In spite of reduced cue situations in VR, and the absence or presence of stereoscopic viewing and haptic feedback, we found that participants tend to calibrate or adapt their perception-action coordination rapidly with a set of at least 5 trials.

No abstract available.

A novel eye-tracked measure of the frequency of pupil diameter oscillation is proposed for capturing what is thought to be an indicator of cognitive load. The proposed metric, termed the Index of Pupillary Activity, is shown to discriminate task difficulty vis-a-vis cognitive load (if the implied causality can be assumed) in an experiment where participants performed easy and difficult mental arithmetic tasks while fixating a central target (a requirement for replication of prior work). The paper's contribution is twofold: full documentation is provided for the calculation of the proposed measurement which can be considered as an alternative to the existing proprietary Index of Cognitive Activity (ICA). Thus, it is possible for researchers to replicate the experiment and build their own software which implements this measurement. Second, several aspects of the ICA are approached in a more data-sensitive way with the goal of improving the measurement's performance.

A variety of psychological disorders like antisocial personality disorder have been linked to impairments in facial emotion recognition. Exploring eye movements during categorization of emotional faces is a promising approach with the potential to reveal possible differences in cognitive processes underlying these deficits. Based on this premise we investigated whether antisocial violent offenders exhibit different scan patterns compared to a matched healthy control group while categorizing emotional faces. Group differences were analyzed in terms of attention to the eyes, extent of exploration behavior and structure of switching patterns between Areas of Interest. While we were not able to show clear group differences, the present study is one of the first that demonstrates the feasibility and utility of incorporating recently developed eye movement metrics such as gaze transition entropy into clinical psychology.

Complex information visualizations, such as thematic maps, encode information using a particular symbology that often requires the use of a legend to explain its meaning. Traditional legends are placed at the edge of a visualization, which can be difficult to maintain visually while switching attention between content and legend.

This paper demonstrates the utility of ambient-focal attention and pupil dilation dynamics to describe visual processing of emotional facial expressions. Pupil dilation and focal eye movements reflect deeper cognitive processing and thus shed more light on the dynamics of emotional expression recognition. Socially anxious individuals (N = 24) and non-anxious controls (N = 24) were asked to recognize emotional facial expressions that gradually morphed from a neutral expression to one of happiness, sadness, or anger in 10-sec animations. Anxious cohorts exhibited more ambient face scanning than their non-anxious counterparts. We observed a positive relationship between focal fixations and pupil dilation, indicating deeper processing of viewed faces, but only by non-anxious participants, and only during the last phase of emotion recognition. Group differences in the dynamics of ambient-focal attention support the hypothesis of vigilance to emotional expression processing by socially anxious individuals. We discuss the results by referring to current literature on cognitive psychopathology.

Creating realistic animations of virtual humans remains comparatively complex and expensive. This research explores the degree to which animation fidelity affects users' gaze behavior when interacting in virtual reality training simulations that include virtual humans. Participants were randomly assigned to one of three conditions, wherein the virtual patient either: 1) was not animated; 2) played idle animations; or 3) played idle animations, looked at the participant when speaking, and lip-synced speech and facial gestures when conversing with the participant. Each participant's gaze was recorded in an inter-personal interactive patient surveillance simulation. Results suggest that conversational and passive animations elicited visual attention in a similar manner, as compared to the no animation condition. Results also suggest that when participants face critical situations in inter-personal medical simulations, visual attention towards the virtual human decreases while gaze towards goal directed activities increases.

Abstract Eye movements are an essential part of non‐verbal behavior. Non‐player characters, as they occur in many games, communicate with the player through dialogue and non‐verbal behavior and can have a strong influence on player experience or even on gameplay. In this paper, we evaluate a procedural model designed to synthesize the subtleties of eye motion. More specifically, our model adds microsaccadic jitter and pupil unrest both modeled by 1/fαor pink noise to the saccadic main sequence. In a series of perceptual two‐alternative forced‐choice experiments, we explore the perceived naturalness of different parameters of pink noise by comparing synthesized motions to rendered motion of recorded eye movements at extreme close shot and close shot distances. Our results show that, on average, animations based on a procedural model with pink noise were perceived and evaluated as highly natural, whereas data‐driven motion without any jitter or with unfiltered jitter were consistently selected as the least natural in appearance.

Eye motions constitute an important part of our daily face-to-face interactions. Even subtle details in the eyes’ motions give us clues about a person’s thoughts and emotions. Believable and natural animation of the eyes is therefore crucial when creating appealing virtual characters. In this article, we investigate the perceived naturalness of detailed eye motions, more specifically of jitter of the eyeball rotation and pupil diameter on three virtual characters with differing levels of realism. Participants watched stimuli with six scaling factors from 0 to 1 in increments of 0.2, varying eye rotation and pupil size jitter individually, and they had to indicate if they would like to increase or decrease the level of jitter to make the animation look more natural. Based on participants’ responses, we determine the scaling factors for noise attenuation perceived as most natural for each character when using motion-captured eye motions. We compute the corresponding average jitter amplitudes for the eyeball rotation and pupil size to serve as guidelines for other characters. We find that the amplitudes perceived as most natural depend on the character, with our character with a medium level of realism requiring the largest scaling factors.

We detail the design and evaluation of a rotary interface for gaze-based PIN code entry. Interface design promotes equal distance between PIN numerals, leading to a circular layout resulting in the choice of a rotary telephone dial metaphor. The rotary's speed advantage over the traditional grid-based (e.g., keypad) design is derived from its elimination of dwell time for gaze-based numeral selection, relying instead on a weighted voting scheme of numerals whose boundaries are crossed by the streaming (smoothed) gaze points. Screen center-bias is exploited by requiring users to transition to and from the center to the rotary numerals, in order to enter in the PIN code sequence. Compared with the keypad layout, empirical results show that PIN digit entry errors do not differ significantly between interfaces, although the rotary incurs fewer errors overall. Expressing preference for the rotary, users appeared to quickly grasp its operation.

A convolution-filtering technique is introduced for the synthesis of eye gaze data. Its purpose is to produce, in a controlled manner, a synthetic stream of raw gaze position coordinates, suitable for: (1) testing event detection filters, and (2) rendering synthetic eye movement animations for testing eye tracking gaze estimation algorithms. Synthetic gaze data is parameterized by sampling rate, microsaccadic jitter, and simulated measurement error. Sampled synthetic gaze data is compared against real data captured by an eye tracker showing similar signal characteristics.

We introduce coefficient K , defined on a novel parametric scale, derived from processing a traditionally eye-tracked time course of eye movements. Positive and negative ordinates of K indicate focal or ambient viewing, respectively, while the abscissa serves to indicate time, so that K acts as a dynamic indicator of fluctuation between ambient/focal visual behavior. The coefficient indicates the difference between fixation duration and its subsequent saccade amplitude expressed in standard deviation units, facilitating parametric statistical testing. To validate K empirically, we test its utility by capturing ambient and focal attention during serial and parallel visual search tasks (Study 1). We then show how K quantitatively depicts the difference in scanning behaviors when attention is guided by audio description during perception of art (Study 2).

Eye movements are an essential part of non-verbal behavior. Non-player characters (NPCs), as they occur in many games, communicate with the player through dialogues and non-verbal behavior and can have a strong influence on the player experience or even on gameplay. In this paper we propose a procedural model to synthesize the subtleties of eye motions. More specifically, our model adds microsaccadic jitter and pupil unrest both modeled by 1/f or pink noise to the standard main sequence. In a perceptual two-alternative forced-choice (2AFC) experiment we explore the perceived naturalness of different parameters of pink noise by comparing synthesized motions to rendered motion of recorded eye movements at extreme close shot and close shot distances. Our results show that, on average, data-driven motion is perceived as most natural, followed by parameterized pink noise, with motion lacking microsaccadic jitter being consistently selected as the least natural in appearance.

This article details a two-step method of quantifying eye movement transitions between areas of interest (AOIs). First, individuals' gaze switching patterns, represented by fixated AOI sequences, are modeled as Markov chains. Second, Shannon's entropy coefficient of the fit Markov model is computed to quantify the complexity of individual switching patterns. To determine the overall distribution of attention over AOIs, the entropy coefficient of individuals' stationary distribution of fixations is calculated. The novelty of the method is that it captures the variability of individual differences in eye movement characteristics, which are then summarized statistically. The method is demonstrated on gaze data collected from two studies, during free viewing of classical art paintings. Normalized Shannon's entropy, derived from individual transition matrices, is related to participants' individual differences as well as to either their aesthetic impression or recognition of artwork. Low transition and high stationary entropies suggest greater curiosity mixed with a higher subjective aesthetic affinity toward artwork, possibly indicative of visual scanning of the artwork in a more deliberate way. Meanwhile, both high transition and stationary entropies may be indicative of recognition of familiar artwork.

We report on an experiment testing gaze-contingent depth-of-field (DOF) for the reduction of visual discomfort when viewing stereoscopic displays. Subjective results are compelling, showing, we believe for the first time, that gaze-contingent depth-of-field significantly reduces visual discomfort. When individual stereoacuity is taken into account, objective measurements of gaze vergence corroborate previous reports, showing significant bias toward the zero depth plane, where error is smallest. As with earlier similar attempts, participants expressed a dislike toward gaze-contingent DOF. Although not statistically significant, this dislike is likely attributed to the eye tracker's spatial inaccuracy as well as the DOF simulation's noticeable temporal lag.

We show that the error in 3D gaze depth (vergence) estimated from binocularly-tracked gaze disparity is related to the viewing distance of the screen calibration plane at which 2D gaze is recorded. In a stereoscopic (virtual) environment, this relationship is evident in gaze to target depth error: vergence error behind the screen is greater than in front of the screen and is lowest at the screen depth. In a physical environment, with no accommodation-vergence conflict, the magnitude of vergence error in front of the 2D calibration plane appears reversed, increasing with distance from the viewer.

The paper introduces a two-step method of quantifying eye movement transitions between Areas of Interests (AOIs). First, individuals' gaze switching patterns, represented by fixated AOI sequences, are modeled as Markov chains. Second, Shannon's entropy coefficient of the fit Markov model is computed to quantify the complexity of individual switching patterns. To determine the overall distribution of attention over AOIs, the entropy coefficient of individuals' stationary distribution of fixations is calculated.

Abstract This paper presents the evolution of a data structures and algorithms course based on a specific computer graphics problem, namely, photon mapping, as the teaching medium. The paper reports development of the course through several iterations and evaluations, dating back 5 years. The course originated as a problem‐based graphics course requiring sophomore students to implement Hoppe et al.'s algorithm for surface reconstruction from unorganized points found in their SIGGRAPH '92 paper of the same title. Although the solution to this problem lends itself well to an exploration of data structures and code modularization, both of which are traditionally taught in early computer science courses, the algorithm's complexity was reflected in students' overwhelmingly negative evaluations. Subsequently, because implementation of the kd‐tree was seen as the linchpin data structure, it was again featured in the problem of ray tracing trees consisting of more than 250 000 000 triangles. Eventually, because the tree rendering was thought too specific a problem, the photon mapper was chosen as the semester‐long problem considered to be a suitable replacement. This paper details the resultant course description and outline, from its now three semesters of teaching.

This article summarizes our previous work on developing an online system to allow the estimation of 3D gaze depth using eye tracking in a stereoscopic environment. We report on recent extensions allowing us to report the full 3D gaze position. Our system employs a 3D calibration process that determines the parameters of a mapping from a naive depth estimate, based simply on triangulation, to a refined 3D gaze point estimate tuned to a particular user. We show that our system is an improvement on the geometry-based 3D gaze estimation returned by a proprietary algorithm provided with our tracker. We also compare our approach with that of the Parameterized Self-Organizing Map (PSOM) method, due to Essig and colleagues, which also individually calibrates to each user. We argue that our method is superior in speed and ease of calibration, is easier to implement, and does not require an iterative solver to produce a gaze position, thus guaranteeing computation at the rate of tracker acquisition. In addition, we report on a user study that indicates that, compared with PSOM, our method more accurately estimates gaze depth, and is nearly as accurate in estimating horizontal and vertical position. Results are verified on two different 4D eye tracking systems, a high accuracy Wheatstone haploscope and a medium accuracy active stereo display. Thus, it is the recommended method for applications that primarily require gaze depth information, while its ease of use makes it suitable for many applications requiring full 3D gaze position.

The paper explores the effects of adding audio description to an educational film on children's learning behaviour, manifested by a visual recognition task. We hypothesize that the multimodal educational setting, consisting of both verbal (film dialogue and audio description) and non-verbal (motion pictures) representations of knowledge, fosters knowledge acquisition as it provides information via multiple channels, which in turn strengthens memory retrieval. In the study we employ eye tracking methodology to examine the recognition of previously seen film material, testing whether audio description promotes recognition- rather than elimination-based decision-making in the visual recognition task. The analysis of first fixation duration and first run fixation count measures in the experimental and control groups partially confirmed our hypotheses. Children in the experimental group generally looked longer at the scenes they had seen, which supports the hypothesis that their decision was based on recognition, whereas children in the control group had longer fixations on scenes they were unfamiliar with, suggesting a decision based on elimination.

The use of recorded eye movements, or scanpaths, has been demonstrated as an effective visualization for feed-forward visual search training, instruction, and stimulated retrospective think-aloud usability testing. In this paper we show that creation of a scripted or recorded video of an expert's think-aloud session augmented by an animation of their scanpaths can result in an effective aid for learners of visual search. Because the creation of such a video is relatively easy, the benefits-to-cost ratio may potentially be substantial, especially in settings where learned visual scanning strategies are indicators of expertise. We suggest that two such examples are examinations of Chest X-Rays and histological slides. Results are presented where straightforward construction of an instruction video provides measurable benefit to novice as well as experienced learners in the latter context.

A GPU implementation is given for real-time visualization of aggregate eye movements (gaze) via heatmaps. Parallelization of the algorithm leads to substantial speedup over its CPU-based implementation and, for the first time, allows real-time rendering of heatmaps atop video. GLSL shader colorization allows the choice of color ramps. Several luminance-based color maps are advocated as alternatives to the popular rainbow color map, considered inappropriate (harmful) for depiction of (relative) gaze distributions.

Audio description (AD) has become a cultural revolution for the visually impaired; however, the range of AD beneficiaries can be much broader. We claim that AD is useful for guiding children's attention. The paper presents an eye-tracking study testing the usefulness of AD in selective attention to described elements of a video scene. Forty-four children watched 2 clips from an educational animation series while their eye movements were recorded. Average fixation duration, fixation count, and saccade amplitude served as primary dependent variables. The results confirmed that AD guides children's attention towards described objects resulting e. g., in more fixations on specific regions of interest. We also evaluated eye movement patterns in terms of switching between focal and ambient processing. We postulate that audio description could complement regular teaching tools for guiding and focusing children's attention, especially when new concepts are introduced.

Previous work has shown significant differences in eye movement metrics recorded by devices differing in sampling rates. Two schools of thought have emerged on how to effectively compare such apparently disparate data. The first, termed here as upsampling, strives to process eye movement data recorded at a low sampling rate to allow comparison with data recorded at a high sampling rate, e. g., by fitting a cubic spline to the signal derivative (i.e., velocity). Instead, we suggest downsampling based on a two-pass solution in which data is first downsampled and smoothed prior to its velocity-based classification. Results indicate that given a similar experimental task, this approach gives more equitable results than other single-pass classification methods as they typically do not explicitly consider sampling rates.

Fiducial markers are used with head-mounted eye trackers to facilitate eye movement data aggregation for quantitative analysis. However, use of these markers may be problematic in some situations (e.g., natural tasks) as the markers may be visually distracting. To date, we are aware of no study that has examined the conspicuity of such markers to determine how much (if any) effort should be expended into concealing them from view. This paper presents a study that examines Tobii's infra-red (IR) markers' conspicuity in a 2-D projected environment. Results indicate that even when these 3-D markers are superimposed on a canvas on which the 2-D environment is projected, and no effort is taken to hide them (i.e., by minimizing contrast with the background), the presence of markers does not significantly alter the number or duration of fixations on the location of the markers when a specific task is given.

Recent developments in mobile eye tracking equipment and automated eye movement analysis point the way toward unobtrusive eye-based human-computer interfaces that are pervasively usable in everyday life. We call this new paradigm pervasive eye tracking - continuous eye monitoring and analysis 24/7. PETMEI 2011 provides a forum for researcher from human-computer interaction, context-aware computing, and eye tracking to discuss techniques and applications that go beyond classical eye tracking and stationary eye-based interaction. We aim to discuss the implications of pervasive eye tracking for context-aware computing and to identify the key research challenges of mobile eye-based interaction. The long-term goal is to create a strong interdisciplinary research community linking these research fields together and to establish the workshop as the premier forum for research on pervasive eye tracking and mobile eye-based interaction.

The TEXNH method is an approach to undergraduate computer science education that is based on cognitive constructivisim, in the sense of Piaget, and which invokes several course design directives that include re-combining art and science, problem-based learning, problem selection from the visual problem domain, and cognitive apprenticeship. The paper describes a new TEXNH course in data structures. It includes a full comparative assessment of the realized improvement in student problem solving capability and, for the first time, cognitive authenticity in problem selection, in that the course problem is a variation on a very recent research result.

A remote eye tracker is used to explore its utility for ocular vergence measurement. Subsequently, vergence measurements are compared in response to anaglyphic stereographic stimuli as well as in response to monoscopic stimulus presentation on a standard display. Results indicate a highly significant effect of anaglyphic stereoscopic display on ocular vergence when viewing a stereoscopic calibration video. Significant convergence measurements were obtained for stimuli fused in the anterior image plane.

The scanpath comparison framework based on string editing is revisited. The previous method of clustering based on k-means "preevaluation" is replaced by the mean shift algorithm followed by elliptical modeling via Principal Components Analysis. Ellipse intersection determines cluster overlap, with fast nearest-neighbor search provided by the kd-tree. Subsequent construction of Y - matrices and parsing diagrams is fully automated, obviating prior interactive steps. Empirical validation is performed via analysis of eye movements collected during a variant of the Trail Making Test, where participants were asked to visually connect alphanumeric targets (letters and numbers). The observed repetitive position similarity index matches previously published results, providing ongoing support for the scanpath theory (at least in this situation). Task dependence of eye movements may be indicated by the global position index, which differs considerably from past results based on free viewing.

We present a novel method for the measurement of the similarity between aggregates of scanpaths. This may be thought of as a solution to the "average scanpath" problem. As a by-product of this method, we derive a classifier for groups of scanpaths drawn from various classes. This capability is empirically demonstrated using data gathered from an experiment in an attempt to automatically determine expert/novice classification for a set of visual tasks.

There is no standard software interface in the eye-tracking industry, making it difficult for developers to integrate eye-gaze into their applications. The combination of high cost eye-trackers and lack of applications has resulted in a slow adoption of the technology. To expand the adoption of eye-gaze in everyday applications, we present an eye-gaze specific application programming interface that is platform and language neutral, based on open standards, easily used and extended and free of cost.

Analysis of recordings made by a wearable eye tracker is complicated by video stream synchronization, pupil coordinate mapping, eye movement analysis, and tracking of dynamic Areas Of Interest (AOIs) within the scene. In this paper a semi-automatic system is developed to help automate these processes. Synchronization is accomplished via side by side video playback control. A deformable eye template and calibration dot marker allow reliable initialization via simple drag and drop as well as a user-friendly way to correct the algorithm when it fails. Specifically, drift may be corrected by nudging the detected pupil center to the appropriate coordinates. In a case study, the impact of surrogate nature views on physiological health and perceived well-being is examined via analysis of gaze over images of nature. A match-moving methodology was developed to track AOIs for this particular application but is applicable toward similar future studies.

Empirical findings from a gaze-contingent color degradation study report the effects of artificial reduction of the human visual system's sensitivity to peripheral chromaticity on visual search performance. To our knowledge, this is the first such investigation of peripheral color reduction. For unimpeded performance, results suggest that, unlike spatiotemporal content, peripheral chromaticity cannot be reduced within the central 20° visual angle. Somewhat analogous to dark adaptation, reduction of peripheral color tends to simulate scotopic viewing conditions. This holds significant implications for chromatic Level Of Detail management. Specifically, while peripheral spatiotemporal detail can be attenuated without affecting visual search, often dramatically (e.g., spatial detail can be so reduced up to 50% at about 5°), peripheral chromatic reduction is likely to be noticed much sooner. Therefore, color LOD reduction (e.g., via compression), should be maintained isotropically across the central 20° visual field.

A nonisotropic hybrid image/model-based gaze-contingent rendering technique utilizing ray casting on a GPU is discussed. Empirical evidence derived from human subject experiments indicates an inverse relationship between a peripherally degraded scene's high-resolution inset size and mean search time, a trend consistent with existing image-based and model-based techniques. In addition, the data suggest that maintaining a target's silhouette edges decreases search times when compared to targets with degraded edges. However, analysis suggests a point of diminishing returns with an inset larger than 15° when target discrimination is a component of visual search. Benefits of the hybrid technique include simplicity of design and parallelizability, both conducive to GPU implementation.

We present a low-cost wearable eye tracker built from off-the-shelf components. Based on the open source openEyes project (the only other similar effort that we are aware of), our eye tracker operates in the visible spectrum and variable lighting conditions. The novelty of our approach rests in automatically switching between tracking the pupil/iris boundary in bright light to tracking the iris/sclera boundary (limbus) in dim light. Additional improvements include a semi-automatic procedure for calibrating the eye and scene cameras, as well as an automatic procedure for initializing the location of the pupil in the first image frame. The system is accurate to two degrees visual angle in both indoor and outdoor environments.

Eye-typing performance results are reported from controlled studies comparing an on-screen keyboard and Eye Write, a new on-screen gestural input alternative. Results from the first pilot study suggest the presence of a learning curve that novice users must overcome in order to gain proficiency in EyeWrite's use (requiring practice with its letter-like gestural alphabet). Results from the second longitudinal study indicate that EyeWrite's inherent multi-saccade handicap (4.52 saccades per character, frequency-weighted average) is sufficient for the on-screen keyboard to edge out Eye Write in speed performance. Eye-typing speeds with Eye Write approach 5 wpm on average (8 wpm attainable by proficient users), whereas keyboard users achieve about 7 wpm on average (in line with previous results). However, Eye Write users leave significantly fewer uncorrected errors in the final text, with no significant difference in the number of errors corrected during entry, indicating a speed-accuracy trade-off. Subjective results indicate that participants consider Eye Write significantly faster, easier to use, and prone to cause less ocular fatigue than the on-screen keyboard. In addition, Eye-Write consumes much less screen real-estate than an on-screen keyboard, giving it practical advantages for eye-based text entry.

Advancements in graphics hardware have allowed development of hardware-accelerated imaging displays. This article reviews techniques for real-time simulation of arbitrary visual fields over still images and video. The goal is to provide the vision sciences and perceptual graphics communities techniques for the investigation of fundamental processes of visual perception. Classic gaze-contingent displays used for these purposes are reviewed and for the first time a pixel shader is introduced for display of a high-resolution window over peripherally degraded stimulus. The pixel shader advances current state-of-the-art by allowing real-time processing of still or streamed images, obviating the need for preprocessing or storage.

Aircraft inspection is a vital element in assuring safety and reliability of the air transportation system. The human inspector performing visual inspection of an aircraft is the backbone of this process and training is an effective strategy for improving their inspection performance. Previous studies have shown offline feedback training to be effective in improving subsequent visual inspection performance. Because experienced inspectors are known to adopt a better inspection strategy than novices, providing visualization of experts' cognitive processes a priori can accelerate novices' adoption of the experts' strategy. Using eye tracking equipment, we record the point of regard of an expert inspector performing an inspection task in a virtual reality simulator. Analysis of their eye movements leads to a visualization of their scanpaths and allows us to display the inspector's visual search (hence cognitive) strategy. We show how providing this type of scanpath-based feedforward training of novices leads to improved accuracy performance in the simulator coupled with an observed speed-accuracy tradeoff. We contend that the tradeoff results from trained novices adopting a slower paced strategy through increased fixation durations, suggesting trained novices learn a more deliberate target search/discrimination strategy that requires more time to execute.

Navigation in Virtual Environments (VEs) requires users to develop spatial knowledge of the environment primarily through visual cues provided to the user. Thus, the design and display of visual navigation cues is important for efficient navigation in a VE. In this paper, we report the results of an experiment in which three different visual cues were tested for their benefit toward users' navigation in a 3D virtual maze. The experiment varied the form of visual cue: a 2D map, a 2D map with a directionally ambiguous cue, and a 2D map with a directional cue. Eye tracking data was collected and analyzed to examine the correlation between the type of visual cue presented and the navigational efficiency of the user through the virtual maze. It was observed that the cue type affected performance of the participant in the 3D maze. The directional cue was most effective in the time taken by users to reach the center of the maze. Results of this study have implications for VE design as well as for game development.

Recent advancements in computer graphics hardware have made it possible to develop hardware-accelerated real-time imaging displays. This poster presents technical details of an OpenGL multitexturing approach for real-time simulation of arbitrary visual fields over a still image. Mipmapping facilitates in-hardware dyadic (power-of-two) degradation of the image to serve as the low-resolution periphery. Multitexture compositing provides a mechanism to combine the image's high-resolution pixels within a window (of arbitrary shape). The poster presents code examples to achieve gaze-contingent movement of the high-resolution window over a smoothly or coarsely degraded version of the image background as shown in Figure 1.To test the display rate of the multitexturing approach, the "runway" image (1024 × 512) of Figure 1 was shown in a 1280 × 1024 display window (sufficient to fill a 17'' flat panel). Image rendering was timed in code as the foveal Region Of Interest (ROI) was made to move continuously over the image (diagonally from the top-right corner to the bottom-left, maintaining the full ROI within the window). Sustained display rates (in frames per second) reported by the timer averaged 120 fps on three different platforms.The multitexturing display was successfully adapted to a gaze-contingent application using the Tobii eye tracker for real-time gaze measurement. The Tobii eye tracker can be configured in several ways, one of which is acting as a server for a (possibly remote) eye tracking client application. Communication between client and server occurs over TCP/IP. A Linux PC, used for most of the eye tracker code testing, was set up with dual monitors, as shown in Figure 1, with the Tobii server running on a Windows 2000 PC.

This paper evaluates the use of Visual Deictic Reference (VDR) in Collaborative Virtual Environments (CVEs). A simple CVE capable of hosting two (or more) participants simultaneously immersed in the same virtual environment is used as the testbed. One participant's VDR, obtained by tracking the participant's gaze, is projected to co-participants' environments in real-time as a colored lightspot. We compare the VDR lightspot when it is eye-slaved to when it is head-slaved and show that an eye-slaved VDR helps disambiguate the deictic point of reference, especially during conditions when the user's line of sight is decoupled from their head direction.

No abstract available.

This paper presents an improved 3D eye movement analysis algorithm for binocular eye tracking within Virtual Reality for visual inspection training. The user's gaze direction, head position and orientation are tracked to allow recording of the user's fixations within the environment. The paper summarizes methods for (1) integrating the eye tracker into a Virtual Reality framework, (2) calculating the user's 3D gaze vector, and (3) calibrating the software to estimate the user's inter-pupillary distance post-facto. New techniques are presented for eye movement analysis in 3D for improved signal noise suppression. The paper describes (1) the use of Finite Impulse Response (FIR) filters for eye movement analysis, (2) the utility of adaptive thresholding and fixation grouping, and (3) a heuristic method to recover lost eye movement data due to miscalibration. While the linear signal analysis approach is itself not new, its application to eye movement analysis in three dimensions advances traditional 2D approaches since it takes into account the 6 degrees of freedom of head movements and is resolution independent. Results indicate improved noise suppression over our previous signal analysis approach.

This paper presents novel software techniques for binocular eye tracking within Virtual Reality and discusses their application to aircraft inspection training. The aesthetic appearance of the environment is driven by standard graphical techniques augmented by realistic texture maps of the physical environment. The user's gaze direction, as well as head position and orientation, are tracked to allow recording of the user's fixations within the environment. Methods are given for (1) integration of the eye tracker into a Virtual Reality framework, (2) stereo calculation of the user's 3D gaze vector, (3) a new 3D calibration technique developed to estimate the user's inter-pupillary distance post-facto, and (4) a new technique for eye movement analysis in 3-space. The 3D eye movement analysis technique is an improvement over traditional 2D approaches since it takes into account the 6 degrees of freedom of head movements and is resolution independent. Results indicate that although the current signal analysis approach is somewhat noisy and tends to underestimate the identified number of fixations, recorded eye movements provide valuable human factors process measures complementing performance statistics used to gauge training effectiveness.

This paper describes the development of a binocular eye tracking Virtual Reality system for aircraft inspection training. The aesthetic appearance of the environment is driven by standard graphical techniques augmented by realistic texture maps of the physical environment. A “virtual flashlight” is provided to simulate a tool used by inspectors. The user's gaze direction, as well as head position and orientation, are tracked to allow recording of the user's gaze locations within the environment. These gaze locations, or scanpaths, are calculated as gaze/polygon intersections, enabling comparison of fixated points with stored locations of artificially generated defects located in the environment interior. Recorded scanpaths provide a means of comparison of the performance of experts to novices, thereby gauging the effects of training.

A wavelet-based multiresolution image representation method is developed matching human visual system (HVS) spatial acuity within multiple regions of interest (ROIs). ROIs are maintained at high (original) resolution while peripheral areas are gracefully degraded. Variable resolution images are generated by selectively scaling wavelet (detail) coefficients prior to reconstruction. The technique is equivalent to linear interpolation MIP-mapping which involves smooth subsampling (decomposition) prior to texture mapping (reconstruction). Multiple ROI degradation is achieved through wavelet coefficient scaling following Voronoi partitioning of the image plane.

Interactive visualization tools for models of computation provide a more compelling means of exploration and feedback than traditional paper and pencil methods in theory of computation courses. The Java Computability Toolkit (JCT) is introduced here as a new teaching aide and as an exploratory student's supplement to a course on theory of computation. JCT consists of two Java multiple-window, web-accessible, graphical environments, allowing the construction and simulation of finite automata and Turing machines. This paper discusses JCT's use, design, and applications in teaching.

In this article we develop an automatic deadlock detection scheme for general purpose simulation systems and offer deadlock-resolution algorithms for grouped resources and overlapping resources. Our intent is to provide capabilities that can be incorporated in professional simulation systems to facilitate modeling situations where deadlocking is a problem. Two general modeling situations are analyzed. In grouped-resources models, a set of resources is required to complete a task, whereas in overlapping-resources models, movement to the next process requires first gaining control of the next resource. Deadlocks occurring in these two different situations require different resolution algorithms. We develop a graph-theoretic description of the simulation over time and define deadlocks in terms of strongly connected components of this graph. We make a distinction between transient deadlocks and permanent deadlocks. A prototype implementation of the detection and resolution algorithms was incorporated in a simulation system to provide information concerning the computational requirements and performance of these algorithms. We demonstrate that resolving a possible transient deadlock before it becomes permanent is worthwhile in conjunction with overlapping resources, and that the computational performance of the approach is reasonable except for situations with frequent occurrences of overlapping-resources deadlocks.