Eye-hand coordination under the microscope is a fundamental skill to many microsurgical specializations. In this work, we investigate the oculomotor and motor differences between novice and expert surgeons in the classical eye-hand coordination Fitts’ law task. However, the field of view under the microscope is never static or stable due to inherent internal and external artifacts. In this work, we present methodological steps necessary for stabilizing the operation field of view using RANSAC and for defining static areas of interest (i.e., task targets) and dynamic areas of interest using YOLOv8 (i.e., instrument movement). When combined, we observed that under the microscope, eye movements were spatially more focused than hand movements and that experts relied on parafoveal vision when targeting large targets. Our work opens the path to objective and autonomous skill-acquisition assessment in microsurgical expertise.

Data quality is essential in eye tracking. One of the most common aspects of it is the spatial difference between actual and reported gaze position, typically measured with accuracy and precision. While reporting these numbers is important for understanding and evaluating eye tracking studies, it is equally important to understand how these numbers came about. Without knowledge of the underlying operationalization choices, the numbers themselves are difficult to compare or interpret meaningfully. We focus on computational decisions that arise once 2D gaze data for a screen-based study has been established. Within this scope, we identify eight choices that researchers deliberately or implicitly make when computing data quality metrics: computation and reporting units, small-angle approximation, actual or normalized gaze origin, distance metric, central tendency function, coordinate system, precision measure, and aggregation method. For each choice, we formalize the mathematical alternatives and illustrate their practical consequences. Our goal is to showcase the variety of computational operationalization approaches, to help researchers understand what choices they have made, and to motivate making these choices explicit. This contributes to strengthening the comparability and reproducibility of data quality reports across eye tracking studies.

We present PyeLink and SyeLink, two complementary open-source Python tools for running eye-tracking experiments and parsing data from SR Research Ltd. EyeLink eye tracking hardware. PyeLink simplifies experiment creation through plug-and-play support for multiple display backends (Pygame, PsychoPy, Pyglet) while enabling two features not available in existing tools: recording data during calibration and validation phases, and simultaneously recording raw P-CR data alongside calibrated gaze data. These enriched data files cannot be processed by existing parsers, which motivated the development of SyeLink. SyeLink parser processes recorded files and makes the data accessible through both Python API and command-line interface. SyeLink converts raw ASC data into structured JSON and CSV formats and supports visualization of calibration and validation results. Together, these tools provide a complete Python workflow for running eye-tracking experiments with enriched data recording capabilities while exposing all configurable aspects of EyeLink eye trackers.

Certain research paradigms require close eye-to-screen distances. EyeLink 1000 Plus is one of the most widely used eye trackers, yet there is no documented guidance on the minimum eye-to-screen distance that still allows successful calibration. We tested three lenses (16mm, 25mm, 35mm) to determine this minimum while maintaining at least 15° of visual angle in the smallest dimension. The 16mm and 25mm lenses allowed the closest distances (330–335 mm to screen top), while the 35mm lens required 457 mm. All configurations achieved average validation accuracy under 0.30°.

This paper presents PyEtSimul, an open-source Python-based framework for simulating video-based eye trackers by generating synthetic eye features through geometric modeling. The framework allows flexible positioning of eyes, cameras, and light sources in 3D space, with controlled variation of eye anatomical features and camera properties. PyEtSimul generalizes corneal modeling by representing the cornea as a conic surface rather than the common sphere. It also supports non-circular pupil shapes, size-dependent pupil decentration, eyelid occlusion, and camera lens distortion. It supports systematic data generation and principled comparison of gaze estimation algorithms across calibrated and uncalibrated settings. These features enable analyses not possible with other available simulators. PyEtSimul facilitates controlled experiments with known parameters often latent in normal settings, enabling reproducible benchmarking and systematic exploration of hardware designs. By generating fully synthetic data, PyEtSimul removes privacy concerns and the need for costly hardware, making it practical for both educational and research applications.

Pupil size is a key eye-based indicator of mental processing and internal states. Nonetheless, pupil size is also influenced by light, causing challenges for monitoring internal states with eye tracking in dynamic and naturalistic settings. We investigated how convolution-based modeling can capture the influence of luminance and emotional arousal on pupil size, and conversely, how pupil dynamics inform arousal prediction. In a lab study, we analyzed data collected from 19 participants who watched travel-themed videos with different arousal levels and their pixelated counterparts. We present and evaluate a convolution-based approach for pupil size modeling that integrates low-level visual features and higher-order emotional factors, as previous work has shown effectiveness of such methods in modeling pupil light reflex. Our results show that incorporating luminance and contrast indeed enhances arousal prediction from pupil data, although performance varies due to different user behavior and stimuli.

Eye tracking has shown promise in detecting groupwise differences between healthy controls and people with Mild Cognitive Impairment (MCI) who might progress to develop Alzheimer's disease. As there is currently no cure, only medications to slow the progress of the disease, it is of paramount importance to find the people at risk early. We extracted saccade- and Region of Interest (ROI)-centric feature sets from a clinical eye tracking dataset and analyzed the results using machine learning to establish which features were most beneficial to correctly classify individuals at risk. Our results show that the analysis of multimodal eye tracking recordings of number and text reading tasks produces feature sets that can have good predictive value to classify MCIs from healthy controls. Changes in participants' pupil sizes from the personalized baseline appear to be especially promising candidates for improving the classification efficiency of MCIs.

The HumanEYEze 2024 workshop aims to explore the role of eye tracking in developing human-centered multimodal AI systems. Over the past two decades, eye tracking has evolved from a diagnostic tool to an important input modality for real-time interactive systems, driven by advancements in hardware that have improved its affordability, availability, and performance. Initially used in specialized applications, eye tracking now significantly impacts research on gaze-based multimodal interaction. Recently, eye-based user and context modeling has emerged, utilizing eye movements to provide rich insights into user behavior and interaction contexts. The workshop aims to bring together researchers from eye tracking, multimodal human-computer interaction, and AI. It aims to enhance understanding of integrating eye tracking into multimodal human-centered computing. The expected outcomes include fostering collaborations and promoting knowledge exchange.

People with vision impairments see the world differently. Combining the forces of ophthalmology and modern computing we employ Unreal Engine 5 with Steam VR base stations, an eye-tracking capable Varjo Aero VR headset, and a handheld slate with an embedded HTC Vive tracker to record the changes in the person’s functional vision and make the implied changes in visual functions explicit by applying the resulting modifications to a live VR scene. The setup enables us to track participant’s head, gaze and object movements simultaneously, which in turn produces more data that can help in diagnosis and in teaching of assistive skills like steady eye technique.

The required skills for medical personnel in surgical operating rooms are not confined to medical and technical expertise. To understand how scrub nurses perform their critical role, cognitive, social, and collaborative skills are at the centre of interest. To navigate in a life-critical environment and react both appropriately and in a timely manner to the surgeon's actions, nurses need to have a fast and accurate understanding of the situation. We present how this reading of the situation is conducted during microsurgeries, where a surgical microscope takes a central role in the treatment. We employed contextual observation, gaze-tracking, video recordings, and interviews to describe the reliance of the surgeons’ work on the microscope, as well as the situation-understanding strategies of the nurses in such a setting. Also, we present pieces of evidence of anticipation strategies that the nurses exploit in the face of challenges due to the main surgeon’s lack of attention toward them because of the constant use of the microscope. To ameliorate these circumstances, the scrub nurse proactively employs a set of mechanisms to obtain and maintain situation awareness. Lastly, we present the implications of future technologies for operating rooms aiming to improve collaborative microsurgical work and patient safety.

Real-time mental stress monitoring from surgeons and surgical staff in operating rooms may reduce surgical injuries, improve performance and quality of medical care, and accelerate implementation of stress-management strategies. Motivated by the increase in usage of objective and subjective metrics for cognitive monitoring and by the gap in reviews of experimental design setups and data analytics, a systematic review of 71 studies on mental stress and workload measurement in surgical settings, published in 2001-2020, is presented. Almost 61% of selected papers used both objective and subjective measures, followed by 25% that only administered subjective tools - mostly consisting of validated instruments and customized surveys. An overall increase in the total number of publications on intraoperative stress assessment was observed from mid-2010 s along with a momentum in the use of both subjective and real-time objective measures. Cardiac activity, including heart-rate variability metrics, stress hormones, and eye-tracking metrics were the most frequently and electroencephalography (EEG) was the least frequently used objective measures. Around 40% of selected papers collected at least two objective measures, 41% used wearable devices, 23% performed synchronization and annotation, and 76% conducted baseline or multi-point data acquisition. Furthermore, 93% used a variety of statistical techniques, 14% applied regression models, and only one study released a public, anonymized dataset. This review of data modalities, experimental setups, and analysis techniques for intraoperative stress monitoring highlights the initiatives of surgical data science and motivates research on computational techniques for mental and surgical skills assessment and cognition-guided surgery.

In this work-in-progress, we present the design and implementation of a novel higher education medical computing continuous program that addresses on-demand, relevant clinical and health-care industry needs. A consortium of computing science and engineering academics and researchers, entrepreneurship and industry representatives, and clinicians is jointly designing an ecosystem that channels relevant needs from medical, clinical, and industrial environments into requirements for medical computing academic training. The design of this interdisciplinary program adheres to participatory design principles. Here, we outline the motivation and demands of such program, the design of the program, the principles and practices planned and implemented, and we report on first implementation steps. Since the graduate programs of computing and engineering mostly provide broader education, the graduates from these programs lack relevant clinical perspectives, necessary to work in- and advance the domain of medical computing. Likewise, the graduates from clinical and medical programs are often missing up-to-date computing skills, essential to comprehend and manage the fast-developing industry of medical computing technology. By implementing the processes of funnelling industrial and clinical aspects, we ensure that the graduates are equipped with relevant knowledge, and become a part of germane networks. We also report on challenges, experience, and lessons learned.

A large dataset that contains the eye movements of N=216 programmers of different experience levels captured during two code comprehension tasks is presented. Data are grouped in terms of programming expertise (from none to high) and other demographic descriptors. Data were collected through an international collaborative effort that involved eleven research teams across eight countries on four continents. The same eye tracking apparatus and software was used for the data collection. The Eye Movements in Programming (EMIP) dataset is freely available for download. The varied metadata in the EMIP dataset provides fertile ground for the analysis of gaze behavior and may be used to make novel insights about code comprehension.

Imaging is an integral part of most operating room procedures and is used on a daily basis in the operating theater. Near real-time imaging during surgical procedures can significantly enhance the procedures by providing information about the anatomy and pathological conditions. So far, customizable spectral-imaging systems were only rarely investigated, developed and even less often installed in operational settings, especially microsurgery. In this paper we describe a design of portable imaging system for high throughput spectral characterization of ex vivo samples. The setup presented in this work was used for non-invasive collection of spectral signatures from a set of biological tissues.

Numerous visualization tools are involved in surgeon's decision making during a procedure. Image-guided navigation systems such as computed tomography and magnetic resonance have become an integral part of many surgical procedures. In tumor removal microsurgeries, the distinction between a tumor tissue and surrounding normal tissues are often negligible, highly impacted by color contrast and illumination of operative field. To enhance surgical decision making, we investigate the use of real-time spectral imaging in operating room. Coupling a spectral camera with a surgical microscope, however, is challenging due to numerous standards and ergonomic requirements of operating room. In this paper, we fulfill these ergonomic considerations and describe the process of integration of spectral camera in the clinical environment.

Eye-hand coordination is a central skill in microsurgery. To develop efficacious microsurgical training environments to support development of eye-hand coordination, it is important to understand the workload associated with visuo-motor tasks in microsurgery. We embedded an eye-tracker to a surgical microscope and collected eye-blink data of 10 participants during a microsuture training task. Blink-rate was shown to drop to low levels compared to a resting-state rate and be sensitive to the phases of microsurgical suture. We discuss these findings in the light of operator training in microsurgical environments.

In video-based training, clinicians practice and advance their skills on surgeries performed by their colleagues and themselves. Although microsurgeries are recorded daily, training centers are lacking the workforce to manually annotate the segments important for practitioners, such as instrument presence and position. In this work, we propose intelligent instrument detection using Convolutional Neural Network (CNN) to augment microsurgical training. The network was trained on real microsurgical practice videos for which human annotators manually gathered a large corpus of instrument positions. Under challenging conditions of highly magnified and often blurred view, the CNN was capable to correctly detect a needle-holder (a dominant tool in suturing practice) with 78.3% accuracy (F-score = 0.84) with recognition speed above 15 FPS. The result is promising in the emerging domain of augmented medical training where instrument recognition presents benefits to the microsurgical training.

Pupillary responses have been for long linked to cognitive workload in numerous tasks. In this work, we investigate the role of pupil dilations in the context of microsurgical training, handling of microinstruments and the suturing act in particular. With an eye-tracker embedded on the surgical microscope oculars, eleven medical participants repeated 12 sutures of artificial skin under high magnification. A detailed analysis of pupillary dilations in suture segments revealed that pupillary responses indeed varied not only according to the main suture segments but also in relation to participants' expertise.

When modeling natural conversational behavior of an agent, a head direction becomes an intuitive proxy to visual attention. We examine this assumption and carefully investigate the relationship between head directions and gaze dynamics through the use of eye-movement tracking. In a group conversation settings, we analyze relationships of the two nonverbal social signals - head directions and gaze dynamics - linked to influential and non-influential statements. We develop a clustering method to estimate the number of gaze targets. We employ this method to show that head and gaze dynamic behaviors are not correlated, and thus head cannot be used as a direct proxy to a person's gaze in the context of conversations. We also describe in detail how influential statements affect head and gaze behaviors. The findings have implications on methodology, modeling and design of natural conversational agents and present a supportive evidence for employing gaze-tracking into the future conversational technologies.

Eye gaze has been used broadly in interactive intelligent systems. The research area has grown in recent years to cover emerging topics that go beyond the traditional focus on interaction between a single user and an interactive system. This special issue presents five articles that explore new directions of gaze-based interactive intelligent systems, ranging from communication robots in dyadic and multiparty conversations to a driving simulator that uses eye gaze evidence to critique learners’ behavior.

Code reading is an important skill in programming. Inspired by the linearity that people exhibit while natural language text reading, we designed local and global gaze-based measures to characterize linearity (left-to-right and top-to-bottom) in reading source code. Unlike natural language text, source code is executable and requires a specific reading approach. To validate these measures, we compared the eye movements of novice and expert programmers who were asked to read and comprehend short snippets of natural language text and Java programs. Our results show that novices read source code less linearly than natural language text. Moreover, experts read code less linearly than novices. These findings indicate that there are specific differences between reading natural language and source code, and suggest that non-linear reading skills increase with expertise. We discuss the implications for practitioners and educators.

Pupil size is known to correlate with the changes of cognitive task workloads, but how the pupil responds to requirements of basic goal-directed motor tasks involved in human-machine interactions is not yet clear. This work conducted a user study to investigate the pupil dilations during aiming in a tele-operation setting, with the purpose of better understanding how the changes in task requirements are reflected by the changes of pupil size. The task requirements, managed by Fitts' index of difficulty (ID), i.e. the size and distance apart of the targets, were varied between tasks, and pupil responses to different task IDs were recorded. The results showed that pupil diameter can be employed as an indicator of task requirements in goal-directed movements-higher task difficulty evoked higher valley to peak pupil dilation, and the peak pupil dilation occurred after a longer delay. These findings contribute to the foundation for developing methods to objectively evaluate interactive task requirements using pupil parameters during goal-directed movements in HCI.

While knowledge about reading behavior in natural-language text is abundant, little is known about the visual attention distribution when reading source code of computer programs. Yet, this knowledge is important for teaching programming skills as well as designing IDEs and programming languages. We conducted a study in which 15 programmers with various expertise read short source codes and recorded their eye movements. In order to study attention distribution on code elements, we introduced the following procedure: First we (pre)-processed the eye movement data using log-transformation. Taking into account the word lengths, we then analyzed the time spent on different lexical elements. It shows that most attention is oriented towards understanding of identifiers, operators, keywords and literals, relatively little reading time is spent on separators. We further inspected the attention on keywords and provide a description of the gaze on these primary building blocks for any formal language. The analysis indicates that approaches from research on natural-language text reading can be applied to source code as well, however not without review.

Heatmap is one of the most popular visualizations of gaze behavior, however, increasingly voluminous streams of eye-tracking data make processing of such visualization computationally demanding. Because of high requirements on a single processing machine, real-time visualizations from multiple users are unfeasible if rendered locally. We designed a framework that collects data from multiple eye-trackers regardless of their physical location, analyses these streams, and renders heatmaps in real-time. We propose a cloud computing architecture (EyeCloud) consisting of master and slave nodes on a cloud cluster, and a web interface for fast computation and effective aggregation of the large volumes of eye-tracking data. In experimental studies of the feasibility and effectiveness, we built a cloud cluster on a well-known service, implemented the architecture and reported on a comparison between the proposed system and traditional local processing. The results showed efficiency of the EyeCloud when recordings vary in durations. To our knowledge, this is the first solution to implement cloud computing for gaze visualization.

Pupil size is known to correlate with changes of cognitive task workloads, but the pupillary response to requirements of basic goal-directed motor tasks is not yet clear, although pointing with tools is a ubiquitous human task. This work describes a user study to investigate the pupil dilations during aiming in two tele-operation tasks with different target settings, one aiming at targets with different sizes located at constant distance apart, and the other aiming at targets varying in different distances. The task requirements in each task were defined by Fitts' index of difficulty (ID). The purpose of this work is to further explore how the changes in task requirements are reflected by the changes of pupil size, i.e., whether the pupil responds to either target size or target distance, or to both of them. Pupil responses to different task IDs were recorded in each task. The results showed that the pupil responds to the changes of ID, not just to the change of target size. This implies that pupil diameter can be employed as an indicator of task requirement in goal-directed movements, because higher task difficulty evoked higher peak pupil dilation which occurred with longer delay. These findings can be used for detailed understanding of eye-hand coordination mechanisms in interactive systems and contribute to the foundation for developing methods to objectively evaluate interactive task requirements using pupil parameters during goal-directed movements.

Novices were trained to perform a unimanual peg transport task in a laparoscopic training box with an illuminated interior displayed on a monitor. Subjects were divided into two groups; one group was verbally instructed to direct their gaze at distant targets, while the other group had their gaze behaviour implicitly manipulated using distant target illumination. Both groups achieved similar task completion times post-training and developed peripheral vision strategies leading to delayed foveation on targets until the instrument was closer to its destination, although the ability to focus on targets earlier during manual movements as done by an expert surgeon was quickly regained by the verbal instruction group post-training. This suggests that care should be taken when employing visual attention cuing methods such as target highlighting for training eye-hand coordination skills, as simple verbal instruction may be sufficient to help trainees to adopt more expert-like gaze behaviours.

The methodology of eye tracking has been gradually making its way into various fields of science, assisted by the diminishing cost of the associated technology. In an international collaboration to open up the prospect of eye movement research for programming educators, we present a case study on program comprehension and preliminary analyses together with some useful tools.

We analyze the Computing Education Research (CER) literature to discover what theories, conceptual models and frameworks recent CER builds on. This gives rise to a broad understanding of the theoretical basis of CER that is useful for researchers working in that area, and has the potential to help CER develop its own identity as an independent field of study.

This paper presents a summary of the seventh workshop on Eye Gaze in Intelligent Human Machine Interaction. The Gaze-in 2014 workshop is a part of a series of workshops held around the topics related to gaze and multimodal interaction. The workshop web-site can be found at http://hhhuang.homelinux.com/gaze_in/.

This paper presents a summary of the sixth workshop in Eye Gaze in Intelligent Human Machine Interaction. The GazeIn'13 workshop is a part of a series of workshops held around the topics related to gaze and multimodal interaction.

When interacting with a problem on a computer screen, users need to select from a range of strategies that the user interface supports. Previous research shows that the problem-solving user interface plays a significant role in the strategy selection, such as by making recall unnecessary or by facilitating planning. What is not very well understood is how the interaction devices, themselves, affect problem-solving strategies. We conducted an experiment in which users interacted with a problem using either a computer mouse or gaze-augmented input. We analysed the qualitative differences in the problem-solving process by investigating the content of verbal protocols. We present a new evidence that shows that the gaze-based interaction is characterised by an increased cognitive processing of thecurrently attended information, leading to better plan development and improved problem-solving strategies. The findings have implications on the design of future gaze-aware problem-solving user interfaces.

Interaction intent prediction and the Midas touch have been a longstanding challenge for eye-tracking researchers and users of gaze-based interaction. Inspired by machine learning approaches in biometric person authentication, we developed and tested an offline framework for task-independent prediction of interaction intents. We describe the principles of the method, the features extracted, normalization methods, and evaluation metrics. We systematically evaluated the proposed approach on an example dataset of gaze-augmented problem-solving sessions. We present results of three normalization methods, different feature sets and fusion of multiple feature types. Our results show that accuracy of up to 76% can be achieved with Area Under Curve around 80%. We discuss the possibility of applying the results for an online system capable of interaction intent prediction.

Differences between visual attention strategies of experts and novices have been investigated in many fields, but little has been done in the field of microneurosurgery. In the hands of an experienced surgeon, microneurosurgery seems like an elegant, routine and clean procedure with minimal blood loss. However, microneurosurgery is a multifaceted task with clinical risks associated to surgeons' skills. In a preliminary study, eye movements of eight surgeons were recorded while observing four images representing four phases in a tumor removal surgery. A comparison of the eye movement strategies shows clear markers of expertise depending on the phase of the surgery.

This paper discusses estimation of active speaker in multi-party video-mediated communication from gaze data of one of the participants. In the explored settings, we predict voice activity of participants in one room based on gaze recordings of a single participant in another room. The two rooms were connected by high definition, low delay audio and video links and the participants engaged in different activities ranging from casual discussion to simple problem-solving games. We treat the task as a classification problem. We evaluate several types of features and parameter settings in the context of Support Vector Machine classification framework. The results show that using the proposed approach vocal activity of a speaker can be correctly predicted in 89 % of the time for which the gaze data are available.

We investigate the role of gaze in communication and collaboration between humans in computerised environments. In this paper we describe a software solution to a multidirectional gaze display in shared-display collaborative tasks and we report on initial results of an experiment in which the system was applied to allow gaze display in a remote pair-programming lecture. Remotely, an expert was explaining two algorithms to a novice who then attempted to apply the algorithm on an unseen problem. We report on an analysis of data from the conditions when there was a gaze transmitted and displayed to the peer with the no-gaze condition. The remote gaze display reliably affected the variability of novices visual attention patterns and they were better able to focus on the information presented. Without the experts gaze display their strategies contained greater variety.

We propose a person authentication system using eye movement signals. In security scenarios, eye-tracking has earlier been used for gaze-based password entry. A few authors have also used physical features of eye movement signals for authentication in a task-dependent scenario with matched training and test samples. We propose and implement a task-independent scenario whereby the training and test samples can be arbitrary. We use short-term eye gaze direction to construct feature vectors which are modeled using Gaussian mixtures. The results suggest that there are personspecific features in the eye movements that can be modeled in a task-independent manner. The range of possible applications extends beyond the security-type of authentication to proactive and user-convenience systems.

This paper presents a preliminary analysis of research papers in computing education. While previous analysis has explored what research is being done in computing education, this project explores how that research is being done. We present our classification system, then the results of applying it to the papers from all five years of ICER. We find that this subset of computing education research has more in common with research in information systems than with that in computer science or software engineering; and that the papers published at ICER generally appear to conform to the specified ICER requirements.

Our research was conducted within a larger HIV and AIDS counseling project in the southern part of Tanzania. We spent six months creating a pilot version of an interactive digital platform that would allow learners to share their experiences with the disease. The platform, called Sura ya UKIMWI (The Face of AIDS), was designed by a team consisting of secondary school children, university counseling students, HIV counseling experts and experts in ICT. We took the case study approach and report on observations and experiences with the participatory design (PD) process and its outcomes, in this specific context. The results of the project indicate that the technology created under the PD paradigm helps students to easily compose their own stories thereby supporting their understanding and reflection on how these stories relate to their own life experiences. Interacting with the stories was entertaining to the students and they could identify the lessons to be learned. The approach can be an alternative to past attempts that mostly imported technology for HIV and AIDS education. Our experience can be used as a guide in establishing similar digital projects in other parts of the HIV and AIDS affected world.

As collaborative learning in general, and pair programming in particular, has become widely adopted in computer science education, so has the use of pedagogical visualization tools for facilitating collaboration. However, there is little theory on collaborative learning with visualization, and few studies on their effect on each other. We build on the concept of the engagement taxonomy and extend it to classify finer variations in the engagement that result from the use of a visualization tool. We analyze the applicability of the taxonomy to the description of the differences in the collaboration process when visualization is used. Our hypothesis is that increasing the level of engagement between learners and the visualization tool results in a higher positive impact of the visualization on the collaboration process. This article describes an empirical investigation designed to test the hypothesis. The results provide support for our extended engagement taxonomy and hypothesis by showing that the collaborative activities of the students and the engagement levels are correlated.

The challenges in empirical eye-tracking studies of usability or complex problem solving include 1) how to effectively analyze the eye-tracking data, and 2) how to interpret and relate the resulting measures to the user cognitive processing. We conducted a reanalysis of eye-tracking data from a recent study that involved programmers of two experience groups debugging a program with the help of multiple representations. The proportional fixation time on each area of interest (AOI), frequency of visual attention switches between the areas, and the type of switch were investigated during five consequential phases of ten minutes of debugging. We increased the granularity of the focus on the user processing several times, allowing us to construct a better picture of the process. In addition, plotting the areas of interest in time supported a visual analysis and comparison with the quantitative data.

The previous research of visual attention has mostly considered the situations in which a single person performs a task. The current eye-tracking devices and software support this research situation. Applications of eye-tracking in the research of collaborative tasks have been rare to date. We present a methodological framework of a research in which visual attention of pair programmers with a single display has been studied. We discuss the challenges of such research when conducted in real-world settings and the requirements on the eye-tracking setups. The hardware setups and software solutions to the problems of acquisition and synchronization of streams of eye-tracking data are presented. We outline the methodological questions of future visual attention research of collaborative tasks.

Previous research on program visualization tools has suggested that both static and dynamic aspects of the program shall be shown to novice students. We attempted to expand the state-of-art environment, BlueJ, by a dynamic low-level visualization of program execution, with an extension based on a visualization tool, Jeliot 3. We discuss the technical issues and how the combination can be used in teaching and learning of programming.

Program comprehension processes have previously been studied using methodologies such as think-aloud or comprehension summary analysis. Eye-tracking, however, has not been previously widely applied to studies of behavioral aspects of programming. We present a study in which program comprehension was investigated with a help of a remote eye-tracker. Novice and intermediate programmers used a program visualization tool to aid their comprehension while the location of fixations, fixation durations and attention switching between the areas of interest were recorded.In this paper 1) we propose an approach how to investigate trends in repeated-measures sparse-data of few cases captured by an eye-tracker and 2) using this technique, we characterize the development of program comprehension strategies during dynamic program visualization with help of eye-movement data.

Five reviewers conducted a methodological review of the full papers in the proceedings of ICALT 2004. They analyzed the papers in terms of research design, common defects, section proportions, measures used, and region of the first author's affiliation. It was found that the majority of articles were methodological/theoretical articles or program descriptions that did not report research on human participants. Of the articles that involved human participants - posttest-only designs without controls were the most common, many articles lacked appropriate interpretations and needed controls, the majority of space was devoted to program description, and student surveys were the most common type of measurement. Recommendations for ICALT authors and reviewers are given.

We describe an evaluation of a program animation tool, Jeliot 3. Eye movement data, interaction data, and audio and video protocols were recorded for sixteen participants comprehending three Java programs. In this paper, we report and analyze a subset of the captured data, the patterns of visual attention in time and space during animation of a program. We discuss the possible potentials and prospects of using eye-gaze as an interaction and evaluation modality for building intelligent, gaze-aware program animation tools.

Usability evaluation is a crucial step in the iterative software development life-cycle, leading into successful usage of systems. However, most of the usability evaluation methods and tools do not consider all aspects of educational environments. These lacks materialize as unusable results that lead to failures in educational software. Often these results are not comparable and credible, or easily accessible to educators selecting the proper systems for use in their settings; the technology, usability, pedagogy (TUP) evaluation model equally concentrates on the technological, usability, and instructional aspects of educational environments, leading to complete evaluations. In this paper we present the TUP model with the use scenario included and our current WWW prototype of evaluation system based upon it.