This special issue of the Proceedings of the ACM on Human-Computer Interaction includes accepted full papers from the ACM Symposium on Eye Tracking Research and Applications (ETRA). ETRA is the premier eye-tracking conference that brings together researchers from across disciplines to present advances in eye-tracking systems and methods, oculomotor research, eye movement data analysis, gaze-based interaction, and eye-tracking applications. A total of 24 full papers were accepted from 80 submissions after a rigorous reviewing process (30% acceptance rate). Accepted contributions were split into special issues in two journals, depending on the fit of topic and authors' preferences. 16 accepted papers are included in this issue of the Proceedings of the ACM on Human-Computer Interaction. 8 will be published in the Proceedings of the ACM on Computer Graphics and Interactive Techniques. All accepted papers are invited to present at ETRA 2025 (May 26 - May 29, 2025, in Tokyo). We would like to thank all members of the Editorial Board and all external reviewers for their effort and dedication, as well as all authors for their high-quality contributions.

In an attempt to improve student learning process, schools are looking for new solutions to enhance the educational scenario. A particular direction is the adoption of educational games, mainly for their playful aspects, quick feedback, and structure with increasing levels of challenges. Within this di-rection, one relevant question is how to improve the learning of fractions, which is an essential concept to any STEM career. Targeting these issues, boost the fractions learning process, iFractions was born - a collection of Web games aimed at teaching fractions to elementary school children. It is implemented in JavaScript and is distributed as a free software. Considering that in computer-based games, giving formative feedback goes beyond checking answers to explaining and modifying a student's thinking, but can also promote learning, we propose a redesign for one of the iFractions mini-games. This redesign consisted of revising, restructuring, and renewing feedback on the balloon mini-game. The results of the redesign evaluation showed that experts involved considered that iFractions interfaces were improved and brought ideas about future modifications to the iFractions platform.

Motion correlation interfaces are those that present targets moving in different patterns, which the user can select by matching their motion. In this paper, we re-formulate the task of target selection as a probabilistic inference problem. We demonstrate that previous interaction techniques can be modelled using a Bayesian approach and that how modelling the selection task as transmission of information can help us make explicit the assumptions behind similarity measures. We propose ways of incorporating uncertainty into the decision-making process and demonstrate how the concept of entropy can illuminate the measurement of the quality of a design. We apply these techniques in a case study and suggest guidelines for future work.

Despite recent developments in eye tracking technology, mobile eye trackers (ET) are still expensive devices limited to a few hundred samples per second. High speed ETs (closer to 1 KHz) can provide improved flexibility for data filtering and more reliable event detection. To address these challenges, we present the Stroboscopic Catadioptric Eye Tracking (SCET) system, a novel approach for mobile ET based on rolling shutter cameras and stroboscopic structured infrared lighting. SCET proposes a geometric model where the cornea acts as a spherical mirror in a catadioptric system, changing the projection as it moves. Calibration methods for the geometry of the system and for the gaze estimation are presented. Instead of tracking common eye features, such as the pupil center, we track multiple glints on the cornea. By carefully adjusting the camera exposure and the lighting period, we show how one image frame can be divided into several bands to increase the temporal resolution of the gaze estimates. We assess the model in a simulated environment and also describe a prototype implementation that demonstrates the feasibility of SCET, which we envision as a step further in the direction of a mobile, robust, affordable, and high-speed eye tracker.

Recently, interaction techniques in which the user selects screen targets by matching their movement with the input device have been gaining popularity, particularly in the context of gaze interaction (e.g. Pursuits, Orbits, AmbiGaze, etc.). However, though many algorithms for enabling such interaction techniques have been proposed, we still lack an understanding of how they compare to each other. In this paper, we introduce two new algorithms for matching eye movements: Profile Matching and 2D Correlation, and present a systematic comparison of these algorithms with two other state-of-the-art algorithms: the Basic Correlation algorithm used in Pursuits and the Rotated Correlation algorithm used in PathSync. We also examine the effects of two thresholding techniques and post-hoc filtering. We evaluated the algorithms on a user dataset and found the 2D Correlation with one-level thresholding and post-hoc filtering to be the best performing algorithm.

Text entry using gaze-based interaction is a vital communication tool for people with motor impairments. Most solutions require the user to fixate on a key for a given dwell time to select it, thus limiting the typing speed. In this paper we introduce EyeSwipe, a dwell-time-free gaze-typing method. With EyeSwipe, the user gaze-types the first and last characters of a word using the novel selection mechanism "reverse crossing." To gaze-type the characters in the middle of the word, the user only needs to glance at the vicinity of the respective keys. We compared the performance of EyeSwipe with that of a dwell-time-based virtual keyboard. EyeSwipe afforded statistically significantly higher typing rates and more comfortable interaction in experiments with ten participants who reached 11.7 words per minute (wpm) after 30 min typing with EyeSwipe.

Eye-typing is an important tool for people with physical disabilities and, for some, it is their main form of communication. By observing expert typists using physical keyboards, we notice that visual throughput is considerably reduced in current eye-typing solutions. We propose AugKey to improve throughput by augmenting keys with a prefix, to allow continuous text inspection, and suffixes to speed up typing with word prediction. AugKey limits the visual information to the foveal region to minimize eye movements (i.e., reduce eye work). We have applied AugKey to a dwell-time keyboard and compared its performance with two conditions with no augmented feedback: a keyboard with and one without word prediction. Results show that AugKey can be about 28% faster than no word prediction and 20% faster than traditional word prediction, with a smaller workload index.

Video-based eye trackers (VETs) have become the dominant eye tracking technology due to its reasonable cost, accuracy, and easy of use. VETs require real-time image processing to detect and track eye features such as the center of the pupil and corneal reflection to estimate the point of regard. Despite the continuous evolution of cameras and computers that made head mounted eye trackers easier to use in natural activities, real-time processing of high resolution images in mobile devices remains a challenge. In this paper we investigate the feasibility of a novel eye-tracking technique intended for wearable applications that use mice chips as imaging sensors. Such devices are widely available at very low cost, and provide high speed and accurate 2D tracking data. Though mice chips have been used for many purposes other than a computer's pointing device, to our knowledge this is the first attempt to use it as an eye tracker. To validate the technique, we built an episcleral database with about 100 high resolution episcleral patches from 7 individuals. The episclera is the outer most layer of the sclera, which is the white part of the eye, and consists of dense vascular connective tissue. We have used the patches to determine if the episclera contains enough texture to be reliably tracked. We also present results from a prototype built using an off-the-shelf mouse sensor. Our results show that a mouse-based eye tracker has the potential to be very accurate, precise, and fast (measuring 2.1' of visual angle at 1 KHz speed), with little overhead for the wearable computer.

Remote gaze estimation using a single non-calibrated camera, simple user calibration or calibration free, and robust to head movements are very desirable features of eye tracking systems. Because cross-ratio (CR) is an invariant property of projective geometry, gaze estimation methods that rely on this property have the potential to provide these features, though most current implementations rely on a few simplifications that compromise the performance of the method. In this paper, the CR method for gaze tracking is revisited, and we introduce a new method that explicitly compensates head movements using a simple 3 parameter eye model. The method uses a single non-calibrated camera and requires a simple calibration procedure per user to estimate the eye parameters. We have conducted simulations and experiments with real users that show significant improvements over current state-of-the-art CR methods that do not explicitly compensate for head motion.

This paper introduces Dynamic Context Switching (DCS) as an extension of the Context Switching (CS) paradigm for gaze-based interaction. CS replicates information in each context. The user can freely explore one context without worrying about the Midas touch problem, and a saccade to the other context triggers the selection of the item under focus. Because CS has to display two contexts simultaneously, the amount of useful screen space is limited. DCS dynamically adjusts the context sizes, where the context that has the focus is displayed in full size, while the other is minimized, thus improving useful screen space. A saccade to the minimized context triggers selection, and properly readjusts the sizes of the contexts. Results from a pilot user experiment show that DCS improves user performance and do not cause disorientation due to the dynamic context resizing.

Traditional cross-ratio methods (TCR) project a light pattern and use invariant properties of projective geometry to estimate the gaze position. Advantages of the TCR methods include robustness to large head movements and in general requires just a one time per user calibration. However, the accuracy of TCR methods decay significantly for head movements along the camera optical axis, mainly due to the angular difference between the optical and visual axis of the eye. In this paper we propose a depth compensation cross-ratio (DCR) method that improves the accuracy of TCR methods for large head depth variations. Our solution compensates the angular offset using a 2D onscreen vector computed from a simple calibration procedure. The length of the 2D vector, which varies with head distance, is adjusted by a scale factor that is estimated from relative size variations of the corneal reflection pattern. The proposed DCR solution was compared to a TCR method using synthetic and real data from 2 users. An average improvement of 40% was observed with synthetic data, and 8% with the real data.

This paper presents context switching as an alternative to selection by dwell time. The technique trades screen space for comfort and speed. By replicating the interface on two separate regions called contexts, the user can comfortably explore the whole content of a context without the effects of the Midas touch problem. Focus within a context is set by a short dwell time and fast selection is done by switching contexts. We present experimental results for a text entry application with 7 participants that show significant speed improvement over traditional fixed dwell time gaze controlled keyboards. After 8 sessions, 6 participants were able to type about 12 words per minute (wpm), and the fastest participant was able to type above 20 wpm with error rate under 2%.

This paper introduces the Gaze based Interface Extensions (GInX) architecture designed for the development of eye-gaze enhanced attentive interfaces. The architecture is composed of 3 modules, the domain, user, and attentive modules. In the absence of information about the user and the domain, the attentive module controls the cursor using gaze and target position information alone. The cursor control can be refined in an attentive way [Vertegaal 2002] as more information about the application and the user are added. The system currently offers 3 different operation modes: Latency, MAGIC, and GInX default mode. In the Latency mode, the cursor position is controlled by gaze and selection is done using dwell time. MAGIC Pointing [Zhai et al. 1999] was suggested to combine the speed of eye tracking with the accuracy of manual pointing devices. GInX extends the concept of Magic Pointing by introducing information about the user and application context in order to eliminate the time required for cursor reacquisition and position adjustment inherent in the original MAGIC Pointing interface. A prototype of GInX was implemented and used to compared the performance of all these 3 modes with a mouse. Our experiments show that GInX outperforms MAGIC Pointing, although the mouse has the best performance overall.

The focus of visual attention is closely related to eye movements and fixations, while episodic memory has been defined as the ability to be consciously aware of an earlier experience [Bond, 2005]. The role of emotional content plays a crucial role in the ability of recalling a previous event. Here we investigate eye fixation patterns and their spatial distribution using ordinary kriging, a geostatistical interpolation method [Canto-Pereira et al., 2005], in a task where emotion and episodic memory were assessed.

In previous experiments investigating motor control in a Time-To-Contact task [Morya et al., 2003], events occurring 400-600 ms prior to contact (but not earlier or later) caused volunteers to anticipate their estimate of when contact occurred. Many such mislocalization or mistiming effects have been discussed in the literature [Nijhuan, 1994; van Beers et al. 2001]. In preliminary eye-tracking experiments [Morya et al. 2004], with a simplified version of the task, involuntary shifts in gaze suggested the presence of attentional shifts as volunteers prepared to respond, that might be associated with their anticipations. To better understand the factors involved in these observations, gaze was sistematically recorded changing the speed of the moving target, and with different instructions as to where the volunteers should look as they performed the Time-To-Contact task.

This paper introduces JEcoSys, an extensible framework developed in Java to generate interactive simulations of plants. JEcoSys uses a simple physical model of plants, based on dynamic constraints, and pre-computations to achieve realism and efficiency during animation. The framework also includes a force field based technique to model the interaction of plants with different agents, that can simulate natural phenomena such as wind and rain. Experimental results show that JEcosys is able to simulate complex scenes containing hundreds of tufts of grass and flowers in real-time.

This paper presents a multiple face detector based on a robust pupil detection technique. The pupil detector uses active illumination that exploits the retro-reflectivity property of eyes to facilitate detection. The detection range of this method is appropriate for interactive desktop and kiosk applications. Once the location of the pupil candidates are computed, the candidates are filtered and grouped into pairs that correspond to faces using heuristic rules. To demonstrate the robustness of the face detection technique, a dual-mode face tracker was developed, which is initialized with the most salient detected face. Recursive estimators are used to guarantee the stability of the process and combine the measurements from the multi-face detector and a feature correlation tracker. The estimated position of the face is used to control a pan-tilt servo mechanism in real-time, that moves the camera to keep the tracked face always centered in the image.

This work explores a new direction in utilizing eye gaze for computer input. Gaze tracking has long been considered as an alternative or potentially superior pointing method for computer input. We believe that many fundamental limitations exist with traditional gaze pointing. In particular, it is unnatural to overload a perceptual channel such as vision with a motor control task. We therefore propose an alternative approach, dubbed MAGIC (Manual And Gaze Input Cascaded) pointing. With such an approach, pointing appears to the user to be a manual task, used for fine manipulation and selection. However, a large portion of the cursor movement is eliminated by warping the cursor to the eye gaze area, which encompasses the target. Two specific MAGIC pointing techniques, one conservative and one liberal, were designed, analyzed, and implemented with an eye tracker we developed. They were then tested in a pilot study. This early- stage exploration showed that the MAGIC pointing techniques might offer many advantages, including reduced physical effort and fatigue as compared to traditional manual pointing, greater accuracy and naturalness than traditional gaze pointing, and possibly faster speed than manual pointing. The pros and cons of the two techniques are discussed in light of both performance data and subjective reports.

Several techniques for electronic image stabilization have been proposed, but very little research has been done to compare and evaluate such techniques. We propose a set of measures to evaluate image stabilization algorithms based of their fidelity, displacement range, and performance. These measures do not require calibration or ground truth, making the evaluation procedure very simple and flexible, i.e., it provides the means to compare techniques based on different motion models. We have used this procedure to compare several image stabilization algorithms and also evaluate the sensitivity of these algorithms to some of its parameters. These same procedures could also be used for the comparison and evaluation of motion estimation and image registration techniques.

The authors present a fast electronic image stabilization system that compensates for 3D rotation. The extended Kalman filter framework is employed to estimate the rotation between frames, which is represented using unit quaternions. A small set of automatically selected and tracked feature points are used as measurements. The effectiveness of this technique is also demonstrated by constructing mosaic images from the motion estimates, and comparing them to mosaics built from 2D stabilization algorithms. Two different stabilization schemes are presented. The first, implemented in a real-time platform based on a Datacube MV200 board, estimates the motion between two consecutive frames and is able to process gray level images of resolution 128/spl times/120 at 10 Hz. The second scheme estimates the motion between the current frame and an inverse mosaic; this allows better estimation without the need for indexing the new image frames. Experimental results for both schemes using real and synthetic image sequences are presented.

We propose a novel video coding scheme to improve the performance of established block-based motion compensation codecs such as MPEG, H261, and H263. The proposed method is a hybrid scheme which introduces model-based global motion compensation as a pre-processing step to the basic block-based motion compensation technique. Performance evaluation tests show that the new method is capable of achieving higher compression rates with the addition of a very small overhead due to global motion estimation. In this paper we describe a codec based on the hybrid motion compensation technique in the context of H263, and present experimental results, comparing the performance of different codecs using rate-distortion curves for several test sequences.

We study the performance associated with model-based video coding schemes using global motion models for motion compensation. Reference frames and compensated frame differences are coded using a method similar to MPEG, employing transform coding, quantization, and entropy coding. The traditional block matching motion compensation approach is compared to global motion compensation approaches derived from 3D motion stabilization methods, using similarity and projective transformations. 3D model-based motion compensation is achieved by derotating the input sequence. This leads to a projective transformation, which under certain circumstances, is well approximated by a similarity transformation. Experiments are carried out to analyze the performance of each scheme for comparable coding rates.

We present a fast implementation of an electronic digital image stabilization system that is able to handle large image displacements. The system has been implemented in a parallel pipeline image processing hardware (Datacube Max Video 200) connected to a SUN SPARCstation 20/612. Our technique is based on a 2D feature-based multi-resolution motion estimation algorithm, that tracks a small set of features to estimate the motion of the camera. The combination of the estimates from a reference frame is used to warp the current frame in order to achieve stabilization. Experimental results using video sequences taken from a camera mounted on a moving vehicle demonstrate the robustness of the system when processing 15 frames per second.

This paper explores the use of hidden Markov models (HMMs) for the recognition of head gestures. A gesture corresponds to a particular pattern of head movement. The facial plane is tracked using a parameterized model and the temporal sequence of three image rotation parameters are used to describe four gestures. A dynamic vector quantization scheme was implemented to transform the parameters into suitable input data for the HMMs. Each model was trained by the iterative Baum-Welch procedure using 28 sequences taken from 5 persons. Experimental results from a different data set (33 new sequences from 6 other persons) demonstrate the effectiveness of this approach.

We improve the performance of conventional motion compensated discrete cosine transform video coding. For motion compensation, we employ a two step algorithm in which the camera motion is compensated first and then the motion of moving objects is estimated. We use a feature matching algorithm for camera motion compensation. Motion compensated frame differences are divided into three regions called stationary background, moving objects, and newly emerging area. A region adaptive subband image coding scheme is used for spatial coding of these regions.