In-the-wild eye tracking datasets to train gaze vector estimation methods are noisy and unreliable, due both to limited image quality and to weak guarantees on label accuracy. At the same time, simply increasing dataset size does not guarantee better results, especially when diversity, balance and label reliability are not taken into account. In this work, we adopt a data-centric perspective and introduce an innovative data-pruning pipeline for robust gaze vector estimation. Our methodology leverages vision-language model embeddings and their zero-shot classification capabilities, together with an early-learning prediction-error analysis, to assign task-aware quality scores to individual samples. Moreover, we incorporate a region-weighted loss conditioned on gaze direction to better handle the uneven difficulty across the gaze space and to improve learning in under-represented regions. Finally, we validate our approach on a large relabeled subset of the GazeCapture dataset by training a hybrid gaze-estimation model, and show that a curated subset of 84K samples yields an accuracy improvement over a model trained on the full 2.45M-sample dataset (https://dx.doi.org/10.21227/c9ap-zm59doi:10.21227/c9ap-zm59).

Abstract In clinical renal assessment, image registration plays a pivotal role, as patient movement during data acquisition can significantly impede image post-processing and the accurate estimation of hemodynamic parameters. This study introduces a deep learning-based image registration framework specifically for arterial spin labeling (ASL) imaging. ASL is a magnetic resonance imaging technique that modifies the longitudinal magnetization of blood perfusing the kidney using a series of radiofrequency pulses combined with slice-selective gradients. After tagging the arterial blood, label images are captured following a delay, allowing the tagged blood bolus to enter the renal tissue, while control images are acquired without tagging the arterial spins. Given that perfusion maps are generated at the pixel level by subtracting control images from label images and considering the relatively small signal intensity difference, precise alignment of these images is crucial to minimize motion artefacts and prevent significant errors in perfusion calculations. Moreover, due to the extended ASL acquisition times and the anatomical location of the kidneys, renal images are often susceptible to pulsation, peristalsis, and breathing motion. These motion-induced noises and other instabilities can adversely affect ASL imaging outcomes, making image registration essential. However, research on renal MRI registration, particularly with respect to learning-based techniques, remains limited, with even less focus on renal ASL. Our study proposes a learning-based image registration approach that builds upon VoxelMorph and introduces groupwise inference as a key enhancement. The dataset includes 2448 images of transplanted kidneys (TK) and 2456 images of healthy kidneys (HK). We compared the automatic image registration results with the widely recognized optimization method Elastix. The model’s performance was evaluated using the mean structural similarity index (MSSIM), normalized correlation coefficient (NCC), temporal signal-to-noise ratio (TSNR) of the samples, and the mean cortical signal (CSIM) in perfusion-weighted images, thereby extending the evaluation beyond traditional similarity-based metrics. Our method achieved superior image registration performance, with peak NCC (0.987 ± 0.006) and MSSIM (0.869 ± 0.048) values in the kidney region, significantly surpassing Elastix and the unregistered series (p < 0.05) on TK and HK datasets. Regularization analysis showed that higher λ values (1, 2) produced smoother deformation fields, while moderate λ values (0.5, 0.9) balanced smoothness and detail, maintaining low non-positive Jacobian percentages (<1%) comparable to Elastix. Additionally, our method improved CSIM by 14.3% (2.304 ± 1.167) and TSNR by 13.1% (3.888 ± 2.170) in TK, and achieved up to 13.2% (CSIM) and 29.8% (TSNR) enhancements in HK, demonstrating robustness and improved signal quality across datasets and acquisition techniques.

Calibrating gaze estimation models is crucial to maximize the effectiveness of these systems, although its implementation also poses challenges related to usability. Therefore, the simplification of this process is key. In this work, we dissect the impact of calibration due to both the environment and the user in gaze estimation models that employ general-purpose devices. We aim to replicate a workflow close to the final application by starting with pre-trained models and subsequently calibrating them using different strategies, testing under various camera arrangements and user-specific variability. The results indicate differentiation between the impact due to the user and the setup, being the components due to the users a slightly more pronounced impact than those related to the setup, opening the door to understanding calibration as a composite process. In any case, the development of calibration-free remote gaze estimation solutions remains a great challenge, given the crucial role of calibration.

This paper aims to advance the fields of unsupervised data labeling and incremental cross-domain training techniques. We apply these innovative methods to develop a model tailored for the Augmentative and Alternative Communication (AAC) application domain, introducing a new perspective in hybrid eye Gaze Estimation (GE). These hybrid eye GE models combine the generalization strengths of appearance-based models with the scene understanding capabilities inherent in geometrical reconstruction. The use of open eye tracking datasets for the AAC domain introduces domain shift, while accurately labeling gaze vectors is challenging without specialized hardware for proper 3D dimensional reconstruction. We propose an approach to solve this challenges by conducting standardized unsupervised gaze vector labeling across multiple open GE datasets and subsequently performing incremental training to adapt to the target domain. Using a proprietary dataset we were able to reduce the gaze error from 4.87º to 3.95º, compared to a traditional single-step training.

In this study we propose a competent low-cost eye tracking solution that is able to run on any mobile device, independently of the hardware that is equipped with. The rapid evolution of technologies has enabled to work with many neural network structures that some years ago were out of reach. The project will start from a solution which Irisbond (https://www.irisbond.com/) company has been working on, which gives precision values of 3 and 6 degrees for calibration and calibration-free use cases respectively. The goal of the solution is to try to develop a usable solution in the Augmented and Alternative Communication (AAC) field across different types of devices, from mobile to embedded devices. To achieve such an objective, two main goals have been set out during this study. One the one hand I (we) aim at removing the initial calibration step to reach a calibration-free solution. On the other hand, I (we) seek to separate the functionality of a software into independent, interchangeable modules to fit the different target device limitations.

Eye tracking technology in low resolution scenarios is not a completely solved issue to date. The possibility of using eye tracking in a mobile gadget is a challenging objective that would permit to spread this technology to non-explored fields. In this paper, a knowledge based approach is presented to solve gaze estimation in low resolution settings. The understanding of the high resolution paradigm permits to propose alternative models to solve gaze estimation. In this manner, three models are presented: a geometrical model, an interpolation model and a compound model, as solutions for gaze estimation for remote low resolution systems. Since this work considers head position essential to improve gaze accuracy, a method for head pose estimation is also proposed. The methods are validated in an optimal framework, I2Head database, which combines head and gaze data. The experimental validation of the models demonstrates their sensitivity to image processing inaccuracies, critical in the case of the geometrical model. Static and extreme movement scenarios are analyzed showing the higher robustness of compound and geometrical models in the presence of user's displacement. Accuracy values of about 3° have been obtained, increasing to values close to 5° in extreme displacement settings, results fully comparable with the state-of-the-art.

We study attention processes to brand, price and visual information about products in online retailing websites, simultaneously considering the effects of consumers' goals, purchase category and consumers' statements. We use an intra-subject experimental design, simulated web stores and a combination of observational eye-tracking data and declarative measures.

Availability of large scale tagged datasets is a must in the field of deep learning applied to the eye tracking challenge. In this paper, the potential of Supervised-Descent-Method (SDM) as a semiautomatic labelling tool for eye tracking images is shown. The objective of the paper is to evidence how the human effort needed for manually labelling large eye tracking datasets can be radically reduced by the use of cascaded regressors. Different applications are provided in the fields of high and low resolution systems. An iris/pupil center labelling is shown as example for low resolution images while a pupil contour points detection is demonstrated in high resolution. In both cases manual annotation requirements are drastically reduced.

Head pose estimation (HPE) is currently a growing research field, mainly because of the proliferation of human–computer interfaces (HCI) in the last decade. It offers a wide variety of applications, including human behavior analysis, driver assistance systems or gaze estimation systems. This article aims to contribute to the development of robust and accurate HPE methods based on 2D tracking of the face, enhancing performance of both 2D point tracking and 3D pose estimation. We start with a baseline method for pose estimation based on POSIT algorithm. A novel weighted variant of POSIT is then proposed, together with a methodology to estimate weights for the 2D–3D point correspondences. Further, outlier detection and correction methods are also proposed in order to enhance both point tracking and pose estimation. With the aim of achieving a wider impact, the problem is addressed using a global approach: all the methods proposed are generalizable to any kind of object for which an approximate 3D model is available. These methods have been evaluated for the specific task of HPE using two different head pose video databases; a recently published one that reflects the expected performance of the system in current technological conditions, and an older one that allows an extensive comparison with state-of-the-art HPE methods. Results show that the proposed enhancements improve the accuracy of both 2D facial point tracking and 3D HPE, with respect to the implemented baseline method, by over 15% in normal tracking conditions and over 30% in noisy tracking conditions. Moreover, the proposed HPE system outperforms the state of the art on the two databases.

The precise detection of pupil/iris center is key to estimate gaze accurately. This fact becomes specially challenging in low cost frameworks in which the algorithms employed for high performance systems fail. In the last years an outstanding effort has been made in order to apply training-based methods to low resolution images. In this paper, Supervised Descent Method (SDM) is applied to GI4E database. The 2D landmarks employed for training are the corners of the eyes and the pupil centers. In order to validate the algorithm proposed, a cross validation procedure is performed. The strategy employed for the training allows us to affirm that our method can potentially outperform the state of the art algorithms applied to the same dataset in terms of 2D accuracy. The promising results encourage to carry on in the study of training-based methods for eye tracking.

In head-mounted eye tracking systems, the correct detection of pupil position is a key factor in estimating gaze direction. However, this is a challenging issue when the videos are recorded in real-world conditions, due to the many sources of noise and artifacts that exist in these scenarios, such as rapid changes in illumination, reflections, occlusions and an elliptical appearance of the pupil. Thus, it is an indispensable prerequisite that a pupil detection algorithm is robust in these challenging conditions. In this work, we present one pupil center detection method based on searching the maximum contribution point to the radial symmetry of the image. Additionally, two different center refinement steps were incorporated with the aim of adapting the algorithm to images with highly elliptical pupil appearances. The performance of the proposed algorithm is evaluated using a dataset consisting of 225,569 head-mounted annotated eye images from publicly available sources. The results are compared with the better algorithm found in the bibliography, with our algorithm being shown as superior.

One of the goals of the eye tracking community is to build systems that allow users to move freely. In general, there is a trade-off between the field of view of an eye tracking system and the gaze estimation accuracy. We aim to study how much the field of view of an eye tracking system can be increased, while maintaining acceptable accuracy. In this paper, we investigate all the issues concerning remote eye tracking systems with large range of movement in a simulated environment and we give some guidelines that can facilitate the process of designing an eye tracker. Given a desired range of movement and a working distance, we can calculate the camera focal length and sensor size or given a certain camera, we can determine the user's range of movement. The robustness against large head movement of two gaze estimation methods based on infrared light is analyzed: an interpolation and a geometrical method. We relate the accuracy of the gaze estimation methods with the image resolution around the eye area for a certain feature detector's accuracy and provide possible combinations of pixel size and focal length for different gaze estimation accuracies. Finally, we give the gaze estimation accuracy as a function of a new defined eye error, which is independent of any design parameters.

A multistage procedure to detect eye features is presented. Multiresolution and topographic classification are used to detect the iris center. The eye corner is calculated combining valley detection and eyelid curve extraction. The algorithm is tested in the BioID database and in a proprietary database containing more than 1200 images. The results show that the suggested algorithm is robust and accurate. Regarding the iris center our method obtains the best average behavior for the BioID database compared to other available algorithms. Additional contributions are that our algorithm functions in real time and does not require complex post processing stages.

The development of systems that track the eye while allowing head movement is one of the most challenging objectives of gaze tracking researchers. Tracker accuracy decreases as the subject moves from the calibration position and is especially influenced by changes in depth with respect to the screen. In this paper, we demonstrate that the pattern of error produced due to user movement mainly depends on the system configuration and hardware element placement rather than the user. Thus, we suggest alternative calibration techniques for error reduction that compensate for the lack of accuracy due to subject movement. Using these techniques, we can achieve an error reduction of more than 50%.

Low cost eye tracking is an actual challenging research topic for the eye tracking community. Gaze tracking based on a web cam and without infrared light is a searched goal to broaden the applications of eye tracking systems. Web cam based eye tracking results in new challenges to solve such as a wider field of view and a lower image quality. In addition, no infrared light implies that glints cannot be used anymore as a tracking feature. In this paper, a thorough study has been carried out to evaluate pupil (iris) center-eye corner (PC-EC) vector as feature for gaze estimation based on interpolation methods in low cost eye tracking, as it is considered to be partially equivalent to the pupil center-corneal reflection (PC-CR) vector. The analysis is carried out both based on simulated and real data. The experiments show that eye corner positions in the image move slightly when the user is looking at different points of the screen, even with a static head position. This lowers the possible accuracy of the gaze estimation, significantly reducing the accuracy of the system under standard working conditions to 2--3 degrees.

When performing eye detection in a driving scenario, new challenges arise that do not occur in a standard indoor eye tracking session. Rapid subject movement, non-controlled fast light variation and partial or total occlusions are the main problems that must be overcome. Furthermore, sunlight's infrared component makes it difficult the use of active artificial infrared light sources. In this paper, we describe a novel algorithm that combines Viola Jones face detector and TLD (Tracking Learning Detection) algorithm. In a standard driving scenario, it achieves a 84% rate of detection. Furthermore, we have designed a filtering stage that allows a low false positive rate. The algorithms hardware requirement is a standard web cam, and it can potentially work in real time.

Being able to perform eye tracking with low cost technology is the key to broaden its applications and one of the major goals for the eye tracking community nowadays. Furthermore, new datasets to evaluate the different methods are needed to reproduce the real conditions in which these algorithms work. In this paper, we present a dataset containing images of subjects with different gaze orientations as a new evaluation tool. First step in eye tracking algorithms is to detect the region of the eyes, and using the Gi4e dataset, we evaluate the best performing public Haar based classifiers under different gaze orientations to detect the eye area, proving this dataset to be a fair evaluation method.

Adaptive-weight algorithms currently represent the state of the art in local stereo matching. However, due to their computational requirements, these types of solutions are not suitable for real-time implementation. Here, we present a novel aggregation method inspired by the anisotropic diffusion technique used in image filtering. The proposed aggregation algorithm produces results similar to adaptive-weight solutions while reducing the computational requirements. Moreover, near real-time performance is demonstrated with a GPU implementation of the algorithm.

The accurate segmentation of subcortical brain structures in magnetic resonance (MR) images is of crucial importance in the interdisciplinary field of medical imaging. Although statistical approaches such as active shape models (ASMs) have proven to be particularly useful in the modeling of multiobject shapes, they are inefficient when facing challenging problems. Based on the wavelet transform, the fully generic multiresolution framework presented in this paper allows us to decompose the interobject relationships into different levels of detail. The aim of this hierarchical decomposition is twofold: to efficiently characterize the relationships between objects and their particular localities. Experiments performed on an eight-object structure defined in axial cross sectional MR brain images show that the new hierarchical segmentation significantly improves the accuracy of the segmentation, and while it exhibits a remarkable robustness with respect to the size of the training set.

Gaze-tracking data have been used successfully in the design of new input devices and as an observational technique in usability studies. Polynomial-based Video-Oculography (VOG) systems are one of the most attractive gaze estimation methods thanks to their simplicity and ease of implementation. Although the functionality of these systems is generally acceptable, there has been no thorough comparative study to date of how the mapping equations affect the final system response. After developing a taxonomic classification of calibration functions, we examined over 400,000 models and evaluated the validity of several conventional assumptions. Our rigorous experimental procedure enabled us to optimize the calibration process for a real VOG gaze-tracking system and halve the calibration time while avoiding a detrimental effect on the accuracy or tolerance to head movement. Finally, a geometry-based method is implemented and tested. The results and performance is compared with those obtained by the general purpose expressions.

Active Shape Models are some of the most actively researched model-based segmentation approaches. An accurate estimation of the shape probability distribution is essential to provide the prior knowledge that makes ASMs able to handle the large inherent variability of anatomical structures, differentiating between allowed and invalid instances. Under the typical assumption of normality the subspace of allowed shapes (SAS) is confined within a hyperellipsoid. Although the approximation of the SAS by a hypercube provides computational advantages, this simplification allows the occurrence of highly improbable instances. In addition, a high dependency on the rest of the configuration parameters is observed when the general segmentation algorithm incorporates the hypercube simplification. In this work, a new, efficient hyperelliptical approximation of the SAS based on the Newton-Raphson optimisation method is presented. To perform a detailed comparative study of the effect that four different SAS estimation approaches have on the general segmentation process, a generalisation of the typical two-factor factorial design is used on two different image databases. The results obtained by means of this statistical technique not only reveal the superiority of the new hyperelliptical method in terms of both accuracy and robustness but also provide information of great interest for optimising the segmentation process.

Recent local stereo matching algorithms based on an adaptive-weight strategy achieve accuracy similar to global approaches. One of the major problems of these algorithms is that they are computationally expensive and this complexity increases proportionally to the window size. This paper proposes a novel cost aggregation step with complexity independent of the window size (i.e. O(1)) that outperforms state-of-the-art O(1) methods. Moreover, compared to other O(1) approaches, our method does not rely on integral histograms enabling aggregation using colour images instead of grayscale ones. Finally, to improve the results of the proposed algorithm a disparity refinement pipeline is also proposed. The overall algorithm produces results comparable to those of state-of-the-art stereo matching algorithms.

Homography normalization is presented as a novel gaze estimation method for uncalibrated setups. The method applies when head movements are present but without any requirements to camera calibration or geometric calibration. The method is geometrically and empirically demonstrated to be robust to head pose changes and despite being less constrained than cross-ratio methods, it consistently performs favorably by several degrees on both simulated data and data from physical setups. The physical setups include the use of off-the-shelf web cameras with infrared light (night vision) and standard cameras with and without infrared light. The benefits of homography normalization and uncalibrated setups in general are also demonstrated through obtaining gaze estimates (in the visible spectrum) using only the screen reflections on the cornea.

Of gaze tracking techniques, video-oculography (VOG) is one of the most attractive because of its versatility and simplicity. VOG systems based on general purpose mapping methods use simple polynomial expressions to estimate a user's point of regard. Although the behaviour of such systems is generally acceptable, a detailed study of the calibration process is needed to facilitate progress in improving accuracy and tolerance to user head movement. To date, there has been no thorough comparative study of how mapping equations affect final system response. After developing a taxonomic classification of calibration functions, we examine over 400,000 models and evaluate the validity of several conventional assumptions. The rigorous experimental procedure employed enabled us to optimize the calibration process for a real VOG gaze tracking system and, thereby, halve the calibration time without detrimental effect on accuracy or tolerance to head movement.

The design of robust and high-performance gaze-tracking systems is one of the most important objectives of the eye-tracking community. In general, a subject calibration procedure is needed to learn system parameters and be able to estimate the gaze direction accurately. In this paper, we attempt to determine if subject calibration can be eliminated. A geometric analysis of a gaze-tracking system is conducted to determine user calibration requirements. The eye model used considers the offset between optical and visual axes, the refraction of the cornea, and Donder's law. This paper demonstrates the minimal number of cameras, light sources, and user calibration points needed to solve for gaze estimation. The underlying geometric model is based on glint positions and pupil ellipse in the image, and the minimal hardware needed for this model is one camera and multiple light-emitting diodes. This paper proves that subject calibration is compulsory for correct gaze estimation and proposes a model based on a single point for subject calibration. The experiments carried out show that, although two glints and one calibration point are sufficient to perform gaze estimation (error approximately 1 degree), using more light sources and calibration points can result in lower average errors.

In the past years, research in eye tracking development and applications has attracted much attention and the possibility of interacting with a computer employing just gaze information is becoming more and more feasible. Efforts in eye tracking cover a broad spectrum of fields, system mathematical modeling being an important aspect in this research. Expressions relating to several elements and variables of the gaze tracker would lead to establish geometric relations and to find out symmetrical behaviors of the human eye when looking at a screen. To this end a deep knowledge of projective geometry as well as eye physiology and kinematics are basic. This paper presents a model for a bright-pupil technique tracker fully based on realistic parameters describing the system elements. The system so modeled is superior to that obtained with generic expressions based on linear or quadratic expressions. Moreover, model symmetry knowledge leads to more effective and simpler calibration strategies, resulting in just two calibration points needed to fit the optical axis and only three points to adjust the visual axis. Reducing considerably the time spent by other systems employing more calibration points renders a more attractive model.

Pupil detection represents one of the most critical aspects for eye tracking systems based on video oculography. A robust segmentation of the aforementioned feature determines to a large extent the degree of performance of the system. However, a question remains unsolved... why does the pupil gray level change in the image? Apart from the possible room lighting variation, can the eyeball physiology influence its final level in the image by itself? The answer is yes. In this paper a further step in the work by Nguyen et al. [Nguyen et al. 2002] is proposed in which this eyeball characteristic was noticed but not explained. This paper gives some enlightenment to this effect finding a physiological reason for it. From the results it is clear that the pupil brightness can be a valid image feature and can contribute together with alternative ones to improve the tracking [Hammoud 2005]. A deep knowledge about its behavior is undoubtedly highly interesting. The matter should be to study how the retina reacts to the light in order to know how it can influence its final level in the image. The retina is not a uniform surface; in the fovea there is a higher density of cones and the ganglion cells are highly packed. When the eye is entered with a beam of light, the light can be reflected and absorbed at the various layers of the retina. Normally near infrared lighting is used because it is not visible for humans. Actually in this range of wavelength the reflected light is dominated by the light scattered back from the choroid: the last layer before the sclera that supports the retina, and it is precisely for this wavelength for which the retina presents the highest reflectance. A bright pupil tracking is conducted following the same method as in the works by Nguyen et al. [Nguyen et al. 2002] and Miller [Miller et al. 1995] but more exhaustive experiments are conducted. A ray of light directed to the fovea needs to cross a thicker layer in order to reach the choroid which produces a decrease of effective light intensity, a stronger reflection and consequently a brighter pupil can be expected if the most eccentric part of the retina is reached. Vertical rotations of the eyeball about its center are sketched in figure 1. From the figure it is clear that the pupil will appear brighter if the subject is looking at the upper part of the screen than for cases in which points in the lower part are fixated. Regarding to left and right eye rotations the fovea is horizontally and temporally displaced from the eyeball back pole. That means that the visual axis and the fovea present an angular offset with respect to the symmetry axis of the eye but with opposite sign depending on the eye. Following the same reasoning as the one used for vertical rotations it is clear that a brighter pupil could be expected for points on the right part of the screen for the left eye. A symmetrical behavior appears for the right eye being the points on the left part of the screen the ones with higher pupil levels.

Calibration is one of the most tedious and often annoying aspects of many eye tracking systems. It normally consists in looking at several marks on a screen in order to collect enough data to modify the parameters of an adjustable model. Unfortunately this step is unavoidable if a competent tracking system is desired. Many efforts have been made to achieve more competent and improved eye tracking systems. Maybe the search for an accurate mathematical model is one of the least researched fields. The lack of a parametric description of the gaze estimation problem makes it difficult to find the most suitable model, and therefore generic expressions in calibration and tracking sessions are employed instead. In other words, a model based on parameters describing the elements involved in the tracking system would provide a stronger basis and robustness. The aim of this work is to build up a mathematical model totally based in realistic variables describing elements taking part in an eye tracking system employing the well known bright pupil technique i.e. user, camera, illumination and screen. The model is said to be defined when the expression relating the point the user is looking at with the extracted features of the image (glint position and center of the pupil) is found. The desired model would have to be simple, realistic, accurate and easy to calibrate.