This paper presents a novel Contrastive Self-Attention Convolutional Neural Network (CSA-CNN) model with enhanced Difficulty Aware (DA) loss function to improve the segmentation of pupils in eye images. The incorporation of transformer-style self-attention and Difficulty-Aware loss in a UNET-style architecture allows for robust feature representation and promotes shape alignment. The novel model was trained on two public databases (LPW and RIT-Eyes) and evaluated on two other public datasets (ExCuSe and ElSe). When compared with seven state-of-the-art pupil center detection methods, the CSA-CNN showed improvement of over 6% in pupil center detection accuracy (detection within 5 pixels of the labeled center) and more than 9% in Intersection Over Union (IOU) accuracy, compared to the best performer among the other seven methods. Furthermore, when the CSA-CNN model was integrated into a glint-based eye tracking system that uses learning based methods to detect pupil-center, we saw a 25% improvement in gaze accuracy.

Eye tracking systems that estimate the point-of-gaze are essential in extended reality (XR) systems as they enable new interaction paradigms and technological improvements. It is important for these systems to maintain accuracy when the headset moves relative to the head (known as device slippage) due to head movements or user adjustment. One of the most accurate eye tracking techniques, which is also insensitive to shifts of the system relative to the head, uses two or more infrared (IR) light emitting diodes to illuminate the eye and an IR camera to capture images of the eye. An essential step in estimating the point-of-gaze in these systems is the precise determination of the location of two or more corneal reflections (virtual images of the IR-LEDs that illuminate the eye) in images of the eye. Eye trackers tend to have multiple light sources to ensure at least one pair of reflections for each gaze position. The use of multiple light sources introduces a difficult problem: the need to match the corneal reflections with the corresponding light source over the range of expected eye movements. Corneal reflection detection and matching often fail in XR systems due to the proximity of camera and steep illumination angles of light sources with respect to the eye. The failures are caused by corneal reflections having varying shape and intensity levels or disappearance due to rotation of the eye, or the presence of spurious reflections. We have developed a fully convolutional neural network, based on the UNET architecture, that solves the detection and matching problem in the presence of spurious and missing reflections. Eye images of 25 people were collected in a virtual reality headset using a binocular eye tracking module consisting of five infrared light sources per eye. A set of 4,000 eye images were manually labelled for each of the corneal reflections, and data augmentation was used to generate a dataset of 40,000 images. The network is able to correctly identify and match 91% of corneal reflections present in the test set. This is comparable to a state-of-the-art deep learning system, but our approach requires 33 times less memory and executes 10 times faster. The proposed algorithm, when used in an eye tracker in a VR system, achieved an average mean absolute gaze error of 1°. This is a significant improvement over the state-of-the-art learning-based XR eye tracking systems that have reported gaze errors of 2-3°.

The growing availability of eye-gaze tracking technology has allowed for its employment in a wide variety of applications, one of which is the objective diagnosis and monitoring of neuropsychiatric disorders from features of attentional bias extracted from visual scanning patterns. Current techniques in this field are largely comprised of non-generalizable methodologies that rely on domain expertise and study-specific assumptions. In this paper, we present a general, data-driven, end-to-end framework that extracts relevant features of attentional bias from visual scanning behaviour and uses these features to classify between subject groups with standard machine learning techniques. The general framework uses visual scanning data from free-viewing tasks. In these tasks, subjects look at sets of slides with several thematic images while their visual scanning patterns (sets of ordered fixations) are monitored by an eye-tracking system. Subjects' fixations are encoded into relative visual attention maps (RVAMs), and two data-driven methods are proposed to segment regions of interests (ROIs) from RVAMs: 1) using group average RVAMs, and 2) using differences of group average RVAMs. Relative fixation times within the segmented ROIs are then used as input features for a vanilla multilayered perceptron to classify between patient groups. The methods were evaluated on data from two studies: an anorexia nervosa (AN)/healthy controls study (AN study) with 37 subjects, and a bipolar disorder (BD)/major depressive disorder (MDD) study (BD-MDD study) with 73 subjects. Using leave-one-subject-out cross validation, the general methods achieved an area under the receiver operating curve (AUROC) score of 0.935 for the AN study and 0.888 for the BD-MDD study, the latter of which exceeds the performance of the state-of-the-art analysis model designed specifically for the BD-MDD study, which had an AUROC of 0.879. The results validate the proposed framework's efficacy as a generalizable, standard baseline for analyzing visual scanning data.

Bipolar Disorder (BD) and Major Depressive Disorder (MDD) are two common and debilitating mood disorders. Misdiagnosing BD as MDD is relatively common and the introduction of markers to improve diagnostic accuracy early in the course of the illness has been identified as one of the top unmet needs in the field. In this paper, we present novel methods to differentiate between BD and MDD patients. The methods use deep learning techniques to quantify differences between visual scanning patterns of BD and MDD patients. In the methods, visual scanning patterns that are described by ordered sequences of fixations on emotional faces are encoded into a lower dimensional space and are fed into a long-short term memory recurrent neural network (RNN). Fixation sequences are encoded by three different methods: 1) using semantic regions of interests (RoIs) that are manually defined by experts, 2) using semi-automatically defined grids of RoIs, or 3) using a convolutional neural network (CNN) to automatically extract visual features from saliency maps. Using data from 47 patients with MDD and 26 patients with BD we showed that using semantic RoIs, the RNN improved the performance of a baseline classifier from an AUC of 0.603 to an AUC of 0.878. Similarly using grid RoIs, the RNN improved the performance of a baseline classifier from an AUC of 0.450 to an AUC of 0.828. The classifier that automatically extracted visual features from saliency maps (a long recurrent convolutional network that is fully data-driven) had an AUC of 0.879. The results of the study suggest that by using RNNs to learn differences between fixation sequences the diagnosis of individual patients with BD or MDD can be disambiguated with high accuracy. Moreover, by using saliency maps and CNN to encode the fixation sequences the method can be fully automated and achieve high accuracy without relying on user expertise and/or manual labelling. When compared with other markers, the performance of the class of classifiers that was introduced in this paper is better than that of detectors that use differences in neural structures, neural activity or cortical hemodynamics to differentiate between BD and MDD patients. The novel use of RNNs to quantify differences between fixation sequences of patients with mood disorders can be easily generalized to studies of other neuropsychological disorders and to other fields such as psychology and advertising.

Assessment of apathy in patients with Alzheimer's disease (AD) relies heavily on interviews with caregivers and patients, which can be ambiguous and time consuming. More precise and objective methods of evaluation can better inform treatment decisions. In this study, visual scanning behaviours (VSBs) on emotional and non-emotional stimuli were used to detect apathy in patients with AD. Forty-eight AD patients participated in the study. Sixteen of the patients were apathetic. Patients looked at 48 slides with non-emotional images and 32 slides with emotional images. We described two methods that use recurrent neural networks (RNNs) to learn differences between the VSBs of apathetic and non-apathetic AD patients. Method 1 uses two separate RNNs to learn group differences between visual scanning sequences on emotional and non-emotional stimuli. The outputs of the RNNs are then combined and used by a logistic regression classifier to characterise patients as either apathetic or non-apathetic. Method 1 achieved an AUC gain of 0.074 compared to a previously presented handcrafted feature method of detecting emotional blunting (AUC handcrafted = 0.646). Method 2 assumes that each individual's "style of scanning" (stereotypical eye movements) is independent of the content of the visual stimuli and uses the "style of scanning" to normalise the individual's VSBs on emotional and non-emotional stimuli. Method 2 uses RNNs in a sequence-to-sequence configuration to learn the individual's "style of scanning". The trained model is then used to create vector representations that contain information on the individual's "style of scanning" (content independent) and her/his VSBs (content dependent) on emotional and non-emotional stimuli. The distance between these vector representations is used by a logistic regression classifier to characterise patients as either apathetic or non-apathetic. Using Method 2 the AUC of the classifier improved to 0.814. The results presented suggest that using RNNs to analyse differences between VSBs on emotional and non-emotional stimuli (a measure of emotional blunting) can improve objective detection of apathy in individual patients with AD.

Post-concussion syndrome (PCS) is associated with incomplete recovery following a mild traumatic brain injury (mTBI). Currently, there is no biomarker to diagnose post-concussion syndrome. Although microstructural damage to white matter tracts is postulated and reported in some studies, there is no diagnostic biomarker available. In this paper, we present preliminary results of a novel and simple-to-administer eye-tracking paradigm that relates visual scanning behaviour to microstructural integrity of the corpus callosum (CC). The novel paradigm is based on the Matching Familiar Figures Test (MFFT). In the MFFT test, a series of slides each containing a familiar standard figure and six variants are presented. Only one of the variants is identical to the standard figure and subjects are instructed to select the matched variant. The normalised number of transitions to the standard figure prior to the first selection (NNTP) is used as an indirect measure of the microstructural integrity of the CC. Microstructural integrity was evaluated by measuring the fractional anisotropy (FA) of the CC derived from tract based spatial statistics on diffusion tensor imaging (DTI). Fifty-five patients with PCS completed the visual scanning paradigm and magnetic resonance imaging. Significant inverse correlation was found between the NNTP and the FA of the CC (r= -0.463, p<;0.001). Mediation analysis showed that damage to the CC could affect NNTP directly or through pathways that affect attention, somatic and cognitive symptoms. The results suggest that visual scanning behaviour during the MFFT is a multifactorial cognitive and behavioural process that can provide a surrogate marker of the structural integrity of the corpus callosum.

An indoor tracking and navigation system based on measurements of received signal strength (RSS) in wireless local area network (WLAN) is proposed. In the system, the location determination problem is solved by first applying a proximity constraint to limit the distance between a coarse estimate of the current position and a previous estimate. Then, a Compressive Sensing-based (CS--based) positioning scheme, proposed in our previous work , , is applied to obtain a refined position estimate. The refined estimate is used with a map-adaptive Kalman filter, which assumes a linear motion between intersections on a map that describes the user's path, to obtain a more robust position estimate. Experimental results with the system that is implemented on a PDA with limited resources (HP iPAQ hx2750 PDA) show that the proposed tracking system outperforms the widely used traditional positioning and tracking systems. Meanwhile, the tracking system leads to 12.6 percent reduction in the mean position error compared to the CS-based stationary positioning system when three APs are used. A navigation module that is integrated with the tracking system provides users with instructions to guide them to predefined destinations. Thirty visually impaired subjects from the Canadian National Institute for the Blind (CNIB) were invited to further evaluate the performance of the navigation system. Testing results suggest that the proposed system can be used to guide visually impaired subjects to their desired destinations.

This paper presents a probabilistic approach for the estimation of the angle between the optical and visual axes (angle kappa) in infants. The approach assumes that when patterned calibration targets are presented on a uniform background, subjects are more likely to look at the calibration targets than at the uniform background, but it does not require accurate and continuous fixation on presented targets. Simulations results show that when subjects attend to roughly half of the presented targets, angle kappa can be estimated accurately with low probability (< 1%) of false detection. In experiments with five babies who attended to the calibration target for only 47% of the time (range from 26% to 70%), the average difference between repeated measurements of angle kappa was 0.04 ± 0.31°.

Stereo-camera Remote Eye-Gaze Tracking (REGT) systems can provide calibration-free estimation of gaze. However, such systems have a limited tracking range due to the requirement for the eye to be tracked in both cameras. This paper presents a general framework for extension of a tracking range of stereo-camera user-calibration-free REGT systems. The proposed method consists of two distinct phases. In the brief initial phase, estimates of eye-features [the center of the pupil and corneal reflections] in pairs of stereo-images are used to estimate automatically a set of subject-specific eye parameters. In the second phase, these subject-specific eye parameters are used with estimates of eye-features in images from any one of the systems' cameras to compute the Point-of-Gaze (PoG). Experiments were conducted with a system that includes two cameras in a horizontal plane. The experimental results demonstrate that the tracking range for horizontal gaze directions can be extended by more than 50%: from ±23.2° when the two cameras are used as a stereo pair to ±35.5° when the two cameras are used independently to estimate the PoG. By adding more cameras to the system, the proposed framework allows further extension of the tracking range in both horizontal and vertical direction, while preserving a user-calibration-free status of a REGT system.

This paper examines the use of Listing's and Donders' laws for the calculation of the torsion of the eye in the estimation of the point-of-gaze. After describing Listing's and Donders' laws and providing their analytical representation, experimental results obtained while subjects looked at a computer screen are presented. The experimental results show that when the point-of-gaze was estimated using Listing's and Donders' laws there was no significant accuracy improvement relative to when eye torsion was ignored. While for a larger range of eye rotation the torsion would be more significant and should be taken into account, the torsion predicted by Listing's and Donders' laws may be inaccurate, even in ideal conditions. Moreover, eye torsion resulting from lateral head tilt can be significantly larger than the torsion predicted by Listing's and Donders' laws, and even have opposite direction. To properly account for eye torsion, it should be measured independently (e.g., by tracking the iris pattern and/or the scleral blood vessels).

Gaze estimation systems use calibration procedures that require active subject participation to estimate the point-of-gaze accurately. In these procedures, subjects are required to fixate on a specific point or points in space at specific time instances. This paper describes a gaze estimation system that does not use calibration procedures that require active user participation. The system estimates the optical axes of both eyes using images from a stereo pair of video cameras without a personal calibration procedure. To estimate the point-of-gaze, which lies along the visual axis, the angles between the optical and visual axes are estimated by a novel automatic procedure that minimizes the distance between the intersections of the visual axes of the left and right eyes with the surface of a display while subjects look naturally at the display (e.g., watching a video clip). Experiments with four subjects demonstrate that the RMS error of this point-of-gaze estimation system is 1.3°.

This paper describes a method for remote, non-contact point-of-gaze estimation that tolerates free head movements and requires a simple calibration procedure in which the subject has to fixate only on a single point. This method uses the centers of the pupil and at least two corneal reflections that are estimated from eye images captured by at least two cameras. Experimental results obtained with three adult subjects exhibited RMS point-of-gaze estimation errors ranging from 7 to 12 mm (equivalent to about 0.6 -- 1° of visual angle) for head movements in a volume of about 1 dm3. Preliminary results with two infants demonstrated the ability of a system that requires a single-point calibration procedure to estimate infants' point-of-gaze. The ability to record infants' visual scanning behavior can be used for the study of visual development, the determination of attention allocation and the assessment of visual function in preverbal infants.

The cross-ratios method for point-of-gaze estimation uses the invariance property of cross-ratios in projective transformations. The inherent causes of the subject-dependent point-of-gaze estimation bias exhibited by this method have not been well characterized in the literature. Using a model of the eye and the components of a system (camera, light sources) that estimates point-of-gaze, a theoretical framework for the cross-ratios method is developed. The analysis of the cross-ratios method within this framework shows that the subject-dependent estimation bias is caused mainly by (i) the angular deviation of the visual axis from the optic axis and (ii) the fact that the virtual image of the pupil center is not coplanar with the virtual images of the light sources that illuminate the eye (corneal reflections). The theoretical framework provides a closed-form analytical expression that predicts the estimation bias as a function of subject-specific eye parameters.

The use of head movements in control applications leaves the hands free for other tasks and utilizes the mobility of the head to acquire and track targets over a wide field of view. We present the results of applying a Kalman filter to generate prediction estimates for tracking head positions. A simple kinematics approach based on the assumption of a piecewise constant acceleration process is suggested and is shown to track head positions with an rms error under 2 degrees for head movements with accelerations smaller than 3000 degrees /s. To account for the wide range of head dynamic characteristics, an adaptive approach with input estimation is developed. The performance of the Kalman filter is compared to that based on a simple polynomial predictor.

The structure and performance of the maximum likelihood (ML) estimator of slowly varying time delays when the received signals are contaminated by additive noise containing a strongly directional component, due to the presence of a fixed, localized source of interference in the acoustic environment is investigated. The ML estimator is obtained by maximizing the output of the log-likelihood processor. The latter is shown to consist of a slowly varying noise canceler followed by a minimum mean square error estimator of the source signal and a correlator. Closed-form expressions are obtained for the Cramer-Rao lower bound (CRLB) on the error covariance matrix of time-varying delay estimators. The effects of directional interference on the CRLB are investigated numerically for a simplified configuration consisting of two sensors and linearly varying time delays.>

The problem of optimum space-time processing for multiple Gaussian source signals transmitted through a slowly-varying linear channel and monitored with a passive array of sensors in the presence of spatially correlated noise is addressed. To solve this problem, a new class of linear systems (LS), referred to as semistationary, is introduced. These LS are characterized by time-frequency representations whose variations in time occur over intervals much larger than the corresponding system correlation time. The general conditions under which semistationary LS can be used in array processing are investigated and shown to be satisfied in many applications. By modeling the slowly varying linear channel as a semistationary LS and using the factorization properties of the optimum processor, closed form expressions are obtained for the log-likelihood function of the array output and for the associated Cramer-Rao lower bound on estimator variance.>

The authors present an exact solution to the problem of maximum-likelihood time-delay estimation over arbitrary observation time T. That is, the standard assumption T>> tau /sub c/+d/sub max/ made in the derivation of the asymptotic maximum-likelihood (AML) estimator, where t/sub c/ is the correlation time of the various processes involved and d/sub max/ the maximum permissible delay, is relaxed. The exact maximum-likelihood (EML) processor is shown to consist of a special finite-time beamformer, followed by a scalar postprocessor based on the eigenvalues and eigenfunctions of a certain integral equation. The solution of this integral equation is obtained for the case of stationary signals with rational power spectral densities (PSD). The performance of EML and AML are compared by means of computer simulations for a first-order low-pass PSD. The results show that EML can lead to a significant improvement in performances (bias, variance, large errors) when the condition T>> tau /sub c/+d/sub max/ is not satisfied.>

An approach is presented for reducing the threshold observation time required to perform high resolution broadband bearing estimation. The proposed technique is based on a space-time statistic called the steered covariance matrix (STCM). In broadband settings, the STCM has an advantage over the well-known cross-spectral density matrix (CSDM) in that it can be estimated with much greater statistical stability. The STCM is used in conjunction with minimum variance spectral estimation to obtain a broadband spatial spectral estimate with a much lower threshold observation time than the CSDM-based minimum-variance distortionless response (MVDR) method.>

In this paper, the use of adaptive beamforming prior to generalized correlation (GC) is demonstrated as a means of time delay estimate (TDE) bias reduction in interference dominated environments. Analysis of GC TDE bias with both minimum variance distortionless response and conventional delay-and-sum beamforming is given. At high interference-to-noise ratios, theoretical and simulation results indicate that the adaptive structure can facilitate improved delay estimation performance compared to conventional methods.

This paper demonstrates that the LMS adaptive line enhancer (ALE) can be used to enhance estimation-performance in situations where source signal spectra have tonal components. Time delay estimation obtained by using ALE's as pre-filters followed by a cross correlator (ALE-CC) achieve better performance than the smoothed coherence transform (SCOT) when tonals are present while achieving the basic cross-correlation performance in purely broadband conditions.

This paper provides a comparison of the LMS adaptive filter versus conventional Generalized Correlation (GC) methods of time delay estimation in terms of their mean-square-error performance. The treatment is restricted to broadband stationary inputs with finite observation time without a priori knowledge of the processor input statistics. The time delay estimator probability distribution, variance, and error probability are derived for the LMS adaptive filter approach. Further, a performance index is given which is optimized with respect to choice of step size. The simulation results presented indicate that without a priori knowledge of the input statistics, both approaches yield similar sub-optimal results. On the other hand, optimal processing of the adaptive filter weights can yield an estimator with variance similar to that of the minimum variance GC method which approaches the Cramer-Rao Lower Bound.