This OSF contains the data for the paper 'Detection of Alcohol Inebriation from Eye Movements using Remote and Wearable Eye Trackers'. The raw data can be found in the folder raw_data (each zip file contains recorded data up- /downsampled to 1,000 Hz as csv-files). - Each csv file contains the recording (remote and wearable) for one subject for one PVT trial. - Each csv file contains the following columns: trial_id: trial-id for current recording block_id: block-id for current recording x_pix_eyelink: x-pixel coordinates using eyelink remote eye-tracker y_pix_eyelink: y-pixel coordinates using eyelink remote eye-tracker eyelink_timestamp: timestamp or recording in ms x_pix_pupilcore_interpolated: x-pixel coordinates using pupil-core eye-tracker upsampled to 1,000 Hz y_pix_pupilcore_interpolated: y-pixel coordinates using pupil-core eye-tracker upsampled to 1,000 Hz pupil_size_eyelink: pupil-size of pupil using eyelink remote eye-tracker target_distance: distance to eyelink remote eye-tracker (screen) in mm pupil_size_pupilcore_interpolated: pupil-size of pupil pupil-core eye-tracker upsampled to 1,000 Hz pupil_confidence_interpolated: pupil detection confidence of pupil pupil-core eye-tracker upsampled to 1,000 Hz time_to_prev_bac: elapsed time from previous BAC testing in ms time_to_next_bac: remaining time for next BAC testing in ms prev_bac: previous BAC concentration next_bac: next BAC concentration For more details see: https://github.com/aeye-lab/etra-potsdam-binge-pvt

We present EyeBench, the first benchmark designed to evaluate machine learning models that decode cognitive and linguistic information from eye movements during reading. EyeBench offers an accessible entry point to the challenging and underexplored domain of modeling eye tracking data paired with text, aiming to foster innovation at the intersection of multimodal AI and cognitive science. The benchmark provides a standardized evaluation framework for predictive models, covering a diverse set of datasets and tasks, ranging from assessment of reading comprehension to detection of developmental dyslexia. Progress on the EyeBench challenge will pave the way for both practical real-world applications, such as adaptive user interfaces and personalized education, and scientific advances in understanding human language processing. The benchmark is released as an open-source software package which includes data downloading and harmonization scripts, baselines and state-of-the-art models, as well as evaluation code, publicly available at https://github.com/EyeBench/eyebench.

This paper develops a deep neural network that accepts cell descriptors and molecules of multiple administered drugs and predicts the joint dose-response hypersurface of the combinatorial treatment. Since the dose-response hypersurface over several concentration dimensions fully characterizes the interaction dynamics of the administered drugs, the model is a computational tool that guides the discovery of synergistic treatments. The neural network is a biochemistry-informed universal approximator; it can estimate any shape of a dose-response hypersurface and has desirable invariances built into its architecture. The model excels at interpolating and extrapolating dose-response surfaces; its predictions align well with known mechanisms of action (MOA). It is the first model that can estimate joint dose-response hypersurfaces of arbitrarily many drugs, including untried combinations, in the presence of arbitrary, potentially nonlinear interactions between drugs. We release the model itself as well as a database of likely synergistic drug triplets. Our code is available at https://github.com/alonsocampana/PanThera/; the database of likely synergistic drug triplets at https://zenodo.org/records/14001717.

Eye movements offer valuable, non-invasive insights into cognitive processes and are widely used in both psycholinguistic research and machine-learning applications, such as assessing reading comprehension and cognitive load. These applications typically rely on fixations and saccades detected through gaze event algorithms, which may be either proprietary or open-source. The impact of different gaze event detection algorithms on subsequent analysis is underexplored and often overlooked. This study investigates how two threshold-based algorithms, I-DT and I-VT, influence both machine-learning classification tasks and psycholinguistic statistical modeling. Using diverse datasets-including stationary, remote, and VR eye-tracking data across multiple sampling frequencies-our findings show significant differences in downstream performance. For ML tasks, I-DT generally outperforms I-VT, with I-VT being highly sensitive to threshold choices. In psycholinguistic analysis, results confirm established findings only when thresholds align with established fixation metrics, emphasizing the importance of appropriate threshold selection for meaningful analysis. Our code is publicly available: https://github.com/aeye-lab/eye-movement-preprocessing.

Dose-response curves characterize the relationship between the concentration of drugs and their inhibitory effect on the growth of specific types of cells. The predominant Hill-equation model of an ideal enzymatic inhibition unduly simplifies the biochemical reality of many drugs; and for these drugs the widely-used drug performance indicator of the half-inhibitory concentration $IC_{50}$ can lead to poor therapeutic recommendations and poor selections of promising drug candidates. We develop a neural model that uses an embedding of the interaction between drug molecules and the tissue transcriptome to estimate the entire dose-response curve rather than a scalar aggregate. We find that, compared to the prior state of the art, this model excels at interpolating and extrapolating the inhibitory effect of untried concentrations. Unlike prevalent parametric models, it it able to accurately predict dose-response curves of drugs on previously unseen tumor tissues as well as of previously untested drug molecules on established tumor cell lines.

Eye movements can be used to analyze a viewer’s cognitive capacities or mental state. Neural networks that process the raw eye-tracking signal can outperform methods that operate on scan paths preprocessed into fixations and saccades. However, the scarcity of such data poses a major challenge. We therefore develop SP-EyeGAN, a neural network that generates synthetic raw eye-tracking data. SP-EyeGAN consists of Generative Adversarial Networks; it produces a sequence of gaze angles indistinguishable from human ocular micro- and macro-movements. We explore the use of these synthetic eye movements for pre-training neural networks using contrastive learning. We find that pre-training on synthetic data does not help for biometric identification, while results are inconclusive for the detection of ADHD and gender classification. However, for the eye movement-based assessment of higher-level cognitive skills such general reading comprehension, text comprehension, and the distinction of native from non-native readers, pre-training on synthetic eye-gaze data improves the models’ performance and even advances the state-of-the-art for reading comprehension. The SP-EyeGAN model, pre-trained on GazeBase, along with the code for developing your own raw eye-tracking machine learning model with contrastive learning, is available at https://github.com/aeye-lab/sp-eyegan.

Eye movements in reading play a crucial role in psycholinguistic research studying the cognitive mechanisms underlying human language processing.More recently, the tight coupling between eye movements and cognition has also been leveraged for language-related machine learning tasks such as the interpretability, enhancement, and pre-training of language models, as well as the inference of reader-and text-specific properties.However, scarcity of eye movement data and its unavailability at application time poses a major challenge for this line of research.Initially, this problem was tackled by resorting to cognitive models for synthesizing eye movement data.However, for the sole purpose of generating humanlike scanpaths, purely data-driven machinelearning-based methods have proven to be more suitable.Following recent advances in adapting diffusion processes to discrete data, we propose SCANDL, a novel discrete sequence-tosequence diffusion model that generates synthetic scanpaths on texts.By leveraging pretrained word representations and jointly embedding both the stimulus text and the fixation sequence, our model captures multi-modal interactions between the two inputs.We evaluate SCANDL within-and across-dataset and demonstrate that it significantly outperforms state-of-the-art scanpath generation methods.Finally, we provide an extensive psycholinguistic analysis that underlines the model's ability to exhibit human-like reading behavior.Our implementation is made available at https://github.com/DiLi-Lab/ScanDL.

Human gaze data offer cognitive information that reflects natural language comprehension.Indeed, augmenting language models with human scanpaths has proven beneficial for a range of NLP tasks, including language understanding.However, the applicability of this approach is hampered because the abundance of text corpora is contrasted by a scarcity of gaze data.Although models for the generation of humanlike scanpaths during reading have been developed, the potential of synthetic gaze data across NLP tasks remains largely unexplored.We develop a model that integrates synthetic scanpath generation with a scanpath-augmented language model, eliminating the need for human gaze data.Since the model's error gradient can be propagated throughout all parts of the model, the scanpath generator can be fine-tuned to downstream tasks.We find that the proposed model not only outperforms the underlying language model, but achieves a performance that is comparable to a language model augmented with real human gaze data.Our code is publicly available.1

Recent work in XAI for eye tracking data has evaluated the suitability of feature attribution methods to explain the output of deep neural sequence models for the task of oculomotric biometric identification. These methods provide saliency maps to highlight important input features of a specific eye gaze sequence. However, to date, its localization analysis has been lacking a quantitative approach across entire datasets. In this work, we employ established gaze event detection algorithms for fixations and saccades and quantitatively evaluate the impact of these events by determining their concept influence. Input features that belong to saccades are shown to be substantially more important than features that belong to fixations. By dissecting saccade events into sub-events, we are able to show that gaze samples that are close to the saccadic peak velocity are most influential. We further investigate the effect of event properties like saccadic amplitude or fixational dispersion on the resulting concept influence.

We introduce pymovements: a Python package for analyzing eye-tracking data that follows best practices in software development, including rigorous testing and adherence to coding standards. The package provides functionality for key processes along the entire preprocessing pipeline. This includes parsing of eye tracker data files, transforming positional data into velocity data, detecting gaze events like saccades and fixations, computing event properties like saccade amplitude and fixational dispersion and visualizing data and results with several types of plotting methods. Moreover, pymovements also provides an easily accessible interface for downloading and processing publicly available datasets. Additionally, we emphasize how rigorous testing in scientific software packages is critical to the reproducibility and transparency of research, enabling other researchers to verify and build upon previous findings.

Neural networks that process the raw eye-tracking signal can outperform traditional methods that operate on scanpaths preprocessed into fixations and saccades. However, the scarcity of such data poses a major challenge. We, therefore, present SP-EyeGAN, a neural network that generates synthetic raw eye-tracking data. SP-EyeGAN consists of Generative Adversarial Networks; it produces a sequence of gaze angles indistinguishable from human micro- and macro-movements. We demonstrate how the generated synthetic data can be used to pre-train a model using contrastive learning. This model is fine-tuned on labeled human data for the task of interest. We show that for the task of predicting reading comprehension from eye movements, this approach outperforms the previous state-of-the-art.

In this repository we provide the Binge / PVT data set, extracted gaze features for the data set and the code to replicate the results presented in the paper "Detection of Alcohol Inebriation from Eye Movements ". The raw data set contains binocular gaze recordings and eye closure signals of 44 subjects, aged 18 to 47 with mean age 24. Each participant is recorded over 3 experimental sessions, with a time lag of at least one week between two sessions. [Entire raw data set will be added upon acceptance.]

Eye movements during reading offer insights into both the reader's cognitive processes and the characteristics of the text that is being read. Hence, the analysis of scanpaths in reading have attracted increasing attention across fields, ranging from cognitive science over linguistics to computer science. In particular, eye-tracking-while-reading data has been argued to bear the potential to make machine-learning-based language models exhibit a more human-like linguistic behavior. However, one of the main challenges in modeling human scanpaths in reading is their dual-sequence nature: the words are ordered following the grammatical rules of the language, whereas the fixations are chronologically ordered. As humans do not strictly read from left-to-right, but rather skip or refixate words and regress to previous words, the alignment of the linguistic and the temporal sequence is non-trivial. In this paper, we develop Eyettention, the first dual-sequence model that simultaneously processes the sequence of words and the chronological sequence of fixations. The alignment of the two sequences is achieved by a cross-sequence attention mechanism. We show that Eyettention outperforms state-of-the-art models in predicting scanpaths. We provide an extensive within- and across-data set evaluation on different languages. An ablation study and qualitative analysis support an in-depth understanding of the model's behavior.

Gaze patterns are known to be highly individual, and therefore eye movements can serve as a biometric characteristic. We explore aspects of the fairness of biometric identification based on gaze patterns. We find that while oculomotoric identification does not favor any particular gender and does not significantly favor by age range, it is unfair with respect to ethnicity. Moreover, fairness concerning ethnicity cannot be achieved by balancing the training data for the best-performing model.

Eye movements in reading are known to reflect cognitive processes involved in reading comprehension at all linguistic levels, from the sub-lexical to the discourse level. This means that reading comprehension and other properties of the text and/or the reader should be possible to infer from eye movements. Consequently, we develop the first neural sequence architecture for this type of tasks which models scan paths in reading and incorporates lexical, semantic and other linguistic features of the stimulus text. Our proposed model outperforms state-of-the-art models in various tasks. These include inferring reading comprehension or text difficulty, and assessing whether the reader is a native speaker of the text’s language. We further conduct an ablation study to investigate the impact of each component of our proposed neural network on its performance.

Patterns of micro- and macro-movements of the eyes are highly individual and can serve as a biometric characteristic. It is also known that both alcohol inebriation and fatigue can reduce saccadic velocity and accuracy. This prompts the question of whether changes of gaze patterns caused by alcohol consumption and fatigue impact the accuracy of oculomotoric biometric identification. We collect an eye tracking data set from 66 participants in sober, fatigued and alcohol-intoxicated states. We find that after enrollment in a rested and sober state, identity verification based on a deep neural embedding of gaze sequences is significantly less accurate when probe sequences are taken in either an inebriated or a fatigued state. Moreover, we find that fatigue and intoxication appear to randomize gaze patterns: when the model is fine-tuned for invariance with respect to inebriation and fatigue, and even when it is trained exclusively on inebriated training person, the model still performs significantly better for sober than for sleep-deprived or intoxicated subjects.

We study involuntary micro-movements of both eyes, in addition to saccadic macro-movements, as biometric characteristic. We develop a deep convolutional neural network that processes binocular oculomotoric signals and identifies the viewer. In order to be able to detect presentation attacks, we develop a model in which the movements are a response to a controlled stimulus. The model detects replay attacks by processing both the controlled but randomized stimulus and the ocular response to this stimulus. We acquire eye movement data from 150 participants, with 4 sessions per participant. We observe that the model detects replay attacks reliably; compared to prior work, the model attains substantially lower error rates.

Distributional properties of fixations and saccades are known to constitute biometric characteristics. Additionally, high-frequency micro-movements of the eyes have recently been found to constitute biometric characteristics that allow for faster and more robust biometric identification than just macro-movements. Micro-movements of the eyes occur on scales that are very close to the precision of currently available eye trackers. This study therefore characterizes the relationship between the temporal and spatial resolution of eye tracking recordings on one hand and the performance of a biometric identification method that processes micro-and macro-movements via a deep convolutional network. We find that that the deteriorating effects of decreasing both, the temporal and spatial resolution are not cumulative. We observe that on low-resolution data, the network reaches performance levels above chance and outperforms statistical approaches.