This special issue of the Proceedings of the ACM on Human-Computer Interaction includes accepted full papers from the ACM Symposium on Eye Tracking Research and Applications (ETRA). ETRA is the premier eye-tracking conference that brings together researchers from across disciplines to present advances in eye-tracking systems and methods, oculomotor research, eye movement data analysis, gaze-based interaction, and eye-tracking applications. A total of 24 full papers were accepted from 80 submissions after a rigorous reviewing process (30% acceptance rate). Accepted contributions were split into special issues in two journals, depending on the fit of topic and authors' preferences. 16 accepted papers are included in this issue of the Proceedings of the ACM on Human-Computer Interaction. 8 will be published in the Proceedings of the ACM on Computer Graphics and Interactive Techniques. All accepted papers are invited to present at ETRA 2025 (May 26 - May 29, 2025, in Tokyo). We would like to thank all members of the Editorial Board and all external reviewers for their effort and dedication, as well as all authors for their high-quality contributions.

We are excited to provide the third issue of the Proceedings of the ACM on Human-Computer Interaction focusing on contributions from the ACM Eye Tracking Research and Applications (ETRA) community. ETRA is the premier eye-tracking conference bringing together researchers from across disciplines to present advances in eye-tracking systems and methods, oculomotor research, eye movement data analysis, gaze-based interaction, and eye-tracking applications. A total of 25 full papers were accepted for ETRA 2024 (June 4 - 7, 2024, in Glasgow, U.K.), selected from 68 submissions after a rigorous reviewing process (37% acceptance rate). Depending on the topic fit and authors' preferences, accepted papers were split into two issues. 19 accepted papers are presented in this issue, and 6 will be included in an issue of the Proceedings of the ACM on Computer Graphics and Interactive Techniques. We want to thank all members of the Editorial Board and all external reviewers for their effort and dedication, as well as all authors for their high-quality contributions.

The relative contribution of bottom-up and top-down attentional guidance is a central topic in vision research. Whereas attention is guided bottom-up by low-level saliency, top-down guidance involves the viewer’s knowledge and expectations accumulated throughout a lifetime. Here we explore the influence of high-level scene-object relations on viewing behavior. To assess top-down guidance, we score the relevance of linguistic object labels using methods from document analysis. Specifically, we computed the term frequency-inverse document frequency (TF-IDF), a statistic that reflects how important a term is to a document. We use object TF-IDF to measure how important a specific object is to a scene category and use these scores to predict eye movement distributions over scenes. Our results show that scene-specific objects are more likely to be fixated. Object TF-IDF had an effect partially independent of image saliency, suggesting that an object’s relevance for a scene category affects attention during scene perception.

A long tradition in eye movement research has focused on three linguistic variables explaining fixation durations during sentence reading: word length, frequency, and predictability. Lengths and frequencies are easily obtainable but predictabilities are tedious to collect, requiring the incremental cloze procedure. Modern large language models are trained using the objective of predicting the next word given previous context, hence they readily provide predictability information. This capability has largely been overlooked in eye movement research. Here we investigate the suitability of a synthetic predictability measure, extracted from pretrained GPT-2 models, as a surrogate for cloze predictability. Using several published eye movement corpora, we find that synthetic and cloze predictabilities are highly correlated, and that their influence on eye movements is qualitatively similar. Similar patterns are obtained when including synthetic predictabilities in data sets lacking cloze predictabilities. In conclusion, synthetic predictabilities can serve as a substitute for empirical cloze predictabilities.

We develop a method for the automated detection and segmentation of speech balloons in comic books, including their carrier and tails. Our method is based on a deep convolutional neural network that was trained on annotated pages of the Graphic Narrative Corpus. More precisely, we are using a fully convolutional network approach inspired by the U-Net architecture, combined with a VGG-16 based encoder. The trained model delivers state-of-the-art performance with an F1-score of over 0.94. Qualitative results suggest that wiggly tails, curved corners, and even illusory contours do not pose a major problem. Furthermore, the model has learned to distinguish speech balloons from captions. We compare our model to earlier results and discuss some possible applications.