Transcripts displayed on dictation interfaces can be hard to read due to recognition errors and disfluencies. LLM-based text auto-correction could help, but changing the text during production could lead to distraction and unintended phrasing. To understand how to balance readability, attention, and accuracy, we conducted an eye-tracking experiment with 20 participants to compare five dictation interfaces: PLAIN (real-time transcription), AOC (periodic corrections), RAKE (keyword highlights), GP-TSM (grammar-preserving highlights), and SUMMARY (LLM-generated abstractive summary). By analyzing participants’ gaze patterns during speech composition and reviewing processes, we found that during composition, participants spent only 7%-11% of their time in active reading regardless of the interface. Although SUMMARY introduced unfamiliar words and phrasing during composition, it was easier to read and more preferred by participants. Our findings suggest a high user tolerance for altering spoken words in LLM-enabled diction interfaces.

No single eye-tracking correction algorithm is universally effective across all reading patterns and distortions. Yet, a single algorithm is often chosen to correct entire datasets, leading to non-optimal correction performance. In this paper, we present an adaptive approach that dynamically selects the optimal algorithm for each trial, significantly enhancing correction accuracy. The proposed adaptive approach relies on identifying the eye movement patterns in each trial, and based on the combination of patterns present the optimal algorithm is used to correct the trial. We assess this approach with synthetic data and four real datasets with a total of 152 trials, comparing the results to 15 algorithms. Our results confirm that none of the previous algorithms is able to handle all reading patterns and distortions. However, the results show that the proposed approach is the best performing in simulated trials and real data.

A number of factors come together to limit the diversity of eye-tracking research, where the majority of papers are conducted with stimuli in the English language. Studying eye movement over other languages is important considering that each language provides unique insights into human cognition. Recently, there have been valued efforts to present datasets from other languages, yet these efforts focused mostly on European languages. In this paper we highlight issues that limit diversity in eye tracking research on reading, and we present our work in collecting an open-access multilingual reading dataset of underrepresented languages. Utilizing a high-frequency research eye tracker (EyeLink 1000 Plus), we record eye tracking data of native and second language readers of English, Spanish, Chinese, Hindi, Russian, Arabic, Japanese, Kazakh, Urdu, and Vietnamese. The dataset includes demographics, language proficiency self-reporting, and answers to comprehension questions. The current version of the dataset, which we make publicly available, consists of 97 trials by 40 participants. With the goal of increasing the number of participants and included languages, we aim to make studying underrepresented languages more accessible to researchers and tool makers.

Recent research highlights the viability of webcam-based eye tracking as a low-cost alternative to dedicated remote eye trackers. Simultaneously, research shows the importance of understanding emotions of software developers, where it was found that emotions have significant effects on productivity, code quality, and team dynamics. In this paper, we present our work towards an integrated eye-tracking and affect recognition tool for use during software development. This combined approach could enhance our understanding of software development by combining information about the code developers are looking at, along with the emotions they experience. The presented tool utilizes an unmodified webcam to capture video of software developers while interacting with code. The tool passes each frame to two modules, an eye tracking module that estimates where the developer is looking on the screen, and an affect recognition module that infers developer emotion from their facial expressions. The proposed work has implications to researchers, educators, and practitioners, and we discuss some potential use cases in this paper.

The viability and need for eye movement-based authentication has been well established in light of the recent adoption of Virtual Reality headsets and Augmented Reality glasses. Previous research has demonstrated the practicality of eye movement-based authentication, but there still remains space for improvement in achieving higher identification accuracy. In this study, we focus on incorporating linguistic features in eye movement based authentication, and we compare our approach to authentication based purely on common first-order metrics across 9 machine learning models. Using GazeBase, a large eye movement dataset with 322 participants, and the CELEX lexical database, we show that AdaBoost classifier is the best performing model with an average F1 score of 74.6%. More importantly, we show that the use of linguistic features increased the accuracy of most classification models. Our results provide insights on the use of machine learning models, and motivate more work on incorporating text analysis in eye movement based authentication.

Background: Recent advancements in large language models have motivated the practical use of such models in code generation and program synthesis. However, little is known about the effects of such tools on code readability and visual attention in practice. Objective: In this paper, we focus on GitHub Copilot to address the issues of readability and visual inspection of model generated code. Readability and low complexity are vital aspects of good source code, and visual inspection of generated code is important in light of automation bias. Method: Through a human experiment (n=21) we compare model generated code to code written completely by human programmers. We use a combination of static code analysis and human annotators to assess code readability, and we use eye tracking to assess the visual inspection of code. Results: Our results suggest that model generated code is comparable in complexity and readability to code written by human pair programmers. At the same time, eye tracking data suggests, to a statistically significant level, that programmers direct less visual attention to model generated code. Conclusion: Our findings highlight that reading code is more important than ever, and programmers should beware of complacency and automation bias with model generated code.

Identifier naming is one of the main sources of information in program comprehension, where a significant portion of software development time is spent. Previous research shows that similarity in identifier names could potentially hinder code comprehension, and subsequently code maintenance and evolution. In this paper, we present an open-source tool for assessing confusing naming combinations in Python programs. The tool which we call Namesake, flags confusing identifier naming combinations that are similar in orthography (word form), phonology (pronunciation), or semantics (meaning). Our tool extracts identifier names from the abstract syntax tree of a program, splits compound names, and evaluates the similarity of each pair in orthography, phonology, and semantics. Problematic identifier combinations are flagged to programmers along with their line numbers. In combination with existing coding style checkers, Namesake can provide programmers with an additional resource to enhance identifier naming quality. The tool can be integrated easily in DevOps pipelines for automated checking and identifier naming appraisal.

Introductory computer science courses are characterized by difficulty, which may contribute to the low success rate, diversity, and retention in these key courses. Difficulty in programming projects was found to result in negative self-efficacy perception among students, in addition to affecting underrepresented students disproportionately. In this paper, we focus on perceived workload in introductory computer science projects and report on the use of NASA Task Load Index (NASA-TLX) as a subjective measure of student workload. Through two experiments involving six CS1 and CS2 courses, we demonstrate how NASA-TLX can be used to gain insights on the contributors and components of workload in programming projects. We show how when combined with race/ethnicity and gender data, NASA-TLX is useful in understanding the experience of underrepresented students. Our results suggest that perceived workload is only partially influenced by actual workload as measured in lines of code, function, and class count. Moreover, we found that the time spent on programming projects in comparison to other courses is a predictor of perceived workload. Finally, we discuss how educators can use NASA-TLX to identify at-risk students and make difficult projects more accessible without sacrificing quality.

Program comprehension is a vital skill in software development. This work investigates program comprehension by examining the eye movement of novice programmers as they gain programming experience over the duration of a Java course. Their eye movement behavior is compared to the eye movement of expert programmers. Eye movement studies of natural text show that word frequency and length influence eye movement duration and act as indicators of reading skill. The study uses an existing longitudinal eye tracking dataset with 20 novice and experienced readers of source code. The work investigates the acquisition of the effects of token frequency and token length in source code reading as an indication of program reading skill. The results show evidence of the frequency and length effects in reading source code and the acquisition of these effects by novices. These results are then leveraged in a machine learning model demonstrating how eye movement can be used to estimate programming proficiency and classify novices from experts with 72% accuracy.