Program comprehension is a central research topic in software engineering, focusing on how developers understand a program's structure, behavior, and intent. Eye-tracking studies have traditionally relied on display-based measurements, where gaze positions are represented as screen coordinates. However, syntax-based analyses have recently emerged. Prior work proposed methods to convert eye movements into transitions between nodes in an abstract syntax tree, but the relationship between task correctness and eye-movement features for specific syntactic elements remains unclear. This study converts eye-tracking data into transitions between syntactic nodes and analyzes fixation proportions and gaze transition patterns. We investigate the relationship between these patterns and task correctness, comparing correct and incorrect groups. Our results reveal distinct differences in gaze transition patterns between the two groups. In particular, successful participants exhibit more systematic transitions across syntactic elements, suggesting the use of structured reading strategies.

Code readability is fundamental to software quality and maintainability. Poor readability extends development time, increases bug-inducing risks, and contributes to technical debt. With the rapid advancement of Large Language Models, AI agent-based approaches have emerged as a promising paradigm for automated refactoring, capable of decomposing complex tasks through autonomous planning and execution. While prior studies have examined refactoring by AI agents, these analyses cover all forms of refactoring, including performance optimization and structural improvement. As a result, the extent to which AI agent-based refactoring specifically improves code readability remains unclear.

This paper presents eye2vec, an infrastructure for analyzing software developers' eye movements while reading source code. In common eye-tracking studies in program comprehension, researchers must preselect analysis targets such as control flow or syntactic elements, and then develop analysis methods to extract appropriate metrics from the fixation for source code. Here, researchers can define various levels of AOIs like words, lines, or code blocks, and the difference leads to different results. Moreover, the interpretation of fixation for word/line can vary across the purposes of the analyses. Hence, the eye-tracking analysis is a difficult task that depends on the time-consuming manual work of the researchers. eye2vec represents continuous two fixations as transitions between syntactic elements using distributed representations. The distributed representation facilitates the adoption of diverse data analysis methods with rich semantic interpretations.

The Online Judge System (OJS) is well-used in programming courses at universities or for self-learning. The system compiles and executes a set of source codes submitted in response to an assignment and automatically grades them by comparing the output results and expectations. In university programming courses, especially courses for beginners, students repeatedly modify and submit source code until they receive a 100-point score. The OJS stores every source code until each student gets 100 points on an assignment; the differences through the first to last submissions contain helpful information to estimate students' understanding of syntax or learning units in the course. In this study, the authors propose difference flow, a series of syntax trees extracted from the differences between the final submission and every previous one. The difference flow contains the node where the difference with the 100-point source code and each parent node; hence, its features (such as the count of each syntax node throughout the flow) may indicate the students' understanding.

Explaining a program to others helps get others to find bugs and for the explainer him/herself to find bugs. However, to the best of our knowledge, there is no quantitative evidence that explaining a program to others helps the explainer find bugs. This study aims to show quantitatively, using an experimental evaluation, that the explainer himself can find new bugs by explaining the program to others. In the experiment, subjects first review a program that contains many bugs and try to find as many bugs as possible. Next, they are required to explain the program aloud to others. We see if they notice any new bugs themselves during the explanation. As a result of the experiment, five out of the six subjects could find new bugs when explaining the program to others. According to the questionnaire to the subjects, the subjects who find many bugs feel that they can understand the program better by explaining it to others.

Understanding program comprehension is one of the fundamental challenges of supporting software development. Although code writing is usually performed on programming specific environment, code reading is forced to be conducted in general environments such as physical paper. Our main hypothesis is that such a non-programming specific environment has some obstacles for program comprehension in terms of code presentation. The goal of this paper is to understand the effects on cognitive load caused by the obstacles. If our hypothesis will be proved and the goal will be achieved, we can provide the best practice of code presentation in non-programming specific environment.

Program comprehension is one of the important cognitive processes in software maintenance. The process typically involves diverse mental activities such as understanding of source code, library usages, and requirements. Systematic supports would be improved if the supports can be aware of such fine-grained mental activities during program comprehension. Here we aim to investigate whether biometric data can be varied according to such mental activity classes and conduct an experiment with program comprehension tasks involving multiple documents. As a result, we successfully classified the success/failure of the tasks at 85.2% from electroencephalogram (EEG) combined with focused document types. This result suggests that our metrics based on EEG and focused document types might be beneficial to detect developers' diverse mental activities triggered by different documents.

We focus on developer code review performance, and analyze whether the age of a subject affects the efficiency and preciseness of their code. Generally, older coders have more experience. Therefore, the age is considered to positively affect code review. However, in our past study, code understanding speed was relatively slow for older subjects, and memory is needed to understand programs. Similarly, during code review, a subject's age may affect efficiency (e.g., the number of indications per unit time). In the experiment, subjects reviewed source code, referring to mini specification documents. When the code did not follow the document, the subjects indicated the error. We classified subjects into senior and junior groups. In the analysis, we stratified the results based on age, and used correlation coefficients and multiple linear regression to clarify the relationship between age and review performance. We found that age does not affect the efficiency and correctness of code review. Also, the software development experience of subjects is not significantly correlated to performance.

Near infrared spectroscopy (NIRS) has been used as a low cost, noninvasive method to measure brain activity. In this paper, we experiment to measure the effects of variables and controls in a source code to the brain activity in program comprehension. The measurement results are evaluated after noise reduction and normalization to statistical analysis. As the result of the experiment, significant differences in brain activity were observed at a task that requires memorizing variables to understand a code snippet. On the other hand, no significant differences between different levels of mental arithmetic tasks were observed. We conclude that the frontal pole reflects workload to short-term memory caused by variables without affected from calculation.

This paper proposes a method to recommend the relevant information of the document which recorded in the development support systems such as BTS and VCS. We improve a system which we implemented in previous work with the method proposed in this paper. Our method get a document from the support systems, extract the word, then calculate the feature vector based on the TF-IDF value of each word. In the experiment, we apply the proposal method to the dataset from an open source software projects, and evaluate the recommendation accuracy between the six clustering algorithm. The result of the experiment shows that the proposed method improves the recommendation accuracy compared with the previous work.

This paper proposes a method to support Personal Software Process (PSP) in a development organization by classify the operations on a computer into a purpose of the user. PSP requires the developers to record and analyze their activity during the development process. There are several methods and systems to support the PSP, they records the operations automatically and also records a purpose of the operations (task) which records manually by the developer. Such manual recording by the developers is a barrier to introduction of the PSP system, and the cause of inaccurate record histories. Our proposal method classifies the operations into the task automatically with the chronological operation history. The method hypothesize that the each task consists of successive operation. The method classify the each operation into the task with a machine learning algorithm, Random Forests. An Experiment result shows the proposal method with chronological operation history classify the operation into the tasks more accurately than the method without the chronological operation history.

In this paper, we present a replicated study to predict fault-prone modules with code clone metrics to follow Baba's experiment. We empirically evaluated the performance of fault prediction models with clone metrics using 3 datasets from the Eclipse project and compared it to fault prediction without clone metrics. Contrary to the original Baba's experiment, we could not significantly support the effect of clone metrics, i.e., the result showed that F1-measure of fault prediction was not improved by adding clone metrics to the prediction model. To explain this result, this paper analyzed the relationship between clone metrics and fault density. The result suggested that clone metrics were effective in fault prediction for large modules but not for small modules.

To clarify the characteristics of cost-overrun software projects, this paper focuses on the cost to sales ratio of software development, computed from financial information of a midsize software company in the embedded systems domain, and analyzes the correlation with outsourcing ratio as well as code reuse ratio and relative effort ratio per development phase. As a result, we found that the lower cost to sales ratio projects had the higher relative effort ratio in external design phase, which indicates that spending less effort in external design can cause decrease of profit. We also found that high outsourcing ratio projects had higher cost to sales ratio, and that projects having moderate code reuse ratio had lower and disperse cost to sales ratio, which suggests troubles in code reuse can damage the profit of a project.

ESEM '08 : the Second ACM-IEEE international symposium on Empirical software engineering and measurement, October 09-10, 2008, Kaiserslautern, Germany

To build high-reliability software in software development, software review is essential. Typically, software review requires documents from multiple phases such as requirements specification, design document and source code to reveal the inconsistencies among them and to ensure the traceability of deliverables. However, most previous studies on software review (reading) techniques focus on finding defects in a single document in their experiments. In this paper, we propose a multi-document review evaluation system, DRESREM2. This system records reviewers' eye movements and mouse/keyboard operations for analysis. We conducted eye gaze analysis of reviewers in design document review with multiple documents (including requirements specification, design document, etc.) to confirm the usefulness of the system. For the performance analysis, we recorded defect detection ratio, detection time per defect, and fixation ratio of eye movements on each document. As a result, reviewers who concentrated their eye movements on requirements specification found more defects in the design document. We believe this result is good evidence to encourage developers to read high-level documents when reviewing lowlevel documents.

This paper proposes to use eye movements to characterize the performance of individuals in reviewing source code of computer programs. We first present an integrated environment to measure and record the eye movements of the code reviewers. Based on the fixation data, the environment computes the line number of the source code that the reviewer is currently looking at. The environment can also record and play back how the eyes moved during the review process. We conducted an experiment to analyze 30 review processes (6 programs, 5 subjects) using the environment. As a result, we have identified a particular pattern, called scan, in the subjects' eye movements. Quantitative analysis showed that reviewers who did not spend enough time for the scan tend to take more time for finding defects.