Publio Silva

dblp:278/0278 · also Publio Blenilio, Publio Blenilio Tavares Silva · DBLP profile ↗
← Back
3ranked-venue papers
1as first author
2since 2021 · last 2025
0000-0002-1655-4476ORCID · reported

Domains — the database's venue-derived domains; a paper can count in several

Software engineering, systems software and programming languages · 3 · 1 first-author · 2 since 2021
YearPublicationVenuePosition
2025 Quality Assessment of Python Tests Generated by Large Language Models
abstract
The manual generation of test scripts is a time-intensive, costly, and error-prone process, indicating the value of automated solutions. Large Language Models (LLMs) have shown great promise in this domain, leveraging their extensive knowledge to produce test code more efficiently. This study investigates the quality of Python test code generated by three LLMs: GPT-4o, Amazon Q, and LLama 3.3. We evaluate the structural reliability of test suites generated under two distinct prompt contexts: Text2Code (T2C) and Code2Code (C2C). Our analysis includes the identification of errors and test smells, with a focus on correlating these issues to inadequate design patterns. Our findings reveal that most test suites generated by the LLMs contained at least one error or test smell. Assertion errors were the most common, comprising 64% of all identified errors, while the test smell Lack of Cohesion of Test Cases was the most frequently detected (41%). Prompt context significantly influenced test quality; textual prompts with detailed instructions often yielded tests with fewer errors but a higher incidence of test smells. Among the evaluated LLMs, GPT-4o produced the fewest errors in both contexts (10% in C2C and 6% in T2C), whereas Amazon Q had the highest error rates (19% in C2C and 28% in T2C). For test smells, Amazon Q had fewer detections in the C2C context (9%), while LLama 3.3 performed best in the T2C context (10%). Additionally, we observed a strong relationship between specific errors, such as assertion or indentation issues, and test case cohesion smells. These findings demonstrate opportunities for improving the quality of test generation by LLMs and highlight the need for future research to explore optimized generation scenarios and better prompt engineering strategies.
Victor Anthony Alves, Carla I. M. Bezerra, Ivan do Carmo Machado, Larissa Rocha Soares, Tássio Virgínio, Publio Silva
EASE6
2023 Automating Feature Model maintainability evaluation using machine learning techniques
Publio Silva, Carla I. M. Bezerra, Ivan do Carmo Machado
J. Syst. Softw.1
2020 How Does Modern Code Review Impact Software Design Degradation? An In-depth Empirical Study
abstract
Software design is an important concern in modern code review through which multiple developers actively discuss and improve each single code change. However, there is little understanding of the impact of such developers' reviews on continuously reducing design degradation over time. It is even less clear to what extent and how design degradation is reversed during the process of each single code change's review. In summary, existing studies have not assessed how the process of design degradation evolution is impacted along: (i) within each single review, and (ii) across multiple reviews. As a consequence, one cannot understand how certain code review practices consistently contribute to either reduce or further increase design degradation as the project evolves. We aim at addressing these gaps through a multi-project retrospective study. By investigating 14,971 code reviews from seven software projects, we report the first study that characterizes how the process of design degradation evolves within each review and across multiple reviews. Moreover, we analyze a comprehensive suite of metrics to enable us to observe the influence of certain code review practices on combating or even accelerating design degradation. Our results show that the majority of code reviews had little to no design degradation impact in the analyzed projects. Even worse, this observation also applies, to some extent, to reviews with an explicit concern on design. Surprisingly, the practices of long discussions and high proportion of review disagreement in code reviews were found to increase design degradation. Finally, we also discuss how the study findings shed light on how to improve the research and practice of modern code review.
Anderson G. Uchôa, Caio Barbosa, Willian Nalepa Oizumi, Publio Silva, Rafael Lima, Alessandro F. Garcia 0001, Carla I. M. Bezerra
ICSME4