Service-based systems (SBS) depend on loosely coupled services that interact via standardized protocols, making testing challenging due to external dependencies. Mocking and service virtualization techniques have been proposed to simulate these dependencies and support automated testing, yet existing research lacks a unified synthesis of their approaches, effectiveness, and limitations.

Context:Code understandability plays a crucial role in software development, as developers spend between 58% and 70% of their time reading source code. Improving code understandability can lead to enhanced productivity and save maintenance costs.Problem:Experimental studies aim to establish what makes code more or less understandable in a controlled setting, but ignore that what makes code easier to understand in the real world also depends on extraneous elements such as developers’ background and project culture and guidelines. Not accounting for the influence of these factors may lead to results that are sound but have little external validity.Goal:We aim to investigate how developers improve code understandability during software development through code review comments. Our assumption is that code reviewers are specialists in code quality within a project.Method and Results:We manually analyzed 2,401 code review comments from Java open-source projects on GitHub and found that over 42% of all comments focus on improving code understandability, demonstrating the significance of this quality attribute in code reviews. We further explored a subset of 385 comments related to code understandability and identified eight categories of code understandability concerns, such as incomplete or inadequate code documentation, bad identifier, and unnecessary code. Among the suggestions to improve code understandability, 83.9% were accepted and integrated into the codebase. Among these, only two (less than 1%) ended up being reverted later. We also identified types of patches that improve code understandability, ranging from simple changes (e.g., removing unused code) to more context-dependent improvements (e.g., replacing method calling chains by existing API). Finally, we investigated the potential coverage of four well-known linters to flag the identified code understandability issues. These linters cover less than 30% of these issues, although some of them could be easily added as new rules.Implications:Our findings motivate and provide practical insight for the construction of tools to make code more understandable, e.g., understandability improvements are rarely reverted and thus can be used as reliable training data for specialized ML-based tools. This is also supported by our dataset, which can be used to train such models. Finally, our findings can also serve as a basis to develop evidence-based code style guides.

Existing studies on the use of Large Language Models (LLMs) in software development leverage methodologies that limit their scalability and require intensive manual data collection and analysis, for example, due to the use of video data or think-aloud protocols. We propose the use of a specialized tool capable of automatically collecting fine-grained, relevant data during experiments and case studies. It enables researchers to understand for example how often participants accept or reject suggestions made by LLMs and what kinds of prompts are more likely to trigger accepted suggestions, even in studies targeting a large number of participants. We implement this idea as a Visual Studio Code plugin named AthenaLLM 1. It mimics the functionalities of GitHub Copilot and offers seamless integration with OpenAI API models like GPT-4 and GPT-3.5, and compatibility with other models providing an OpenAI-compatible API, e.g., Vicuna [6]. It automatically collects data at a fine level of granularity and covers both the interactions of developers with their IDE, e.g., all changes made in the code, and the products of such interactions, e.g., the generated code, when accepted. Thus, the proposed approach also reduces bias that the experimental process itself may introduce, e.g., due to the need for participants to verbalize their thoughts. In this paper we discuss how AthenaLLM could enable researchers to go both broader (in terms of number of participants) and deeper (in terms of the kinds of research questions that can be tackled).