Abstract Agile software development often faces challenges related to architectural uncertainty. ArchHypo addresses this by providing a hypothesis-driven architecture technique that helps teams formulate, test, and learn from architectural assumptions iteratively. However, studies have shown that the lack of tooling integrated into everyday development workflows is a significant barrier to its adoption. In this paper, we present , a Trello plugin that integrates hypothesis-driven architectural reasoning directly into agile boards and validates the ArchHypo technique in real projects. The plugin uses an LLM and a RAG mechanism to help teams generate and classify architectural hypotheses, develop technical plans, and link actions to architecture decision patterns. By operating on top of an existing project management tool, aims to lower the adoption cost of hypothesis engineering and to make architectural decision processes more observable. We evaluated in a controlled study with software professionals working on a realistic architecture scenario. The results indicate that the plugin helps structure architectural discussions, reduces manual effort in documenting hypotheses and plans, clarifies procedural steps, and surfaces differences in risk perception within teams. Qualitative feedback suggests that AI-assisted support facilitates collaborative reasoning about architecture. Our findings show that LLM-based tools can effectively support hypothesis-driven architecture in agile settings and highlight design considerations for integrating such tools into existing workflows.

Abstract Architectural uncertainties arising from incomplete or unclear information pose significant challenges when making architectural decisions in Agile teams. Based on a limited number of case studies that employed a technique called ArchHypo, four patterns were identified that propose small adjustments in the development process to handle architectural uncertainties: Protective Guideline , Bring the Specialist , Plan for Preparation , and Quality Checkpoint . Although the patterns derived from these experiences can be useful in real projects, their applicability and consequences were based on limited evidence and specific scenarios. To address this issue, this paper presents an interview study with experienced software architects and engineers to gather further information on the application of these patterns. The research method employed semi-structured interviews to gather the experiences of professionals with the target practices, and thematic analysis was used to assess their recurrence, applicability, and consequences. The findings confirmed that most professionals recognized those practices in real projects and their suitability as actions in uncertainty management. Moreover, new positive and negative consequences, not previously documented in the patterns, were identified. As a result, this work contributes to the field by providing guidance to professionals on how to better evaluate the trade-offs of those patterns when applied to architecture uncertainty management.

Abstract Previous research has identified mutation testing as a promising technique for detecting unintended changes in test behavior during test code refactorings. Despite its theoretical support, the practical adoption of this approach has been hindered by a lack of corresponding tools. Consequently, these studies have been unable to fully validate the effectiveness of mutation testing as a guardrail to ensure the consistency of the refactored test behavior, leaving an in-depth empirical validation open for future research. To address this gap, this study examines , a tool developed as a reference implementation to support test refactoring by using mutation testing. We leverage to validate the practical applicability of the mutation testing approach across diverse test refactoring scenarios. This evaluation uses a catalog of common test refactorings that reflect real-world practices. The results indicate that effectively detects changes in test behavior in most cases, demonstrating the efficacy of mutation testing to identify problems during test code refactoring. However, the study also identifies limitations, particularly the occurrence of false negatives when refactorings modify the way tests handle dependencies. These findings highlight the potential of the approach and contribute to the state-of-the-art by identifying limitations that can be addressed in future studies.

Uncertainty is present in software architecture decisions due to a lack of knowledge about the requirements and the solutions involved. However, this uncertainty is usually not made explicit, and decisions can be made based on unproven premises or false assumptions. This paper focuses on a technique called ArchHypo that uses hypotheses engineering to manage uncertainties related to software architecture. It proposes formulating a technical plan based on each hypothesis’ assessment, incorporating measures able to mitigate its impact and reduce uncertainty. To evaluate the proposed technique, this paper reports an application of the technique in a mission-critical project that faced several technical challenges. Conclusions were based on data extracted from the project documentation and a questionnaire answered by all team members. As a result, the application of ArchHypo provided a structured approach to dividing the architectural work through iterations, which facilitated architectural decision-making. However, further research is needed to fully understand its impact across different contexts. On the other hand, the team identified the learning curve and process adjustments required for ArchHypo's adoption as significant challenges that could hinder its widespread adoption. In conclusion, the evidence found in this study indicates that the technique has the potential to provide a suitable way to manage the uncertainties related to software architecture, facilitating the strategic postponement of decisions while addressing their potential impact.

The widespread use of machine learning (ML) in software engineering (SE) encounters a notable challenge: the need for various domain-specific parameters in algorithms. The issue arises when attempting to reuse these parameters across different applications, resulting in sub-optimal outcomes. This hindrance significantly contributes to the limited migration of ML solutions from research labs to industrial settings. This paper underscores the pressing need for novel research to tackle the overarching problem of generic algorithm customisation. To address this, we propose leveraging Automated Machine Learning (AutoML) approaches. These techniques automatically select intelligible models and their corresponding hyper-parameters for forecasting software defect-proneness. More specifically, this paper adapts an AutoML approach to the field of software defect prediction, namely a hyper-heuristic evolutionary algorithm for automatically designing decision tree algorithms (HEAD-DT), originally proposed to address a generic optimisation problem. We benchmark against the popular general software defect-proneness prediction framework (GSDP) and some standard classifiers. Experimental results reveal that the proposed HEAD-DT implementation surpasses other algorithms across three distinct evaluation measures.

Software Testing (ST) is one of the least known aspects of software development. Yet, software engineers often argue that it demands more than half of the costs of a software project. Thus, proper testing education is of paramount importance. In fact, the mere exposition to ST knowledge might have an impact on programming skills. In particular, it can encourage the production of more reliable code. Although this is intuitive, to the best of our knowledge, there are no empirical studies about such effects. Evidence on this matter is important to motivate - or demotivate - classical testing education. Concerned with this, we have conducted a study to investigate the possible impact of ST knowledge on the production of reliable code. Our controlled experiment involved 28 senior-level Computer Science students, 8 auxiliary functions with 92 test cases, and a total of 112 implementations. Results show that code delivered after the exposition to ST knowledge is, on average, 20% more reliable (a significant difference at the 0.01 level). Also, implementations delivered afterwards are not significantly larger in terms of lines of code. This indicates that ST knowledge can make developers produce more reliable software with no additional overhead in terms of program size.

A considerable part of software systems is comprised of functions that support the main modules, such as array or string manipulation and basic math computation. These auxiliary functions are usually considered less complex, and thus tend to receive less attention from developers. However, failures in these functions might propagate to more critical modules, thereby affecting the system's overall reliability. Given the complementary role of auxiliary functions, a question that arises is whether agile practices, such as pair programming and test-first programming, can improve their correctness without affecting time-to-market. This paper presents an empirical assessment comparing the application of these agile practices with more traditional approaches. Our study comprises independent experiments of pair versus solo programming, and test-first versus test-last programming. The first study involved 85 novice programmers who applied both traditional and agile approaches in the development of six auxiliary functions within three different domains. Our results suggest that the agile practices might bring benefits in this context. In particular, pair programmers delivered correct implementations much more often, and test-first programming encouraged the production of larger and higher coverage test sets. On the downside, the main experiment showed that both practices significantly increase total development time. A replication of the test-first experiment with professional developers shows similar results.

In the development of automated tests, there is an increase in complexity when the initialization or assertion are related to shared resources. Usually these issues are addressed and implemented in the test classes. This paper presents a pattern that suggests a solution to simplify the initialization and assertion through tests metadata classes. This solution allows each method to has specific assertions and initializations, isolation the solutions out of the test classes, allowing the reusage by other test classes. Current solutions like mock objects do not test actually external dependencies, because they simulate the external resource. For initialization and verification in the tests of these dependencies, other APIs are required in the test class and thus the test becomes more complex.

The usage of metadata-based frameworks is becoming popular for some kinds of software, such as Web and enterprise applications. However, it is not clear for which kinds of problems this approach can be applied. This paper presents a study that investigated the metadata usage in existing frameworks and documented recurrent solutions as architectural patterns. As a result, software architects might use such approaches for similar problems, being aware of their benefits and drawbacks in each scenario.