Architectural Decision Records (ADRs) play a central role in maintaining software architecture quality, yet many decision violations go unnoticed because projects lack both systematic documentation and automated detection mechanisms. Recent advances in Large Language Models (LLMs) open up new possibilities for automating architectural reasoning at scale. We investigated how effectively LLMs can identify decision violations in open-source systems by examining their agreement, accuracy, and inherent limitations. Our study analyzed 980 ADRs across 109 GitHub repositories using a multi-model pipeline in which one LLM primary screens potential decision violations, and three additional LLMs independently validate the reasoning. We assessed agreement, accuracy, precision, and recall, and complemented the quantitative findings with expert evaluation. The models achieved substantial agreement and strong accuracy for explicit, code-inferable decisions. Accuracy falls short for implicit or deployment-oriented decisions that depend on deployment configuration or organizational knowledge. Therefore, LLMs can meaningfully support validation of architectural decision compliance; however, they are not yet replacing human expertise for decisions not focused on code.

Abstract Practitioners and researchers continuously focus on developing automation strategies to cope with the exponentially demanding need for the timely deployment of software projects in tight release schedules. Such automation techniques include Infrastructure-as-Code (IaC) and the DevOps and DevSecOps cycles. Recent studies investigated generative AI (GenAI) for generating infrastructure as code scripts. However, no studies have focused on using GenAI to generate IaC scripts based on DevSecOps stage artifacts. Different IaC tools serve varied purposes, requiring specific infrastructure setups for different project stages. We envision GenAI models leveraging artifacts from each DevSecOps stage to create and refine IaC scripts. We trust our approach to have an impact on practitioners to leverage it as an automatic copilot for infrastructure design and deployment, and for researchers to build on our vision and future empirical validation.

Context: Over the past decade, the adoption of Microservice Architecture (MSA) has led to the identification of various patterns and anti-patterns, such as Nano/Mega/Hub services. Detecting these anti-patterns often involves modeling the system as a Service Dependency Graph (SDG) and applying graph-theoretic approaches. Aim: While previous research has explored software metrics (SMs) such as size, complexity, and quality for assessing MSAs, the potential of graph-specific metrics like network centrality remains largely unexplored. This study investigates whether centrality metrics (CMs) can provide new insights into MSA quality and facilitate the detection of architectural anti-patterns, complementing or extending traditional SMs. Method: We analyzed 24 open-source MSA projects, reconstructing their architectures to study 53 microservices. We measured SMs and CMs for each microservice and tested their correlation to determine the relationship between these metric types. Results and Conclusion: Among 902 computed metric correlations, we found weak to moderate correlation in 282 cases. These findings suggest that centrality metrics offer a novel perspective for understanding MSA properties. Specifically, ratio-based centrality metrics show promise for detecting specific anti-patterns, while subgraph centrality needs further investigation for its applicability in architectural assessments.

Abstract Architecture recovery tools help software engineers obtain an overview of the structure of their software systems during all phases of the software development life cycle. This is especially important for microservice applications because they consist of multiple interacting microservices, which makes it more challenging to oversee the architecture. Various tools and techniques for architecture recovery (also called architecture reconstruction) have been presented in academic and gray literature sources, but no overview and comparison of their accuracy exists. This paper presents the results of a multivocal literature review with the goal of identifying architecture recovery tools for microservice applications and a comparison of the identified tools’ architectural recovery accuracy. We focused on static tools since they can be integrated into fast-paced CI/CD pipelines. 13 such tools were identified from the literature and nine of them could be executed and compared on their capability of detecting different system characteristics. The best-performing tool exhibited an overall F1-score of 0.86. Additionally, the possibility of combining multiple tools to increase the recovery correctness was investigated, yielding a combination of four individual tools that achieves an F1-score of 0.91.

Researchers in the microservices community often resort to demonstrating the impact of their proposed advancements on custom-made microservices projects. This is a possible source of bias that can reduce the trustworthiness of the results. Moreover, it is hard to compare advances in small projects, often developed due to lack of time. It is common across disciplines to recognize benchmarks that mitigate bias and unify the advancements' impact. To facilitate the identification of available open-source microservice projects (OSS-MS), we performed a comprehensive study to identify, curate, and catalog OSS-MS. We started with 389559 projects and filtered them down to 3804 projects that we manually labeled. After manual labeling, our dataset contains 378 projects with three or more microservices and with over 100 commits. We document the projects from many perspectives, including project size, platform, number of contributors, project purpose, and foundation support. This dataset can serve researchers as a roadmap to identify benchmarks, as our dataset can be used to answer questions such as whether the number of services impacts the issue count.