In the fast-evolving field of artificial intelligence, Reinforcement Learning (RL) plays a crucial role in developing agents that can make decisions. As these systems become increasingly complex, the need for standardized and automated training methods becomes apparent. This paper presents a rule-based framework that integrates Large Language Models (LLMs) and heuristic-based code detectors to ensure compliance with best practices in RL training pipelines. We define a set of architectural rules that target best practices in important areas of RL-based architectures, such as checkpoints, hyperparameter tuning, and agent configuration. We validated our approach through a large-scale industrial case study and ten open-source projects. The results show that LLM-based detectors generally outperform heuristic-based detectors, especially when handling more complex code patterns. This approach effectively identifies best practices with high precision and recall, demonstrating its practical applicability.

As artificial intelligence continues to advance, Reinforcement Learning (RL) has established itself as a core approach for developing intelligent agents that make decisions over time. As RL systems grow in complexity, the need for standardized training practices becomes critical. This paper introduces a rule-based assessment approach to enforce best practices in RL training. We define a comprehensive set of architectural rules focused on RL pipeline practices, models versioning, multi-agents deployment and managing models in inference. Our methodology integrates Large Language Models (LLMs) and custom-based code detectors to ensure compliance with these best practices across diverse RL systems. We developed a ML pipeline insights service to automatically validate RL training practices directly from the source code. We validate our approach by applying it in a large-scale industrial case study and sixteen open-source case studies. Our evaluation showed that custom-based detectors achieved near-perfect precision and recall ( $$ F_1 \approx 0.98 $$ ), while LLM-based detectors provided scalable validation with moderate $$ F_1 $$ scores (0.67–0.71), demonstrating the hybrid approach’s strength in balancing accuracy and automation. The results demonstrate our tool’s accuracy in identifying and enforcing best practices with high precision and recall rates, highlighting its practical applicability and automation feasibility.

Infrastructure as Code (IaC) empowers software developers and operations teams to automate the deployment and management of IT infrastructure through code. This is particularly valuable for continuously released deployments such as microservices and cloud-based systems. IaC technologies offer flexibility in provisioning and deploying application architectures. However, if the structure is not well-designed, it can lead to severe issues related to coupling aspects. Unfortunately, a lack of comprehensive coupling guidelines for IaC makes ensuring adherence to best practices challenging. Leveraging IaC-based models, metrics, and source code can enhance the comprehension and implementation of coupling measures. Our objective was to investigate how developers understand information derived from system source code and compare it to formal IaC system diagrams and metrics. We conducted a controlled experiment involving a group of participants to evaluate the understandability of IaC system architecture descriptions through source code inspection and formal representations. We hypothesized that providing formal IaC system diagrams and metrics as supplementary materials would improve the understanding of IaC coupling-related practices measured by task correctness . We also expected that these supplementary resources would lead to a significant increase in task duration and that there would be a notable correlation between correctness and duration . The results suggest that including formal IaC system diagrams and metrics as supplementary materials significantly enhances the comprehension of IaC coupling-related practices, as indicated by task correctness . Moreover, providing these formal representations does not significantly prolong task duration , indicating that they do not hinder understanding. A substantial correlation between task correctness and duration is evident when formal IaC system diagrams and metrics are available.

Machine learning (ML) has emerged as a transformative subject, using various algorithms to help systems analyze data and make predictions. Deep Learning (DL) uses neural networks to address hard problems. Reinforcement Learning (RL) is a way to solve problems by making consecutive decisions. Understanding ML systems based only on the source code is often a challenging task, especially for inexperienced developers. In a controlled experiment involving one hundred fifty-eight participants, we assessed the understandability of ML-based systems and workflows through source code inspection compared to semi-formal representations in models and metrics. We hypothesize that ML system diagrams modeling details of ML workflows and practices like transfer learning and checkpoints can enhance the understandability of ML practices in system design comprehension tasks, assessed through task correctness . Additionally, providing these sources could lead to an increase in task duration , and we expect a significant correlation between correctness and duration . Our findings show that providing semi-formal ML system diagrams with the source code improves the effectiveness of the correctness for the DL relevant tasks. The control group had an average correctness of 0.7121, while the experimental group had a higher average correctness of 0.7759. On the other hand, participants who received only the system source code showed slightly better performance in the correctness task (average correctness 0.6808) within the RL relevant tasks compared to those who also received the semi-formal diagrams (average correctness of 0.6612). However, no significant difference was found in the duration task between the two. The control group, for the DL relevant tasks, took an average of 1571.62 s, whereas the experimental group took an average of 1763.85 s. For the RL relevant tasks, the control group had an average of 1883.80 s, while the experimental group 1925.46 s. However, semi-formal ML system diagrams can benefit specific scenarios.

In machine learning (ML) and machine learning operations (MLOps), automation serves as a fundamental pillar, streamlining the deployment of ML models and representing an architectural quality aspect. Support for automation is especially relevant when dealing with ML deployments characterised by the continuous delivery of ML models. Taking automation in MLOps systems as an example, we present novel metrics that offer reliable insights into support for this vital quality attribute, validated by ordinal regression analysis. Our method introduces novel, technology-agnostic metrics aligned with typical Architectural Design Decisions (ADDs) for automation in MLOps. Through systematic processes, we demonstrate the feasibility of our approach in evaluating automation-related ADDs and decision options. Our approach can itself be automated within continuous integration/continuous delivery pipelines. It can also be modified and extended to evaluate any relevant architectural quality aspects, thereby assisting in enhancing compliance with non-functional requirements and streamlining development, quality assurance and release cycles. • Introduces a semi-automated method for assessing MLOps system architecture qualities. • Models twenty-two MLOps architectures and develops a reusable metamodel. • Defines technology-agnostic metrics for evaluating automation in MLOps. • Employs a systematic process for sourcing and assessing various MLOps architectures. • Validates metrics through ordinal regression for reliable prediction models.

Infrastructure as Code (IaC) automates IT infrastructure deployment, which is particularly beneficial for continuous releases, for instance, in the context of microservices and cloud systems. Despite its flexibility in application architecture, neglecting security can lead to vulnerabilities. The lack of comprehensive architectural security guidelines for IaC poses challenges in adhering to best practices. We studied how developers interpret IaC scripts (source code) in two IaC technologies, Ansible and Terraform, compared to semi-formal IaC deployment architecture models and metrics regarding design-level security understanding. In a controlled experiment involving ninety-four participants, we assessed the understandability of IaC-based deployment architectures through source code inspection compared to semi-formal representations in models and metrics. We hypothesized that providing semi-formal IaC deployment architecture models and metrics as supplementary material would significantly improve the comprehension of IaC security-related practices, as measured by task correctness . Our findings suggest that semi-formal IaC deployment architecture models and metrics as supplementary material enhance the understandability of IaC security-related practices without significantly increasing duration . We also observed a significant correlation between task correctness and duration when models and metrics were provided.

In the dynamic landscape of artificial intelligence and machine learning, Reinforcement Learning (RL) has emerged as a powerful paradigm for training intelligent agents in sequential decision-making. As RL architectures progress in complexity, the need for informed decision-making regarding training strategies and related consequences on the software architecture becomes increasingly intricate. This work addresses this challenge by presenting the outcomes of a qualitative, in-depth study focused on best practices and patterns within training strategies for RL architectures, as articulated by practitioners. Leveraging a model-based qualitative research method, we introduce a formal architecture decision model to bridge the gap between scientific insights and practical implementation. We aim to enhance the understanding of practitioners' approaches in RL architecture. The paper analyzes 33 knowledge sources to discern established industrial practices, patterns, relationships, and decision drivers. Based on this knowledge, we introduce a formal Architectural Design Decision (ADD) model, encapsulating 6 decisions, 29 decision options, and 19 decision drivers, providing robust decision-making support for this critical facet of RL-based software architectures.

The Infrastructure as Code (IaC) concept enables IT infrastructure to be managed as software: resources can be managed, monitored, and provisioned automatically instead of manually by developers or operations teams. Many industries have already embraced this concept widely. However, research on IaC-based deployments, particularly research focusing on loose coupling, often does not offer methods for evaluating architectural conformance, spotting architecture smells, and support for correcting the found smells. Our work strives to provide an automatic method for continuously developing microservice-based systems and the associated infrastructure. We aim to offer an automated architectural refactoring method that checks if IaCbased deployments adhere to patterns and best practices and do not contain potential architectural smells. We provide architects with viable options for enhancing architectural conformance during microservice development. In short, by continuously detecting architectural smells and suggesting possible fixes, we aim to support architecture evolution within the framework of continuous delivery practices. We evaluate our approach using three case studies and variants based on open-source microservice architectures.

Designing Cyber-Physical Systems (CPS) is a complex task involving integrating physical and digital components to achieve specific objectives. This process consolidates data from various Internet of Things (IoT) devices and sources to generate meaningful insights and actionable outcomes. IoT-cloud data communication comprises multiple stages, e.g., data collection, processing, analysis, and visualization. Adopting a comprehensive approach that considers physical and digital aspects is essential to ensure effective data communication in CPS. As a result, architectural design choices are crucial in determining CPS functionality and runtime qualities, e.g., performance, security, and reliability. While numerous CPS architectural patterns and practices have been proposed, much of the relevant knowledge remains scattered across various sources, such as practitioner blogs and system documentation. These sources are often based on personal experiences and lack consistency. To address this gap, our study presents the outcomes of an in-depth qualitative investigation into practitioners' descriptions of the best practices and patterns in CPS architecture. We have developed a formal architectural decision model using a model-based qualitative research method. We aim to bridge the division between scientific understanding and practical use cases, enhance comprehension of practitioners' approaches to CPS, and provide decision-making support for designing CPS applications.

Microservices are one of the most recommended architectural styles for distributed applications that support independent development and deployment, enable rapid release, and are highly scalable and polyglot. Many well-established patterns and best practices have been documented in the literature. As there are many such guidances, they have numerous interdependencies, and system designs must adhere to many other architecture constraints, too, implementations do not always conform to those guidances. In complex or large systems, it can be hard and tedious to spot violations. Our work aims to offer automated support for architecting during the continuous evolution of microservice-based systems. More specifically we aim to provide the foundations for an automated approach for architecture reconstruction, assessing conformance to patterns and practices specific for microservice architectures, and detect possible violations. Based on this, we provide actionable options to architects for improving architecture conformance as part of a continuous feedback loop. That is, our goal is to support architecting in the context of continuous delivery practices, where architecture violations are continuously analyzed and fix options are continuously suggested.

Mature verification and monitoring approaches, such as complex event processing and model checking, can be applied for checking compliance specifications at design time and runtime. Little is known about the understandability of the different formal and technical languages associated with these approaches. This uncertainty regarding understandability might be a major obstacle for the broad practical adoption of those techniques. This article reports a controlled experiment with 215 participants on the understandability of modeling compliance specifications in representative modeling languages, namely linear temporal logic (LTL), the complex event processing-based event processing language (EPL) and property specification patterns (PSP). The formalizations in PSP were overall more correct. That is, the pattern-based approach provides a higher level of understandability than EPL and LTL. More advanced users, however, seemingly are able to cope equally well with PSP and EPL in modeling compliance specifications.