Software architecture optimization aims to enhance non-functional attributes like performance and reliability while meeting functional requirements. Multi-objective optimization employs metaheuristic search techniques, such as genetic algorithms, to explore feasible architectural changes and propose alternatives to designers. However, this resource-intensive process may not always align with practical constraints. This study investigates the impact of designer interactions on multi-objective software architecture optimization. Designers can intervene at intermediate points in the fully automated optimization process, making choices that guide exploration towards more desirable solutions. Through several controlled experiments as well as an initial user study (14 subjects), we compare this interactive approach with a fully automated optimization process, which serves as a baseline. The findings demonstrate that designer interactions lead to a more focused solution space, resulting in improved architectural quality. By directing the search toward regions of interest, the interaction uncovers architectures that remain unexplored in the fully automated process. In the user study, participants found that our interactive approach provides a better trade-off between sufficient exploration of the solution space and the required computation time.

Increased resilience compared to monolithic architectures is both one of the key promises of microservice-based architectures and a big challenge, e.g., due to the systems’ distributed nature. Resilience assessment through simulation requires fewer resources than the measurement-based techniques used in practice. However, there is no existing simulation approach that is suitable for a holistic resilience assessment of microservices comprised of (i) representative fault injections, (ii) common resilience mechanisms, and (iii) time-varying workloads. This paper presents MiSim — an extensible simulator for resilience assessment of microservice-based architectures. It overcomes the stated limitations of related work. MiSim fits resilience engineering practices by supporting scenario-based experiments and requiring only lightweight input models. We demonstrate how MiSim simulates (1) common resilience mechanisms — i.e., circuit breaker, connection limiter, retry, load balancer, and autoscaler — and (2) fault injections — i.e., instance/service killing and latency injections. In addition, we use TeaStore, a reference microservice-based architecture, aiming to reproduce scaling behavior from an experiment by using simulation. Our results show that MiSim allows for quantitative insights into microservice-based systems’ complex transient behavior by providing up to 25 metrics.

In a microservice system, runtime changes such as failures, deployments, or self-adaptation can trigger the system to transition from one steady state to another, i.e., exhibiting transient behavior. To assess a system’s quality, it is imperative that this transient behavior is specified in non-functional requirements and that stakeholders can analyze whether these requirements are met. Yet, there is little support for either specifying transient behavior as a non-functional requirement or analyzing how such a requirement is met in production. We aim to make these two tasks more accessible by utilizing novel human-computer interaction methods. To this end, we developed TransVis, an approach for specifying and analyzing transient behavior based on chatbot interactions and visualizations of the systems’ resilience. We examined the effectiveness of our approach by conducting an exploratory expert study on a prototypical implementation. The study revealed that the developed visualizations are effective for specifying and exploring transient behavior. Participants found especially helpful the feature to compare specifications with the actual behavior. However, the integration of a chatbot did not prove effective for our use cases. In conclusion, our approach is capable of supporting stakeholders in the exploration and specification of transient behavior.

The prevalence of microservice architectures and container orchestration technologies increases the complexity of assessing such systems' resilience. Chaos engineering is an emerging approach for resilience assessment by testing hypotheses after intentionally injecting faults into a distributed system and observing customer- and business-affecting metrics. As the number of potential risks within a complex system is high, the identification and prioritization of effective and efficient chaos experiments are non-trivial. In the scope of an industrial case study, this work investigates means to identify and prioritize chaos experiments by using established risk analysis techniques known from engineering safety-critical systems, namely i) Fault Tree Analysis, ii) Failure Mode and Effects Analysis, iii) and Computer Hazard and Operability Study. We conducted semi-structured interviews to elicit architectural information and resilience requirements of the case study system. The extracted knowledge was leveraged during the application of the risk analysis techniques. A subset of the identified and prioritized risks was used to create and execute chaos experiments. The risk analysis resulted in over 100 findings and revealed that the system is rather fragile as it comprises a high amount of single points of failure. The chaos experiments revealed further weaknesses for formerly unknown system behavior.