During software development, developers often make numerous modifications to the software to address existing issues or implement new features. However, certain changes may inadvertently have a detrimental impact on the overall system performance. To ensure that the performance of new software re-leases does not degrade (i.e., absence of performance regressions), existing practices rely on system-level performance testing, such as load testing, or component-level performance testing, such as microbenchmarking, to detect performance regressions. However, performance testing for the entire system is often expensive and time-consuming, posing challenges to adapting to the rapid release cycles common in modern DevOps practices. In addition, system-level performance testing cannot be conducted until the system is fully built and deployed. On the other hand, component-level testing focuses on isolated components, neglecting overall system performance and the impact of system workloads. In this paper, we propose a novel approach to early detection of performance regressions by bridging the local performance data generated by component-level testing and the system-level architectural models. Our approach uses local performance data to identify deviations at the component level, and then propagate these deviations to the architectural model. We then use the architectural model to predict regressions in the performance of the overall system. In an evaluation of our approach on two representative open-source benchmark systems, we show that it can effectively detect end-to-end system performance regressions from local performance deviations with different intensities and under various system workloads. More importantly, our approach can detect regressions as early as in the development phase, in contrast to existing approaches that require the system to be fully built and deployed. Our approach is lightweight and can complement traditional system performance testing when testing resources are scarce.

Software architecture optimization aims to enhance non-functional attributes like performance and reliability while meeting functional requirements. Multi-objective optimization employs metaheuristic search techniques, such as genetic algorithms, to explore feasible architectural changes and propose alternatives to designers. However, this resource-intensive process may not always align with practical constraints. This study investigates the impact of designer interactions on multi-objective software architecture optimization. Designers can intervene at intermediate points in the fully automated optimization process, making choices that guide exploration towards more desirable solutions. Through several controlled experiments as well as an initial user study (14 subjects), we compare this interactive approach with a fully automated optimization process, which serves as a baseline. The findings demonstrate that designer interactions lead to a more focused solution space, resulting in improved architectural quality. By directing the search toward regions of interest, the interaction uncovers architectures that remain unexplored in the fully automated process. In the user study, participants found that our interactive approach provides a better trade-off between sufficient exploration of the solution space and the required computation time.

Deprecation is a way to inform clients using an application programming interface (API) that the usage of this API is discouraged. Tool support and research for deprecation in local APIs are well established. However, nowadays web APIs are more commonly used, e.g., using the REST architectural style. However, the techniques to detect and handle the usage of deprecated local APIs cannot be directly applied to web APIs. Previous approaches for detecting deprecated web APIs focus on static analysis of client code by detecting calls to web APIs and, subsequently, an investigation of associated API specifications. These approaches currently have two essential limitations: (i) The target of an API call can often not be determined statically. (ii) Deprecation in API specifications is not the only way to signal deprecation for web APIs. We introduce a dynamic approach using tracing to detect calls to web APIs. Subsequently, we check the called APIs for deprecation using an API specification, response meta-data, or a knowledge base. This approach addresses both limitations of the detection with static analysis. We implement the approach and evaluate it on three projects, including client-server calls as well as a microservice benchmark system. The empirical evaluation yields a precision of 1.00 and a recall of 0.95. The false negatives can be attributed to a shortcoming in the automatic instrumentation provided by OpenTelemetry observability framework.

Many dynamic routing architectural patterns are available, including distributed routing, e.g., using the sidecar pattern, or centralized routing, e.g., using event stores or service buses. Different Quality-of-Service (QoS) factors influence routing schemas and technology selection, such as performance, reliability, scalability, and control properties offered by the patterns. An analytical model can formalize the QoS factors and facilitate the architectural decision-making when changing the routing scheme, i.e., to more distributed or centralized routing. So far, the impact of these architectural patterns on performability, i.e., the overall performance of a system with impeded reliability, has not been extensively studied. This is important because deciding to increase performance, e.g., by parallel processing of requests, may lead to decreased reliability because of the added points of a crash. We propose an analytical performability model during component crashes. For the empirical validation of our proposed model, we ran an extensive experiment of 2412 hours of runtime on a private cloud infrastructure and Google Cloud Platform. The low prediction error of 1.75 % indicates the high accuracy of our performability model. These results provide important insights when making architectural decisions regarding service- and cloud-based dynamic routing.

Context. Architecture maintainability smells indicate software quality problems and suggest necessary architecture refactorings. However, there are numerous names for a single smell, including synonyms and subclasses, and the number keeps increasing. This situation leads to confusion and difficulty in identifying the underlying problems. Objective. To address this challenge, we want to organize the existing smells. Method. We collect smell names through a systematic literature review, followed by a four-step data analysis leading to a taxonomy of maintainability smells, comprising smell characteristics, categories, detection methods, and causes. Results. We found 549 architecture smell names in 189 references, from which we identified 318 maintainability smell names. We derived the distinguishing characteristics and reduced the list to 19 essential maintainability smells. Conclusions. Our taxonomy provides an overview of essential maintainability smells, making future research and discussions more manageable. Moreover, the identified synonyms make existing research comparable, and the characteristics, causes, and detection methods provide a framework for classifying future smells.

Test methods that can keep up with the ongoing increase in complexity of semiconductor products and their underlying technologies are an essential prerequisite for maintaining quality and safety of our daily lives and for continued success of our economies and societies. There is a huge potential how test methods can benefit from recent breakthroughs in domains such as artificial intelligence, data analytics, virtual/augmented reality, and security. The Graduate School on “Intelligent Methods for Semiconductor Test and Reliability” (GS-IMTR) at the University of Stuttgart is a large-scale, radically interdisciplinary effort to address the scientific-technological challenges in this domain. It is funded by Advantest, one of the world leaders in automatic test equipment. In this paper, we describe the overall philosophy of the Graduate School and the specific scientific questions targeted by its ten projects.

Dynamic reconfiguration is commonly used to accommodate the dynamic behavior of today’s applications. As cloud-based systems become increasingly complex, it is hard and cost-ineffective to manage them manually. Dynamic routers, such as API Gateways or Message Brokers, in combination with auto-scalers can adapt the system to the resource demands, e.g., when a sudden load spike for a specific part of the system is observed. Without taking costs of cloud resources into account, this reconfiguration can lead to significant increase of charges. We propose a self-adaptive and cost-aware dynamic routing architecture called Adaptive Dynamic Routers. The novel architecture performs a multi-criteria optimization analysis to automatically reconfigure the routers and the services of a cloud-based system considering the costs of reconfiguration. This multidimensional auto-scaling of resources takes incoming load as an input, and uses queuing theory to find an optimal reconfiguration solution. We systematically evaluated our architecture with an extensive number of evaluation cases (9600). On average over cases where an overload is predicted, our approach reduces the overload rate by 46.7% and 61.8% for routers and services, respectively.

Increased resilience compared to monolithic architectures is both one of the key promises of microservice-based architectures and a big challenge, e.g., due to the systems’ distributed nature. Resilience assessment through simulation requires fewer resources than the measurement-based techniques used in practice. However, there is no existing simulation approach that is suitable for a holistic resilience assessment of microservices comprised of (i) representative fault injections, (ii) common resilience mechanisms, and (iii) time-varying workloads. This paper presents MiSim — an extensible simulator for resilience assessment of microservice-based architectures. It overcomes the stated limitations of related work. MiSim fits resilience engineering practices by supporting scenario-based experiments and requiring only lightweight input models. We demonstrate how MiSim simulates (1) common resilience mechanisms — i.e., circuit breaker, connection limiter, retry, load balancer, and autoscaler — and (2) fault injections — i.e., instance/service killing and latency injections. In addition, we use TeaStore, a reference microservice-based architecture, aiming to reproduce scaling behavior from an experiment by using simulation. Our results show that MiSim allows for quantitative insights into microservice-based systems’ complex transient behavior by providing up to 25 metrics.

Context:Various patterns of dynamic routing architectures are used in service- and cloud-based environments, including sidecar-based routing, routing through a central entity such as an event store, or architectures with multiple dynamic routers.Objective:Choosing the wrong architecture may severely impact the reliability or performance of a software system. This article’s objective is to provide models and empirical evidence to precisely estimate the reliability and performance impacts.Method:We propose an analytical model of request loss for reliability modeling. We studied the accuracy of this model’s predictions empirically and calculated the error rate in 200 experiment runs, during which we measured the round-trip time performance and created a performance model based on multiple regression analysis. Finally, we systematically analyzed the reliability and performance impacts and trade-offs.Results and Conclusions:The comparison of the empirical data to the reliability model’s predictions shows a low enough and converging error rate for using the model during system architecting. The predictions of the performance model show that distributed approaches for dynamic data routing have a better performance compared to centralized solutions. Our results provide important new insights on dynamic routing architecture decisions to precisely estimate the trade-off between system reliability and performance.

Today, most Continuous Integration and Delivery (CI/CD) solutions use infrastructure as code to describe the pipeline-based build and deployment process. Each solution uses its own format to describe the CI/CD pipeline, which hinders the interoperability and the analysis of CI/CD pipelines. In this paper, we propose a model-driven DSL-based framework for CI/CD pipeline definition and analysis. It comprises (i) the analysis of the meta-model of the Jenkins pipeline definition language, (ii) the StalkCD domain-specific language providing a base for interoperability and transformation between different formats, and (iii) an extensible set of transformations between tool-specific CI/CD definitions and analysis tools. We demonstrate the specific support for Jenkins as a CI/CD tool and BPMN for exploiting analyses from the workflow domain and visualizing the results. We evaluate the DSL and the transformations empirically based on more than 1,000 publicly available Jenkinsfiles. The evaluation shows that our framework supports 70% of these files without information loss.

Many different dynamic routing architectures are available, including sidecar-based routing, routing through a central entity such as an event store or gateway, or architectures with multiple routers. These architectures are currently based on vastly different implementation concepts, such as API Gateways, Message Brokers, or Service Proxies. We propose a new approach that abstracts all these architecture patterns using one Adaptive Dynamic Routers architecture. We hypothesize that a dynamic self-adaptation of the routing architecture is beneficial over any fixed architecture selections for reliability and performance trade-offs. That is, if encountered with traffic and load changes, our approach dynamically self-adapts between more central or distributed routing to optimize system reliability and performance. We evaluate our approach by analyzing our previously-measured data during an experiment of 1200 hours of runtime. Our extensive systematic evaluation with 1089 cases confirms that our hypothesis holds and our approach is beneficial in terms of reliability and performance. Moreover, we empirically validate our results on Google Cloud Platform infrastructure.

In a microservice system, runtime changes such as failures, deployments, or self-adaptation can trigger the system to transition from one steady state to another, i.e., exhibiting transient behavior. To assess a system’s quality, it is imperative that this transient behavior is specified in non-functional requirements and that stakeholders can analyze whether these requirements are met. Yet, there is little support for either specifying transient behavior as a non-functional requirement or analyzing how such a requirement is met in production. We aim to make these two tasks more accessible by utilizing novel human-computer interaction methods. To this end, we developed TransVis, an approach for specifying and analyzing transient behavior based on chatbot interactions and visualizations of the systems’ resilience. We examined the effectiveness of our approach by conducting an exploratory expert study on a prototypical implementation. The study revealed that the developed visualizations are effective for specifying and exploring transient behavior. Participants found especially helpful the feature to compare specifications with the actual behavior. However, the integration of a chatbot did not prove effective for our use cases. In conclusion, our approach is capable of supporting stakeholders in the exploration and specification of transient behavior.

Context. Software Performance Antipatterns (SPAs) research has focused on algorithms for the characterization, detection, and solution of antipatterns. However, existing algorithms are based on the analysis of runtime behavior to detect trends on several monitored variables (e.g., response time, CPU utilization, and number of threads) using pre-defined thresholds. Objective. In this paper, we introduce a new approach for SPA characterization and detection designed to support continuous integration/delivery/deployment (CI/CDD) pipelines, with the goal of addressing the lack of computationally efficient algorithms.

Load tests evaluate software quality attributes, such as performance and reliability, by e.g., emulating user behavior that is representative of the production workload. Existing approaches extract workload models from recorded user requests. However, a single workload model cannot reflect the complex and evolving workload of today's applications, or take into account workload-influencing contexts, such as special offers, incidents, or weather conditions. In this paper, we propose an integrated framework for generating load tests tailored to the context of interest, which a user can describe in a language we provide. The framework applies multivariate time series forecasting for extracting a context-tailored load test from an initial workload model, which is incrementally learned by clustering user sessions recorded in production and enriched with relevant context information.

The prevalence of microservice architectures and container orchestration technologies increases the complexity of assessing such systems' resilience. Chaos engineering is an emerging approach for resilience assessment by testing hypotheses after intentionally injecting faults into a distributed system and observing customer- and business-affecting metrics. As the number of potential risks within a complex system is high, the identification and prioritization of effective and efficient chaos experiments are non-trivial. In the scope of an industrial case study, this work investigates means to identify and prioritize chaos experiments by using established risk analysis techniques known from engineering safety-critical systems, namely i) Fault Tree Analysis, ii) Failure Mode and Effects Analysis, iii) and Computer Hazard and Operability Study. We conducted semi-structured interviews to elicit architectural information and resilience requirements of the case study system. The extracted knowledge was leveraged during the application of the risk analysis techniques. A subset of the identified and prioritized risks was used to create and execute chaos experiments. The risk analysis resulted in over 100 findings and revealed that the system is rather fragile as it comprises a high amount of single points of failure. The chaos experiments revealed further weaknesses for formerly unknown system behavior.

In recent years, there has been a shift in software development towards microservice-based architectures, which consist of small services that focus on one particular functionality. Many companies are migrating their applications to such architectures to reap the benefits of microservices, such as increased flexibility, scalability and a smaller granularity of the offered functionality by a service. On the one hand, the benefits of microservices for functional testing are often praised, as the focus on one functionality and their smaller granularity allow for more targeted and more convenient testing. On the other hand, using microservices has their consequences (both positive and negative) on other types of testing, such as performance testing. Performance testing is traditionally done by establishing the baseline performance of a software version, which is then used to compare the performance testing results of later software versions. However, as we show in this paper, establishing such a baseline performance is challenging in microservice applications. In this paper, we discuss the benefits and challenges of microservices from a performance tester's point of view. Through a series of experiments on the TeaStore application, we demonstrate how microservices affect the performance testing process, and we demonstrate that it is not straightforward to achieve reliable performance testing results for a microservice application.

Even though load testing is an established technique to assess load-related quality properties of software systems, it is applied only seldom and with questionable results. Indeed, configuring, executing, and interpreting results of a load test require high effort and expertise. Since chatbots have shown promising results for interactively supporting complex tasks in various domains (including software engineering), we hypothesize that chatbots can provide developers suitable support for load testing. In this paper, we present PerformoBot, our chatbot for configuring and running load tests. In a natural language conversation, PerformoBot guides developers through the process of properly specifying the parameters of a load test, which is then automatically executed by PerformoBot using a state-of-the-art load testing tool. After the execution, PerformoBot provides developers a report that answers the respective concern. We report on results of a user study that involved 47 participants, in which we assessed our tool's acceptance and effectiveness. We found that participants in the study, particularly those with a lower level of expertise in performance engineering, had a mostly positive view of PerformoBot.

Microservices have emerged as an architectural style for developing distributed applications. Assessing the performance of architecture deployment configurations — e.g., with respect to deployment alternatives — is challenging and must be aligned with the system usage in the production environment. In this paper, we introduce an approach for using operational profiles to generate load tests to automatically assess scalability pass/fail criteria of microservice configuration alternatives. The approach provides a Domain-based metric for each alternative that can, for instance, be applied to make informed decisions about the selection of alternatives and to conduct production monitoring regarding performance-related system properties, e.g., anomaly detection. We have evaluated our approach using extensive experiments in a large bare metal host environment and a virtualized environment. First, the data presented in this paper supports the need to carefully evaluate the impact of increasing the level of computing resources on performance. Specifically, for the experiments presented in this paper, we observed that the evaluated Domain-based metric is a non-increasing function of the number of CPU resources for one of the environments under study. In a subsequent series of experiments, we investigate the application of the approach to assess the impact of security attacks on the performance of architecture deployment configurations.

Summary Directly affecting the user experience, performance is a crucial aspect of today's software applications. Representative load testing allows to effectively test and preserve the performance before delivery by mimicking the actually expected workload. In the literature, various approaches have been proposed for extracting representative load tests from recorded user sessions. However, these approaches require manual parameterization for specifying input data and adjusting static properties such as a request's domain name. This manual effort accumulates when load tests need to be updated due to changing production workloads and APIs. In this paper, we address the reduction of the maintenance effort for representative load testing. We introduce input data and properties annotations (IDPAs) that store manual parameterizations and can be evolved automatically. Experts only have to parameterize extracted load tests initially. For dealing with API changes, we develop approaches to evolve IDPAs for the types of changes described in the literature. We evaluated our approach in two experimental studies, by deriving effort estimation models, and in an industrial case study including four different software projects. Our evaluation shows that IDPAs can parameterize generated load tests for restoring the representativeness, especially for applications with workloads dominated by request orders and rates. The maintenance effort can be reduced from a quadratic cumulative effort over time to a linear cumulative effort for a typical mix of API changes. Furthermore, we were able to express all parameterizations required by the industrial projects using the IDPA but also had to integrate extensions using the provided extension mechanisms.

Load tests are commonly used to assess the performance of an application system. A representative load test uses workload characteristics according to the user behavior in production. Session-based systems have special workload characteristics as the system is used as sequences of inter-related requests. Approaches exist to automatically extract session-based workload models from production request logs. However, they focus on system-level testing, which is in stark contrast with modern development practices, where one development team is in charge of developing, testing, and deploying a single microservice. Hence, representative session-based workload models for testing single microservices and their integration are desirable. To deal with these issues, we propose a concept for tailoring a representative load test workload to target only certain services, instead of targeting the whole system. Our goal is to transform the workload for one or more specified service(s) from the system-level workload collected in production. Using this approach, only a subset of the application's microservices is deployed for a load test, specifically the targeted services and the services they depend on. We propose two algorithms. The log-based algorithm deals with extracting the workload for a specific service from collected production traces. The model-based algorithm performs the workload tailoring on the level of the workload model. In an experiment series with a representative microservice application, we compare both algorithms with system-level and request-based workoad models. The results show that when load testing a set of services, the tailored workload models outperform untailored workload models in terms of test duration and the capacity of the test infrastructure, and outperform request-based workload models in terms of representativeness.

DevOps is a modern software engineering paradigm that is gaining widespread adoption in industry. The goal of DevOps is to bring software changes into production with a high frequency and fast feedback cycles. This conflicts with software quality assurance activities, particularly with respect to performance. For instance, performance evaluation activities --- such as load testing --- require a considerable amount of time to get statistically significant results.

Load testing is widely considered a meaningful technique for performance quality assurance. However, empirical studies reveal that in practice, load testing is not applied systematically, due to the sound expert knowledge required to specify, implement, and execute load tests.

Summary Pure spectrum‐based fault localization (SBFL) is a well‐studied statistical debugging technique that only takes a set of test cases (some failing and some passing) and their code coverage as input and produces a ranked list of suspicious program elements to help the developer identify the location of a bug that causes a failed test case. Studies show that pure SBFL techniques produce good ranked lists for small programs. However, our previous study based on the iBugs benchmark that uses the A spect J repository shows that, for realistic programs, the accuracy of the ranked list is not suitable for human developers. In this paper, we confirm this based on a combined empirical evaluation with the iBugs and the D efects4 J benchmark. Our experiments show that, on average, at most ∼40 % , ∼80 % , and ∼90 % of the bugs can be localized reliably within the first 10, 100, and 1000 ranked lines, respectively, in the D efects4 J benchmark. To reliably localize 90 % of the bugs with the best performing SBFL metric D ∗ , ∼450 lines have to be inspected by the developer. For human developers, this remains unsuitable, although the results improve compared with the results for the A spect J benchmark. Based on this study, we can clearly see the need to go beyond pure SBFL and take other information, such as information from the bug report or from version history of the code lines, into consideration.

Complex software systems experience failures at runtime even though a lot of effort is put into the development and operation. Reactive approaches detect these failures after they have occurred and already caused serious consequences. In order to execute proactive actions, the goal of online failure prediction is to detect these failures in advance by monitoring the quality of service or the system events. Current failure prediction approaches look at the system or individual components as a monolith without considering the architecture of the system. They disregard the fact that the failure in one component can propagate through the system and cause problems in other components. In this paper, we propose a hierarchical online failure prediction approach, called Hora, which combines component failure predictors with architectural knowledge. The failure propagation is modeled using Bayesian networks which incorporate both prediction results and component dependencies extracted from the architectural models. Our approach is evaluated using Netflix’s server-side distributed RSS reader application to predict failures caused by three representative types of faults: memory leak, system overload, and sudden node crash. We compare Hora to a monolithic approach and the results show that our approach can improve the area under the ROC curve by 9.9%.

The specification of workloads is required in order to evaluate performance characteristics of application systems using load testing and model-based performance prediction. Defining workload specifications that represent the real workload as accurately as possible is one of the biggest challenges in both areas. To overcome this challenge, this paper presents an approach that aims to automate the extraction and transformation of workload specifications for load testing and model-based performance prediction of session-based application systems. The approach (WESSBAS) comprises three main components. First, a system- and tool-agnostic domain-specific language (DSL) allows the layered modeling of workload specifications of session-based systems. Second, instances of this DSL are automatically extracted from recorded session logs of production systems. Third, these instances are transformed into executable workload specifications of load generation tools and model-based performance evaluation tools. We present transformations to the common load testing tool Apache JMeter and to the Palladio Component Model. Our approach is evaluated using the industry-standard benchmark SPECjEnterprise2010 and the World Cup 1998 access logs. Workload-specific characteristics (e.g., session lengths and arrival rates) and performance characteristics (e.g., response times and CPU utilizations) show that the extracted workloads match the measured workloads with high accuracy.

Nowadays, cloud providers offer a broad catalog of services for migrating and distributing applications in the cloud. However, the existence of a wide spectrum of cloud services has become a challenge for deciding where to host applications, as these vary in performance and cost. This work addresses such a challenge, and provides a utility-based decision support model and method that evaluates and ranks during design time potential application distributions spanned among heterogeneous cloud services. The utility model is evaluated using the MediaWiki (Wikipedia) application, and shows an improved efficiency for selecting cloud services in comparison to other decision making approaches.

No abstract available.

In the past, spectrum-based fault localization (SBFL) techniques have been developed to pinpoint a fault location in a program given a set of failing and successful test executions. Most of the algorithms use similarity coefficients and have only been evaluated on established but small benchmark programs from the Software-artifact Infrastructure Repository (SIR). In this paper, we evaluate the feasibility of applying 33 state-of-the-art SBFL techniques to a large real-world project, namely ASPECTJ. From an initial set of 350 faulty version from the iBugs repository of ASPECTJ we manually classified 88 bugs where SBFL techniques are suitable. Notably, only 11 bugs of these bugs can be found after examining the 1000 most suspicious lines and on average 250 source code files need to be inspected per bug. Based on these results, the study showcases the limitations of current SBFL techniques on a larger program.

As the importance of application performance grows in modern enterprise systems, many organizations employ application performance management (APM) tools to help them deal with potential performance problems during production. In addition to monitoring capabilities, these tools provide problem detection and alerting. In large enterprise systems these tools can report a very large number of performance problems. They have to be dealt with individually, in a time-consuming and error-prone manual process, even though many of them have a common root cause. In this vision paper, we propose using automatic categorization for dealing with large numbers of performance problems reported by APM tools. This leads to the aggregation of reported problems, reducing the work required for resolving them. Additionally, our approach opens the possibility of extending the analysis approaches to use this information for a more efficient diagnosis of performance problems.

The performance of application systems has a direct impact on business metrics. For example, companies lose customers and revenue in case of poor performance such as high response times. Application performance management (APM) aims to provide the required processes and tools to have a continuous and up-to-date picture of relevant performance measures during operations, as well as to support the detection and resolution of performance-related incidents.

Over the past decades, various methods, techniques, and tools for modeling and evaluating performance properties of software systems have been proposed covering the entire software life cycle. However, the application of performance engineering approaches to solve a given user concern is still rather challenging and requires expert knowledge and experience. There are no recipes on how to select, configure, and execute suitable methods, tools, and techniques allowing to address the user concerns. In this paper, we describe our vision of Declarative Performance Engineering (DPE), which aims to decouple the description of the user concerns to be solved (performance questions and goals) from the task of selecting and applying a specific solution approach. The strict separation of "what" versus "how" enables the development of different techniques and algorithms to automatically select and apply a suitable approach for a given scenario. The goal is to hide complexity from the user by allowing users to express their concerns and goals without requiring any knowledge about performance engineering techniques. Towards realizing the DPE vision, we discuss the different requirements and propose a reference architecture for implementing and integrating respective methods, algorithms, and tooling.

Kieker is an extensible framework for monitoring and analyzing the runtime behavior of concurrent or distributed software systems. It provides measurement probes for application performance monitoring and control-flow tracing. Analysis plugins extract and visualize architectural models, augmented by quantitative observations. Configurable readers and writers allow Kieker to be used for online and offline analysis. This paper reviews the Kieker framework focusing on its features, its provided extension points for custom components, as well the imposed monitoring overhead.