A key goal of performance testing is the detection of performance degradations (i.e., regressions) compared to previous releases. Prior research has proposed the automation of such analysis through the mining of historical performance data (e.g., CPU and memory usage) from prior test runs. Nevertheless, such research has had limited adoption in practice. Working with a large industrial performance testing lab, we noted that a major hurdle in the adoption of prior work (including our own work) is the incorrect assumption that prior tests are always executed in the same environment (i.e., labs). All too often, tests are performed in heterogenous environments with each test being run in a possibly different lab with different hardware and software configurations. To make automated performance regression analysis techniques work in industry, we propose to model the global expected behaviour of a system as an ensemble (combination) of individual models, one for each successful previous test run (and hence configuration). The ensemble of models of prior test runs are used to flag performance deviations (e.g., CPU counters showing higher usage) in new tests. The deviations are then aggregated using simple voting or more advanced weighting to determine whether the counters really deviate from the expected behaviour or whether it was simply due to an environment-specific variation. Case studies on two open-source systems and a very large scale industrial application show that our weighting approach outperforms a state-of-the-art environment-agnostic approach. Feedback from practitioners who used our approach over a 4 year period (across several major versions) has been very positive.

Abstract A business process, such as the process followed when ordering a book, describes the order of executing tasks (e.g., check inventory, verify credit card, and ship book). Business applications implement the business processes for the daily operations of an organization. Organizations must continuously modify their business applications to accommodate changes to business processes. However, business applications are often designed and developed without referring to the documented definitions of business processes. Modifying business applications is a time‐consuming and error‐prone task. To correctly perform this task, developers require an in‐depth understanding of multi‐tiered applications and the definitions of the business processes that they implement. In this paper, we present an approach that automatically recovers business process definitions from multi‐tiered business applications. Given the starting UI screen of a particular business process, the approach recovers the process definition by tracing the flow of control throughout the different tiers of the business application. We demonstrate the effectiveness of our approach through a case study using 15 business applications from three large open‐source projects. Our case study demonstrates that our approach can recover business process definitions from the implementation with high precision and recall. Copyright © 2009 John Wiley & Sons, Ltd.

A business process is composed of a set of interrelated tasks which are joined together by control flow elements. E-commerce systems implement business processes to automate the daily operations of an organization. Organizations must continuously modify their e-commerce systems to accommodate changes to business processes. However, modifying e-commerce systems is a time consuming and error prone task. To correctly perform this task, developers require an in-depth understanding of multi-tiered e-commerce systems and the business processes that they implement. In this paper, we present a business process explorer tool which automatically recovers business processes from three tier e-commerce systems. Developers can explore the recovered business processes and browse the corresponding source code. We integrate our tool with IBM WebSphere Business Modeler (WBM), a leading commercial tool for business process management and modeling. Business analysts could then visualize and analyze the recovered processes using WBM. The business process explorer eases the co-evolution of business processes and their e-commerce system implementation.