Tracing is a fundamental technique for analyzing the runtime behavior of software systems. By recording the start and end times of method executions together with contextual metadata, tracing enables detailed performance analysis, architecture reconstruction, and program comprehension. However, such instrumentation inevitably introduces runtime overhead that can distort performance measurements and increase variability. Quantifying and comparing this overhead across tracing frameworks and configurations is therefore essential for selecting suitable tools and ensuring reliable performance evaluations. The overhead of different tracing frameworks and their configuration can be measured by the MooBench microbenchmark. In this work, we extend the MooBench microbenchmark to support the established Java tracing frameworks Elastic APM Agent, inspectIT, Kieker, OpenTelemetry, Pinpoint, Scouter, and SkyWalking. By executing MooBench with these agents, we find (1) significant differences in performance overhead, whereby the industry standard implementation of OpenTelemetry is comparably slow, while the Kieker agent has the lowest overhead among the functionally correct frameworks, (2) the agents of Pinpoint and Scouter do not store all records, making their behavior not fulfill the functional requirements, and (3) that avoidable overhead of some of the frameworks is created by extensive metadata gathering and needless copying of data.

To develop software with optimal performance, even small performance changes need to be identified. Identifying performance changes is challenging since the performance of software is influenced by non-deterministic factors. Therefore, not every performance change is measurable with reasonable effort. In this work, we discuss which performance changes are measurable at code level with reasonable measurement effort and how to identify them. We present (1) an analysis of the boundaries of measuring performance changes, (2) an approach for determining a configuration for reproducible performance change identification, and (3) an evaluation comparing of how well our approach is able to identify performance changes in the application server Jetty compared with the usage of Jetty’s own performance regression benchmarks.Thereby, we find (1) that small performance differences are only measurable by fine-grained measurement workloads, (2) that performance changes caused by the change of one operation can be identified using a unit-test-sized workload definition and a suitable configuration, and (3) that using our approach identifies small performance regressions more efficiently than using Jetty’s performance regression benchmarks.

We present PeASS (Performance Analysis of Software System versions), a tool for detecting performance changes at source code level that occur between different code versions. By using PeASS, it is possible to identify performance regressions that happened in the past to fix them. PeASS measures the performance of unit tests in different source code versions. To achieve statistic rigor, measurements are repeated and analyzed using an agnostic t-test. To execute a minimal amount of tests, PeASS uses a regression test selection. We evaluate PeASS on a selection of Apache Commons projects and show that 81% of all unit test covered performance changes can be found by PeASS. A video presentation is available at https://www.youtube.com/watch?v=RORFEGSCh6Y and PeASS can be downloaded from https://github.com/DaGeRe/peass.

Performance problems are well known on architecture level. On code level their occurrences have not been systematically researched so far. Since a lot of everyday work of software developers is done on code level, methods and tools with focus on frequent performance problems are relevant. In the presented thesis, a method for systematically evaluating the occurrence and the frequency of performance problems on code level is presented and applied to repositories. The results of this empirical research will be a classification of performance problems and a quantification of their frequency. This will raise the awareness on certain problem classes for developers and will provide a basis for the development of new performance tools for preventing performance problems.