Regression testing is a testing activity that aims to ensure that existing functionality is preserved when introducing changes. The goal of regression test selection (RTS) is to reduce the cost of regression testing by only re-executing tests that are affected by changes. Lately, research on RTS has focused on the languages Java and C++. Despite Rust being an increasingly relevant systems programming language, there are no RTS tools available for this language so far. In this paper, we present and evaluate RUSTYRTS, the first RTS technique and tool for Rust. It provides both module-level and function-level RTS. Its function-level variants can rely on either static or dynamic code analysis to select affected tests. We evaluate RUSTYRTS in an empirical study in terms of safety, precision, and effectiveness. When applied to changes resulting from mutation testing on 9 open-source projects, RUSTYRTS selected 99.99%, 97.87% and 99.97% of all tests that failed in consequence of a modification in code using a module-level, static and dynamic RTS approach respectively. For cases of unsafe behavior, i.e., tests that have not been selected although failing due to such changes, we find plausible explanations. By applying RUSTYRTS to changes from the Git history of 13 repositories on GitHub, we effectively reduce the average end-to-end (e2e) testing time on the majority of projects. Our results show that the two function-level approaches outperform the coarser module-level one, especially on longer-running test suites. On average, the$e$2$e$testing time has been reduced to 67.80%, 62.52% and 52.79% of retest-all by module-level, static and dynamic RUSTYRTS respectively. Lastly, we provide a novel solution for dynamic dispatch and compile-time function evaluation, two contexts that impose a special challenge on approaches to RTS.

In automotive software engineering, system-level regression testing is crucial to ensure proper integration of often- times safety-critical components. Due to the inherent complexity of such systems and components, testing is commonly performed manually and in a black-box manner, which is particularly costly and leads to slow feedback cycles between testers and developers. Regression Test Prioritization (RTP) aims to reduce feedback time by ordering tests to reveal faults earlier during the testing process. However, most prior RTP research does not incorporate varying fault severity, which must be taken into account when evaluating and designing appropriate RTP approaches for safety-critical automotive software systems. In this work, we present a case study at our industry partner MAN, a leading international provider of commercial vehicles. We design and instantiate a domain-specific, severity-aware RTP assessment model and comparatively assess state-of-the-art RTP approaches. Our results indicate that simple and partly well- known heuristics based on test history and test costs have the best cost-effectiveness, achieving between 85% and 90% of the maximum possible feedback time reduction. On the other hand, search-based and machine-learning-based RTP approaches do not perform better, especially if available test history is sparse.

Efficiently collecting per-test execution traces is a common prerequisite of dynamic regression test optimization techniques. However, as these test traces are typically recorded through language-specific code instrumentation, non-code artifacts and multi-language source code are usually not included. In contrast, more complete test traces can be obtained by instrumenting operating system calls and thereby tracing all accessed files during a test's execution. Yet, existing test optimization techniques that use syscall tracing are impractical as they either modify the Linux kernel or operate in user space, thus raising transferability, performance, and security concerns. Recent advances in operating system development provide versatile, lightweight, and safe kernel instrumentation frameworks: They allow to trace syscalls by instrumenting probes in the operating system kernel. Probe-based Syscall Tracing (ProST), our novel technique, harnesses this potential to collect file-level test traces that go beyond language boundaries and consider non-code artifacts. To evaluate ProST's efficiency and the completeness of obtained test traces, we perform an empirical study on 25 multi-language open-source software projects and compare our approach to existing language-specific instrumentation techniques. Our results show that most studied projects use source files from multiple languages (22/25) or non-code artifacts during testing (22/25) that are missed by language-specific techniques. With the low execution time overhead of 4.6% compared to non-instrumented test execution, ProST is more efficient than language-specific instrumentation. Furthermore, it collects on average 89% more files on top of those collected by language-specific techniques. Consequently, ProST paves the way for efficiently extracting valuable information through dynamic analysis to better understand and optimize testing in multi-language software systems.