Metamorphic testing is a technique that has shown great potential to alleviate the test oracle problem by exploiting the relations among the inputs and outputs of different executions of a system. However, this approach requires multiple test executions. In applications like Cyber-Physical Systems (CPSs), where the test executions can be very expensive in terms of time and resources needed, this can supose a problem. Therefore, it is paramount to optimize the test suite to reduce the costs of verifying the system. Test case selection is an optimization technique which accomplishes this by selecting a subset of test cases while aiming to preserve the effectiveness of the original test suite as much as possible. While there are many approaches for test case selection in the existing literature, none of them has been proposed for the metamorphic test case selection problem, where each metamorphic test case consists of a source and, at least, a follow-up test case pair. In this work, we present an evolutionary multi-objective approach for the metamorphic test case selection problem, adapting existing multi-objective test selection techniques and proposing new evolutionary operators and objective functions. Further-more, we evaluate our approach with a set of metamorphic tests developed for an industrial case study from the elevation domain. The results suggest that our approach outperforms both Random Search and the same metaheuristic algorithm without the new evolutionary operators we propose.

Atomicity is a correctness criterion to reason about isolated code regions in a multithreaded program when they are executed concurrently. However, dynamic instances of these code regions, called transactions , may fail to behave atomically, resulting in transactional atomicity violations. Existing dynamic online atomicity checkers incur either false positives or false negatives in detecting transactions experiencing transactional atomicity violations. This article proposes RegionTrack. RegionTrack tracks cross-thread dependences at the event, dynamic subregion, and transaction levels. It maintains both dynamic subregions within selected transactions and transactional happens-before relations through its novel timestamp propagation approach. We prove that RegionTrack is sound and complete in detecting both transactional atomicity violations and non-serializable traces. To the best of our knowledge, it is the first online technique that precisely captures the transitively closed set of happens-before relations over all conflicting events with respect to every running transaction for the above two kinds of issues. We have evaluated RegionTrack on 19 subjects of the DaCapo and the Java Grande Forum benchmarks. The empirical results confirm that RegionTrack precisely detected all those transactions which experienced transactional atomicity violations and identified all non-serializable traces. The overall results also show that RegionTrack incurred 1.10x and 1.08x lower memory and runtime overheads than Velodrome and 2.10x and 1.21x lower than Aerodrome, respectively. Moreover, it incurred 2.89x lower memory overhead than DoubleChecker. On average, Velodrome detected about 55% fewer violations than RegionTrack, which in turn reported about 3%–70% fewer violations than DoubleChecker.

Existing deterministic replay techniques propose strategies which attempt to reduce record log sizes and achieve successful replay. However, these techniques still generate large logs and achieve replay only under certain conditions. We propose a solution based on the division of the sequence of events of each thread into sequential blocks called transactions. Our insight is that there are usually few to no atomicity violations among transactions reported during a program execution. We present TPLAY, a novel deterministic replay technique which records thread access interleavings on shared memory locations at the transactional level. TPLAY also generates an artificial pair of interleavings when an atomicity violation is reported on a transaction. We present an experiment using the Splash2x extension of the PARSEC benchmark suite. Experimental results indicate that TPLAY experiences a 13-fold improvement in record log sizes and achieves a higher replay probability in comparison to existing work.

Deterministic replay presents challenges and often results in high memory and runtime overheads. Previous studies deterministically reproduce program outputs often only after several replay iterations or may produce a non-deterministic sequence of output to external sources. In this paper, we propose AggrePlay, a deterministic replay technique which is based on recording read-write interleavings leveraging thread-local determinism and summarized read values. During the record phase, AggrePlay records a read count vector clock for each thread on each memory location. Each thread checks the logged vector clock against the current read count in the replay phase before a write event. We present an experiment and analyze the results using the Splash2x benchmark suite as well as two real-world applications. The experimental results show that on average, AggrePlay experiences a better reduction in compressed log size, and 56% better runtime slowdown during the record phase, as well as a 41.58% higher probability in the replay phase than existing work.

Many dynamic hybrid race detectors aim at detecting violations of the lockset discipline in execution traces of multithreaded programs. They are designed to abstract memory accesses appearing in traces as contexts. Nonetheless, they keep these contexts in different extents and partition the sets of contexts into equivalent classes of different granularity. In our case study, we compare three detectors using the PARSEC benchmark suite to examine the impact of using unrestricted strategy or restricted strategy for keeping these contexts in sequence on detection effectiveness, and the impact of partitioning context sequences by different granularities on scalability in time cost. The case study results indicate that using restricted context sequences sufficed to detect very high proportions of locking discipline violations detectable by using unrestricted context sequences, and the partitioning of context sets into finer equivalent classes significantly lowers the scalability in time cost with increasing number of threads to handle the same input workload.