Abstract In this paper, we outline an approach for automatically generating challenging road networks for virtual testing of an automated lane‐keeping system. Based on a set of control points, we construct a parametric curve representing a road network, defining the dynamic driving task an automated lane‐keeping system‐equipped vehicle must perform. Changing control points has a global influence on the resulting road geometry. Our approach uses search to find control‐point sets that result in a challenging road, eventually forcing the vehicle to leave the intended path. We apply our approach in different search variants to evaluate their performance regarding test efficiency and the diversity of failing tests. In addition, we evaluate different genetic algorithm control parameter configurations to investigate the most influential parameters and if specific configurations can be seen asoptimal, leading to better results than others. For both studies, we consider another search‐based test method and two different random test generators as a baseline for comparison. The empirical results indicate that specific control parameter settings increase the overall performance for each search variant. While the population size is the most influential control parameter for all methods, the performance improvement when usingoptimalsettings is only significant for one method.

More automated and autonomous systems are becoming daily use that implements safety–critical functions, e.g., autonomous driving or mobile robots. Testing such systems people depend on is challenging because some environmental interactions may not be expected during development but occur when those systems are in operation. Deciding when to stop testing or answering how to ensure sufficient testing is challenging and very expensive. For generating critical environmental interactions, i.e., critical scenarios, we present and compare two testing solutions focusing on generating critical scenarios utilizing combinatorial and search-based testing, respectively. For combinatorial testing, we suggest using ontologies that describe the environment of an autonomous or highly automated system. For search-based testing, we rely on genetic algorithms. We experimentally compared the two testing approaches using two implementations of an industrial emergency braking function and random testing as the baseline. Furthermore, we compared the approaches qualitatively using several categories. From the experiments, we see that the combinatorial testing approach can find all different types of faults listed in Table 5 considering a combinatorial strength of 3. This is not the case for search-based and random testing in all experiments. Combinatorial testing comes with the highest combinatorial coverage. However, all approaches can reveal faulty behavior utilizing appropriate environmental models. We present the results of an in-depth comparison of combinatorial and search-based testing. The be as fair as possible, the comparison relied on the same environmental model and other parameters like the number of generated test cases. The results show that combinatorial testing comes with the highest coverage and can find all different kinds of failures summarized in Table 5 providing a certain strength. Meanwhile, search-based testing is also capable of finding different failures depending on the coverage it can reach. Both approaches seem complementary and of use for the application domain of autonomous and automated driving functions.

In this paper, we outline the use of a genetic algorithm for test parameter optimization in the context of autonomous and automated driving. Our approach iteratively optimizes test parameters to aim at obtaining critical scenarios that form the basis for virtual verification and validation of Advanced Driver Assistant Systems (ADAS). We consider a test scenario to be critical if the underlying parameter set causes a malfunction of the system equipped with the ADAS function (i.e., near crash or crash of the vehicle). For evaluating the effectiveness of our approach, we set up an automated simulation framework, where we simulated the Euro NCAP car-to-car rear scenario. To assess the criticality of each test scenario we rely on time-to-collision (TTC), which is a well-known and often used time-based safety indicator for recognizing rear-end conflicts. Our genetic algorithm approach showed a higher chance to generate a critical scenario, compared to a random selection of test parameters.