Context: Simulation-based testing is a cost-efficient alternative to field testing for Autonomous Vehicles (AVs), but generating safety-critical test cases is challenging due to the vast search space. Prior work has studied static (road features) and dynamic (AV behavior) features of test scenarios separately, but their inter-dependencies are underexplored. Objective: In this paper, we describe an empirical to analyze how static and dynamic features of test scenarios, and their inter-dependencies, influence AV test scenario outcomes. Method: This study proposes an integrated approach using Instance Space Analysis (ISA) to evaluate both types of features, identify key influences on AV safety, and predict test outcomes without execution. Results: Our study identifies critical features affecting test outcomes (effective/ineffective, depending on whether it leads to a safety-critical condition). Results show that combining static and dynamic features improves prediction accuracy, confirmed by models trained on both feature types outperforming models trained with only one type of feature. Conclusion: The interplay of static and dynamic features enhances fault detection in AV testing. This research underscores the importance of integrating both types of features to create more effective testing frameworks for autonomous systems. Key contributions include: (1) a unified framework for AV safety assessment, (2) identification of influential features using ISA, and (3) efficient test outcome prediction for optimized regression testing.

Before being deployed on roads, Autonomous Vehicles (AVs) must undergo comprehensive testing. Safety-critical situations, however, are infrequent in usual driving conditions, so simulated scenarios are used to create them. A test scenario comprises static and dynamic features related to the AV and the test environment; the representation of these features is complex and makes testing a heavy process. A test scenario is effective if it identifies incorrect behaviors of the AV. In this article, we present a technique for identifying key features of test scenarios associated with their effectiveness using Instance Space Analysis (ISA). ISA generates a ( \(2D\) ) representation of test scenarios and their features. This visualization helps to identify combinations of features that make a test scenario effective. We present a graphical representation of each feature that helps identify how well each testing technique explores the search space. While identifying key features is a primary goal, this study specifically seeks to determine the critical features that differentiate the performance of algorithms. Finally, we present metrics to assess the robustness of testing algorithms and the scenarios generated. Collecting essential features in combination with their values associated with effectiveness can be used for selection and prioritization of effective test cases.

Autonomous Vehicles (AVs) are prone to revolutionise the transportation industry. However, they must be thoroughly tested to avoid safety violations. Simulation testing plays a crucial role in finding safety violations of Automated Driving Systems (ADSs). This paper proposes PAFOT, a position-based approach testing framework, which generates adversarial driving scenarios to expose safety violations of ADSs. We introduce a 9-position grid which is virtually drawn around the Ego Vehicle (EV) and modify the driving behaviours of Non-Playable Characters (NPCs) to move within this grid. PAFOT utilises a single-objective genetic algorithm to search for adversarial test scenarios. We demonstrate PAFOT on a well-known high-fidelity simulator, CARLA. The experimental results show that PAFOT can effectively generate safety-critical scenarios to crash ADSs and is able to find collisions in a short simulation time. Furthermore, it outperforms other search-based testing techniques by finding more safety-critical scenarios under the same driving conditions within less effective simulation time.