Abstract Autonomous vehicles (AVs) must operate safely in the face of uncertainty, including those induced by human behaviors (i.e., external human drivers). Specifically, AVs must exhibit safe responses when encountering previously unseen behaviors from human drivers with different driving styles. For example, aggressive drivers may cut off other vehicles to merge into a lane, or distracted drivers may fail to respond to changing road conditions. A key challenge is how to assess the onboard AV decision-making capabilities to detect and mitigate those potentially unsafe scenarios due to one or more external human-operated vehicles. We observe that AVs and other vehicles on the roadway may share common functional objectives (e.g., to navigate to a given target destination), but otherwise may be motivated by different non-functional objectives, such as safety, minimizing transport time, minimizing fuel consumption, etc. This paper introduces a modular and composable model- and game-based testing framework to enable an AV developer to operationally assess the robustness of an AV in response to human-based uncertainty. Specifically, this work uses goal models to declaratively specify functional and non-functional objectives of vehicles (both the AV under study and those representing external human-operated vehicles) to inform the game-based testing environment that incorporates real-world traffic infrastructure data. We demonstrate the model-based capabilities of our game-based testing approach on a number of scenarios based on real-world traffic accident data involving human drivers.

Increasingly, safety-critical systems include artificial intelligence and machine learning components (i.e., Learning-Enabled Components (LECs)). However, when behavior is learned in a training environment that fails to fully capture real-world phenomena, the response of an LEC to untrained phenomena is uncertain, and therefore cannot be assured as safe. Automated methods are needed for self-assessment and adaptation to decide when learned behavior can be trusted. This work introduces a model-driven approach to manage self-adaptation of a Learning-Enabled System (LES) to account for run-time contexts for which the learned behavior of LECs cannot be trusted. The resulting framework enables an LES to monitor and evaluate goal models at run time to determine whether or not LECs can be expected to meet functional objectives. Using this framework enables stakeholders to have more confidence that LECs are used only in contexts comparable to those validated at design time.