Scenario-based approaches have gained popularity in the context of automated vehicles. As in any model-based approach, validation to real data needs to be considered. This work studies concrete scenarios based on real data from a large-scale and open road user trajectory dataset. In particular, we detail on cut-ins and (hard) braking maneuvers and study them based on existing threat metrics. In this work, we present the complete workflow starting from the pre-processing and validation of the data, the definition of concrete scenarios based on how we extract them from the dataset and their evaluation based on several threat metrics. We identify peculiarities of the dataset itself, as well as of the driving behavior of vehicles therein. As this work relies on an open dataset, results can be reproduced and readily compared.

Automated test generation and evaluation in simulation environments is a key technology for verification of automated driving (AD) applications. Search-based testing (SBT) is an approach for automated test generation that leverages optimization to efficiently generate interesting concrete tests from abstract test descriptions. In this experience paper, we report on our observations after successfully applying SBT to AD control applications in several use cases with different characteristics. Based on our experiences, we derive a number of lessons learned that we consider important for the adoption of SBT methods and tools in industrial settings. The key lesson is that SBT finds relevant errors and provides valuable feedback to the developers, but requires tool support for writing specifications.

Model-based development is an important paradigm for developing cyber-physical systems (CPS). The underlying assumption is that the functional behavior of a model is related to the behavior of a more concretized model or the real system. A formal definition of such a relation is called conformance relation. There are a variety of conformance relations, and the question arises of how to select a conformance relation for the development of CPS. The contribution of this article is a survey of the definitions and algorithms of conformance relations for CPS. Additionally, the article compares several conformance relations and provides guidance on which relation to select for specific problems. Finally, we discuss how to select inputs for testing conformance.

Mobile robots operating in a shared environment with pedestrians are required to move provably safe to avoid harming pedestrians. Current approaches like safety fields use conservative obstacle models for guaranteeing safety, which leads to degraded performance in populated environments. In this paper, we introduce an online verification approach that uses information about the current pedestrian velocities to compute possible occupancies based on a kinematic model of pedestrian motion. We demonstrate that our method reduces the need for stopping while retaining safety guarantees, and thus goals are reached between 1.4 and 3.5 times faster than the standard ROS navigation stack in the tested scenarios.

Industrial-sized hybrid systems are typically not amenable to formal verification techniques. For this reason, a common approach is to formally verify abstractions of (parts of) the original system. However, we need to show that this abstraction conforms to the actual system implementation including its physical dynamics. In particular, verified properties of the abstract system need to transfer to the implementation. To this end, we introduce a formal conformance relation, called reachset conformance, which guarantees transference of safety properties, while being a weaker relation than the existing trace inclusion conformance. Based on this formal relation, we present a conformance testing method which allows us to tune the trade-off between accuracy and computational load. Additionally, we present a test selection algorithm that uses a coverage measure to reduce the number of test cases for conformance testing. We experimentally show the benefits of our novel techniques based on an example from autonomous driving.

Stability is an important system property. In this paper we examine the steps toward fully automated stability verification for a class of discrete-time hybrid systems. These systems are hybrid in the sense that they can have several modes with possibly different discrete-time dynamics. For systems with affine discrete-time dynamics in each mode, we employ a method that is based on Lyapunov theory and reduces the verification task to convex optimization. We detail the steps that are needed for automatic stability verification using this method: 1) choice of a state space partitioning 2) calculation of possible transitions between the different regions of the partitioning and 3) conversion of the problem into linear matrix inequalities, which can be solved through convex optimization. For steps 2) and 3) we present solutions that are suitable for full automation.