Ensuring the safety of autonomous driving systems (ADSs) through rigorous verification in simulated environments is crucial before real-world deployment. However, using simulation environments for ADS testing and verification poses several challenges, including specifying various behaviors of traffic participants and collecting comprehensive real-time data to verify the ADS. To address these challenges, we propose a framework for runtime verification of ADSs, focusing on Autoware, a leading ADS. This framework integrates AWSIM-Script, a scripting language for defining traffic scenarios; Runtime Monitor, a tool to record real-time data during the simulation; and AW-Checker, a Linear Temporal Logic-based property checker for verifying safety requirements. Unlike prior research that primarily focuses on generating critical scenarios, we leverage a well-established ADS safety standard from the Japan Automobile Manufacturers Association and adopt its systematic methodology for safety assessment. We conducted a series of experiments focused on nonintersection road geometry to evaluate Autoware's capability in handling different traffic disturbances such as cut-in, cut-out, and deceleration scenarios. The results revealed that, compared to the competent and careful driver model, which represents the minimum safety requirements for ADSs, Autoware failed to prevent collisions in some cases, particularly during high-speed scenarios and fast lateral movements by other vehicles.

Automated driving systems(ADS) are major trend and the safety of such critical system has become one of the most important research topics. We usually use scenarios in order to define the specifications of ADS. In these scenarios, graphical diagrams are often used to represent abstractly the positioning and behavior of vehicles. However, such diagrams are not suitable for the development of high-reliability systems, because they are informal and may cause discrepancies among different engineers. In this paper, we propose a formal speci-fication language called Bounding Box Specification Language (BBSL) which allows us to write rigorous specifications of ADS. BBSL describe multiple types of objects in a driving environment, such as vehicles and pedestrians, as bounding boxes defined as two-dimensional interval, and describe positional relationships between them in mathematical notation. It is capable of strictly delineating many positional relationships while being also capable of expressing specifications that are concise enough to be read and written manually. Therefore, BBSL is suitable for describing the specification of Object and Event Detection and Response (OEDR) among the tasks of ADS. In this paper, we describe what kind of description BBSL enables, and describe its operations. Then, we show examples of specifications of ADS written in BBSL and discuss the advantages of specifications written in BBSL.

Simulink is a popular tool for modeling cyber-physical systems. As more models are produced in industry, automated quality assurance of models becomes increasingly important. This paper describes an empirical evaluation of four methods for the coverage testing of Simulink models: A) SimuLink Design Verifier (SLDV), a dedicated official tool; B) Template-Based Monte-Carlo (TBMC) method, a random test generation method that utilizes input signal templates; C) SMT- Based Model Checking (SBMC) method that conducts static analysis via encoding models into logic formulas; and D) a hybrid method of B and C. Based on the evaluation results, we carefully designed the hybrid method to complement the features of TBMC and SBMC. In the experiments, we have applied the methods to fourteen models and evaluated their performance. The results show that the hybrid method achieved better results than SLDV for several models.

Summary Handling hardware‐dependent properties at a low level is usually required in developing microcontroller‐based applications. One of these hardware‐dependent properties is cautions, which are described in microcontrollers hardware manuals. The process of verifying these cautions is performed manually, as there is currently no single tool that can directly handle this task. This research aims at automating the verification of these cautions. To obtain the typical cautions of microcontrollers, we investigate two sections which have a considerable number of required cautions in the hardware manual of a popular microcontroller. Subsequently, we analyse these cautions and categorize them into several groups. Based on this analysis, we propose a semi‐automatic approach for verifying the cautions which integrates two static programme analysis techniques (i.e., pattern matching and abstract interpretation). To evaluate our approach, we conducted experiments with generated source code, benchmark source code, and industrial source code. The generated source code, which was created automatically based on several aspects of the C programme, was used to evaluate the performance of the approach based on these aspects. The benchmark and the industrial source code, which were provided by Aisin Software Co., Ltd., were used to assess the feasibility and applicability of the approach. The results show that all expected violations in the benchmark source code were detected. Unexpected but real violations in the benchmark programme were also detected. For the industrial source code, the approach successfully handled and detected most of the expected violations. These results show that the approach is promising in verifying the cautions.

Recently, machine learning, particularly deep learning, is attracting much interest and is applied in various systems. Applications include not only entertainment systems, but safety-critical systems such as those found in autonomous vehicles. The reliability of such safety-critical systems must be guaranteed before they are released into society. However, methods for ensuring the safety of machine learning-based systems have yet to be established. In this paper, we propose a method for systematically evaluating the safety of such systems. The method consists of dataset-based safety analysis and statistical evaluation of testing results. In the safety analysis, we extend the widely used fault tree analysis to deal with datasets. In the testing, we use statistical estimation to guarantee recognition rates obtained in the safety analysis. We conducted experiments using a handwritten character recognition system implemented as a CNN to demonstrate the feasibility and effectiveness of our method.

Microcontrollers are usually supplied with hardware manuals, where information that requires special attention is emphasized as cautions. Currently, the process of verifying these cautions is performed manually as there is no single tool that can directly handle this task. This research aims at automating the verification process for these cautions as much as possible. Firstly, we investigate two sections which have a considerable number of required cautions in the hardware manual of a popular microcontroller to obtain the typical cautions of microcontrollers. Secondly, we analyze and categorize these cautions into several groups. Subsequently, we propose a semi-automatic approach which uses the assertion-based method and integrates two existing static program analysis tools (i.e., Cobra and Eva plugin of Frama-C) to verify the cautions. To show the applicability of this approach, we conduct two experiments with a benchmark source code and an industrial source code provided by Aisin comCruise Co., Ltd.. The results show that this approach is capable of detecting all violations in the benchmark program and only misses one expected violation in the industrial project.

Static analysis tools have demonstrated their ability to find non-compliant code of coding standards. However, for industrial-sized systems, static analysis tools frequently report a large number of warnings, which contain both true positives and false positives. In this research, to enable precise check for SEI CERT C Coding Standard, we combine static analysis with three different techniques. Firstly, a static analysis tool is used to detect non-compliant code, which are positions that may violate a SEI CERT C rule or recommendation. Each detected position is called a warning. Secondly, deductive verification, model checking, and pattern matching are used to verify whether each warning is a true positive or a false positive. Our experiments with two automotive applications show that this approach can help to improve the accuracy to check for SEI CERT C Coding Standard. We verify nearly 60% warnings of Rosecheckers, a static analysis tool. In these verified warnings, 97% of them are automatically detected to be true positives or false positives by our approach.

MATLAB/Simulink is the de facto standard tool for the model-based development (MBD) of control software for automotive systems. A model developed in MBD is called a Simulink model and, for real automotive systems, involves complex computation as well as tens of thousands of blocks. In this paper, we propose an automated test generation tool for such large and complex Simulink models. The tool provides functions for (1) automatically generating high-coverage test-suites for practical models, which cannot be handled by Simulink Design Verifier (SLDV), and (2) measuring decision, condition and MC/DC coverage much more efficiently than Simulink Coverage (SLC). This automatic test-suite generation adopts a Monte-Carlo method with templates of test cases. Our experimental evaluation shows that the tool can provide test suites against practical implementation models with higher coverage and shorter execution times than SLDV.

Linear temporal logic (LTL) synthesis is a formal method for automatically composing a reactive system that realizes a given behavioral specification described in LTL if the specification is realizable. Even if the whole specification is unrealizable, it is preferable to synthesize a best-effort reactive system. That is, a system that maximally realizes its partial specifications. Therefore, we categorized specifications into must specifications (which should never be violated) and desirable specifications (the violation of which may be unavoidable). In this paper, we propose a method for synthesizing a reactive system that realizes all must specifications and strongly endeavors to satisfy each desirable specification. The general form of the desirable specifications without assumptions is $$\mathbf{G }\varphi $$ , which means “ $$\varphi $$ always holds”. In our approach, the best effort to satisfy $$\mathbf{G }\varphi $$ is to maximize the number of steps satisfying $$\varphi $$ in the interaction. To quantitatively evaluate the number of steps, we used a mean-payoff objective based on LTL formulae. Our method applies the Safraless approach to construct safety games from given must and desirable specifications, where the must specification can be written in full LTL and may include assumptions. It then transforms the safety games constructed from the desirable specifications into mean-payoff games and finally composes a reactive system as an optimal strategy on a synchronized product of the games.