The robustness of neural network classifiers is important in the safety-critical domain and can be quantified by robustness verification. At present, efficient and scalable verification techniques are always sound but incomplete, and thus, the improvement of verified robustness results is the key criterion to evaluate the performance of incomplete verification approaches. The multi-variate function MaxPool is widely adopted yet challenging to verify. In this paper, we present Ti-Lin, a robustness verifier for MaxPool-based CNNs with Tight Linear Approximation. Following the sequel of minimizing the over-approximation zone of the nonlinear function of CNNs, we are the first to propose the provably neuron-wise tightest linear bounds for the MaxPool function. By our proposed linear bounds, we can certify larger robustness results for CNNs. We evaluate the effectiveness of Ti-Lin on different verification frameworks with open-sourced benchmarks, including LeNet, PointNet, and networks trained on the MNIST, CIFAR-10, Tiny ImageNet and ModelNet40 datasets. Experimental results show that Ti-Lin significantly outperforms the state-of-the-art methods across all networks with up to 78.6% improvement in terms of the certified accuracy with almost the same time consumption as the fastest tool. Our code is available at https://github.com/xiaoyuanpigo/Ti-Lin-Hybrid-Lin.

Abstract The development of artificial intelligence and information communication technology has significantly propelled advancements in autonomous driving. The advent of autonomous driving has a profound impact on societal development and transportation methods. However, as intelligent systems, autonomous driving systems (ADSs) often make wrong judgements in specific scenarios, resulting in accidents. There is an urgent need for comprehensive testing and validation of ADSs. Metamorphic testing (MT) techniques have demonstrated effectiveness in testing ADSs. Nevertheless, existing testing methods primarily encompass relatively simple metamorphic relations (MRs) that only verify ADSs from a single perspective. To ensure the safety of ADSs, it is essential to consider the various elements of driving scenarios during the testing process. Therefore, this paper proposes MetaSem, a novel metamorphic testing method based on semantic information of autonomous driving scenes. Based on semantic information of the autonomous driving scenes and traffic regulations, we design 11 MRs targeting different scenario elements. Three transformation modules are developed to execute addition, deletion and replacement operations on various scene elements within the images. Finally, corresponding evaluation metrics are defined based on MRs. MetaSem automatically discovers inconsistent behaviours according to the evaluation metrics. Our empirical study on three advanced and popular autonomous driving models demonstrates that MetaSem not only efficiently generates visually natural and realistic scene images but also detects 11,787 inconsistent behaviours on three driving models.

From reducing traffic congestion to improving transportation, autonomous vehicles have immense potential in enhancing productivity and quality of life. As a safety-critical system, autonomous vehicles must undergo extensive testing before being deployed on public roads to ensure their safety and reliability. Given the complexity and high dimensionality of testing scenarios for autonomous driving, this paper proposes a test case generation method based on an improved genetic algorithm. The LGSVL simulator is used to conduct simulation tests on the Baidu Apollo system. The experimental results demonstrate that the test cases generated by this method can effectively test various safety violations of autonomous vehicles and improve the efficiency of generating effective test cases.

Actionable warning identification (AWI) is crucial for improving the usability of static analysis tools. Currently, machine learning (ML)-based AWI approaches are notably common, which mainly focus on seeking high performance by improving the warning feature extraction and advancing the AWI model training. However, these approaches ignore an important fact that the number of actionable warnings is much smaller than that of unactionable warnings in the warning dataset used for the AWI model training (i.e., the class imbalance). Learning from such an imbalanced dataset may limit the performance of ML-based AWI approaches. To bridge the above gap, we are the first to conduct a comprehensive empirical study to investigate the impact of class imbalance on the ML-based AWI performance, whether class rebalancing methods can improve the ML-based AWI performance, and the differences of class rebalancing methods in the ML-based AWI model. Our empirical study is performed on 9 real-world and large-scale warning datasets, 25 typical class rebalancing methods, and 7 commonly used ML models. The experimental results show that 1) the class imbalance has a negative impact on the ML-based AWI performance; 2) 85% class rebalancing methods can significantly improve the ML-based AWI performance, but 8% ones do not work in the imbalanced warning datasets; 3) RandomOverSampler combined with AdaBoost/Random Forest can make the ML-based AWI model achieve optimal performance on nine warning datasets. Finally, we provide three practical guidelines that could help refine ML-based AWI approaches.