Abstract Recent advances in artificial intelligence technology and perception components have promoted the rapid development of autonomous vehicles. However, as safety‐critical software, autonomous driving systems often make wrong judgments, seriously threatening human and property safety. LiDAR is one of the most critical sensors in autonomous vehicles, capable of accurately perceiving the three‐dimensional information of the environment. Nevertheless, the high cost of manually collecting and labeling point cloud data leads to a dearth of testing methods for LiDAR‐based perception modules. To bridge the critical gap, we introduce MetaLiDAR, a novel automated metamorphic testing methodology for LiDAR‐based autonomous driving systems. First, we propose three object‐level metamorphic relations for the domain characteristics of autonomous driving systems. Next, we design three transformation modules so that MetaLiDAR can generate natural‐looking follow‐up point clouds. Finally, we define corresponding evaluation metrics based on metamorphic relations. MetaLiDAR automatically determines whether source and follow‐up test cases meet the metamorphic relations based on the evaluation metrics. Our empirical research on five state‐of‐the‐art LiDAR‐based object detection models shows that MetaLiDAR can not only generate natural‐looking test point clouds to detect 181,547 inconsistent behaviors of different models but also significantly enhance the robustness of models by retraining with synthetic point clouds.

Abstract The development of artificial intelligence and information communication technology has significantly propelled advancements in autonomous driving. The advent of autonomous driving has a profound impact on societal development and transportation methods. However, as intelligent systems, autonomous driving systems (ADSs) often make wrong judgements in specific scenarios, resulting in accidents. There is an urgent need for comprehensive testing and validation of ADSs. Metamorphic testing (MT) techniques have demonstrated effectiveness in testing ADSs. Nevertheless, existing testing methods primarily encompass relatively simple metamorphic relations (MRs) that only verify ADSs from a single perspective. To ensure the safety of ADSs, it is essential to consider the various elements of driving scenarios during the testing process. Therefore, this paper proposes MetaSem, a novel metamorphic testing method based on semantic information of autonomous driving scenes. Based on semantic information of the autonomous driving scenes and traffic regulations, we design 11 MRs targeting different scenario elements. Three transformation modules are developed to execute addition, deletion and replacement operations on various scene elements within the images. Finally, corresponding evaluation metrics are defined based on MRs. MetaSem automatically discovers inconsistent behaviours according to the evaluation metrics. Our empirical study on three advanced and popular autonomous driving models demonstrates that MetaSem not only efficiently generates visually natural and realistic scene images but also detects 11,787 inconsistent behaviours on three driving models.

In recent years, deep neural networks (DNNs) have made great progress in people’s daily life since it becomes easier for data accessing and labeling. However, DNN has been proven to behave uncertainly, especially when facing small perturbations in their input data, which becomes a limitation for its application in self-driving and other safety-critical fields. Those human-made attacks like adversarial attacks would cause extremely serious consequences. In this work, we design and evaluate a safety testing method for DNNs based on mutation testing, and propose an adversarial training method based on testing results and joint optimization. First, we conduct an adversarial mutation on the test datasets and measure the performance of models in response to the adversarial samples by mutation scores. Next, we evaluate the validity of mutation scores as a quantitative indicator of safety by comparing DNN models and their updated versions. Finally, we construct a joint optimization problem with safety scores for adversarial training, thus improving the safety of the model as well as the generalizability of the defense capability.

The Conical Scan Microwave radiometer (CSMR) of the HY-2 satellite is a rotating microwave imager. It covers frequency band from 6.6GHz to 37GHz, which receives the earth radiation by two feed horns, and provides the capability of observing the sea surface temperature, the wind speed, the water vapour and the liquid water etc. It was launched at 16 august 2010. This paper provides an overview of the the instrument's characteristic and the inter-satellite calibration results are also presented.