ABSTRACT The proliferation of driver‐assistance features in vehicles has resulted in a growing interest among the public in fully autonomous driving systems (ADSs). However, the integration of software and hardware in these complex systems presents significant testing challenges, particularly with respect to ensuring passenger safety. To address these challenges, simulation has emerged as a crucial step in the testing of ADSs. This paper presents a solution to the challenges faced in testing ADSs, with a focus on the validation of ADS simulators. The proposed approach involves using simulations and metamorphic testing (MT) to generate multiple concrete metamorphic relations (MRs) for testing ADS simulators. In order to accomplish this goal, we introduce three metamorphic relation patterns (MRPs). Each MRP is accompanied by a metamorphic relation input pattern (MRIP) that aids in generating detailed MRs. These MRs are designed to identify potential issues within the ADS simulator. To simplify the testing process and facilitate MT for testers, a self‐evolving scenario‐testing framework is also presented. The framework allows testers to improve test cases and MRs iteratively until issues detected are confirmed. The benefits and limitations of the framework are demonstrated using an industry case study. Overall, this study offers a practical solution to the challenges in testing ADSs and provides useful insights into improving testing efficiency for researchers and practitioners in the field.

Computing systems are becoming increasingly complex and sophisticated. Technologies such as artificial intelligence, big data, and autonomous vehicles are pushing the boundaries of system size, complexity, and comprehensibility beyond anything seen before. These advances, however, have left the associated software quality assurance (SQA) tools and processes behind. This is compounded by many training and education programs also not attempting to address this inadequacy in the preparation of future software engineering professionals. We face a situation of extensively-deployed advanced computing systems, many of which lack sufficient SQA support. Metamorphic Testing (MT) and Metamorphic Exploration (ME) are SQA approaches that have a record of being able to alleviate some of the challenges associated with the advanced computer systems. This paper reports on an MT/ME experience with the Baidu Apollo autonomous driving system (ADS). The experience included identifying an apparent problem in Apollo, which was later confirmed to be a misunderstanding, but which illustrated the potential for ME to scaffold learning how to perform SQA on such complex systems. The report will be of benefit not only to other ADS developers and testers, but also to other SQA professionals, and especially to SQA trainers and educators.