Deep learning (DL) compilers are crucial for optimizing DL models across diverse hardware platforms, ensuring their reliability is paramount. Although many techniques have been developed to test DL compilers, existing methods often suffer from efficiency issues. To resolve this problem, this study proposes MeDAC, a novelMessage passing neural network based framework forDL compiler testingACceleration. The key insight of MeDAC is to construct a learning model to accurately predict the bug-revealing probabilities of test cases (i.e., DL models), allowing the test cases with higher bug-revealing probabilities to be executed. First, to construct the learning model, three types of features for DL models are extracted. Then, we propose a novel learning model based on the message passing neural network with an attention mechanism to accurately estimate the bug-revealing probabilities of DL models. Finally, MeDAC utilizes the learning model to predict the bug-revealing probability of each DL model, ensuring high-risk models are executed first. Experimental results on TVM and ONNXRuntime demonstrate that MeDAC achieves an average of 59.20% speedup in test execution time for DL compiler testing. Moreover, the peak performance improvements of MeDAC are 385.68% and 217.96% over the baseline approaches LET and GCN, respectively.

Computer-aided engineering (CAE) tools are extensively used in safety-critical domains like aerospace design to simulate real-world physical processes on computers at reduced costs. However, CAE tools are prone to bugs, leading to incorrect simulation and serious design flaws. Existing testing methods have limited success in finding these bugs, since constructing test cases for CAE tools (known as CAE inputs) is challenging, due to the complexity of input parameter constraints and the differences of input syntax for each tool. Therefore, we propose NSGen, a template-based Numerical Simulation test caseGENerator for effective CAE input generation. To bridge input differences, NSGen designs general syntax rules that ignore semantic details of CAE inputs but retaining the format and validity of these inputs for different CAE tools. NSGen then designs semantic rules to define the parameters and their intricate constraints (i.e., dependency, exclusion, and extension). By instantiating these rules as templates, valid CAE inputs are generated using a syntax parser implemented by NSGen. Experiments show that NSGen can effectively generate CAE inputs with less than 0.33 second on average, triggering 6.28 to 10.55 times more potential issues than the baseline. Using these inputs, NSGen finds 12 bugs in popular CAE tools, including crash bugs and instability bugs.

Logic synthesis tools translate Hardware Description Language (HDL) designs into hardware implementation. To test these tools, numerous test cases are usually executed on the tools, yet only a few of them can trigger faults, leading to inefficient testing. Since executing test cases on logic synthesis tools often requires significant cost on complicated synthesis and simulation, fault-triggering test cases should be prioritized to execute. However, existing prioritization methods face challenges in accurately predicting the fault-triggering capability of dynamically generated test cases and modeling the unique syntactic and structure complexities of these HDL-based programs.Therefore, we propose ProFuse, a multi-dimensional feature fusion method for logic synthesis tool test case prioritization. ProFuse leverages Abstract Syntax Trees (AST) and Data Flow Graphs (DFG) to extract novel syntactic and structure features from HDL designs. These features are processed by a joint model of Multilayer Perceptron (MLP) and Graph Convolutional Network (GCN) to rank fault-triggering test cases accurately. ProFuse achieves an Average Percentage of Fault Detection (APFD) score of 0.9285, outperforming the state-of-the-art prioritization methods by 11.38% to 82.49%. ProFuse can efficiently rank randomly generated test cases to discover 15 new faults in logic synthesis tools (i.e., Yosys and Vivado). The Vivado community acknowledged our work for improving their tool.

In modern electronics, printed circuit board (PCB) design tool chain is crucial for product development but may harbor hidden bugs. These bugs can cause simulation inconsistencies and performance issues that risk flawed designs in critical applications. To address this, we propose PCBFen, a novel approach that employs functionally equivalent mutation strategies to generate netlists for bug detection. PCBFen designs circuit topology mutations and conditional control mutations to modify netlist files of a circuit without changing circuit functionality. By simulating the mutated netlists and their corresponding seed netlists, potential bugs can be identified, if their simulation results are inconsistent. We conducted extensive experiments on typical PCB design tool chain (including KiCad and Ngspice). The results show that PCBFen generates netlists up to five times faster than existing methods, and detects significantly more potential issues of the tools. In our tests, PCBFen identified 12 bugs, including both simulation inconsistencies and performance issues. Our findings suggest that PCBFen is effective in uncovering critical bugs in PCB design tool chain, and represents a promising advancement in ensuring the reliability of these essential tools.

Field Programmable Gate Array (FPGA) logic synthesis compilers (e.g., Vivado, Iverilog, Yosys, and Quartus) are widely applied in Electronic Design Automation (EDA), such as the development of FPGA programs. However, defects (e.g. incorrect synthesis) in logic synthesis compilers may lead to unexpected behaviors in target applications, posing security risks. Therefore, it is crucial to thoroughly test logic synthesis compilers to eliminate such defects. Despite several Hardware Design Language (HDL) code generators (e.g., Verismith) having been proposed to find defects in logic synthesis compilers, the effectiveness of these generators is still limited by the simple code generation strategy and the monogeneity of the generated HDL code. This paper proposes EvoHDL, a novel method to generate syntax-valid HDL code for comprehensively testing FPGA logic synthesis compilers. EvoHDL can generate more complex and diverse defect-triggering HDL code (e.g., Verilog, VHDL, and SystemVerilog) by leveraging the guidance of abstract syntax tree and the extensive function block libraries of cyber-physical systems. Extensive experiments show that the diversity and defect-triggering capability of HDL code generated by EvoHDL are significantly better than the state-of-the-art method (i.e., Verismith). In three months, EvoHDL has reported 20 new defects–many of which are deep and important; 16 of them have been confirmed.

In electronic design automation (EDA), printed circuit board (PCB) design plays a crucial role. Ensuring the reliability of the PCB design tool chain is essential, as bugs in the tool chain can cause significant issues and losses during design and production. To improve reliability, a key process is to generate numerous PCB schematics and execute them in the tool chain, to test the correctness of each tool chain functionality. However, it is a challenge to automatically generate valid schematics to simulate the actual use of the PCB design tool chain. To this end, we propose PCBSmith, an effective schematic generator for PCB design tool chain. PCBSmith mimics the steps of a PCB designer for schematic design. PCBSmith first selects the appropriate electronic components from a comprehensive library and connects them according to the constraints of different components. PCBSmith then sets electrical parameters and simulation models for each component, eventually generating simulatable schematics. Experiments show that PCBSmith demonstrates high efficiency in schematic generation, averaging only one schematic per second. PCBSmith maintains a success rate over 61.44% for generating schematics, which outperforms the baseline method by 30.68%. The generated schematics have successfully identified unknown bugs in PCB design tools.