The widespread use of deep neural networks (DNNs) in image classification sofwares underlines the importance of the robustness. Researchers have proposed sparse adversarial attack methods for generating test cases, which add pixel-level perturbations to construct the test case to mislead the target model. However, the existing methods have certain limitations, such as high time cost, poor flexibility, and poor quality of the test cases. To address these issues, we propose a gradient-guided test case generation method (GGTM) to evaluate the robustness of image classification software. The method firstly identifies the key region in the image based on the gradient-weighted class activation mapping (Grad-CAM) and the prediction confidence of the target model on the input image. In the key region, it selects a set of pixels as candidate perturbation pixels according to the gradient value and the change of loss function. Then perturbations are added to the candidate perturbation pixels after applying a random dropout strategy to reduce some candidate perturbation pixels which is used to avoid local optimum. For the initially constructed test case which can mislead the target model, after removing redundant and unimportant perturbations, perturbations are re-added to optimize the test case. Experiments show the effectiveness of GGTM, which achieves 100% attack success rate. And the test cases generated by GGTM have the best perturbation sparsity. Furthermore, compared with the baseline method SparseAG which achieves optimal perturbation sparsity among the baseline methods, GGTM significantly improves the efficiency.

The widespread adoption of DNNs in NLP software has highlighted the need for robustness. Researchers proposed various automatic testing techniques for adversarial test cases. However, existing methods suffer from two limitations: weak error-discovering capabilities, with success rates ranging from 0% to 24.6% for BERT-based NLP software, and time inefficiency, taking 177.8s to 205.28s per test case, making them challenging for time-constrained scenarios. To address these issues, this paper proposes LEAP, an automated test method that uses LEvy flight-based Adaptive Particle swarm optimization integrated with textual features to generate adversarial test cases. Specifically, we adopt Levy flight for population initialization to increase the diversity of generated test cases. We also design an inertial weight adaptive update operator to improve the efficiency of LEAP's global optimization of high-dimensional text examples and a mutation operator based on the greedy strategy to reduce the search time. We conducted a series of experiments to validate LEAP's ability to test NLP software and found that the average success rate of LEAP in generating adversarial test cases is 79.1%, which is 6.1% higher than the next best approach (PSOattack). While ensuring high success rates, LEAP significantly reduces time overhead by up to 147.6s compared to other heuristic-based methods. Additionally, the experimental results demonstrate that LEAP can generate more transferable test cases and significantly enhance the robustness of DNN-based systems.