Summary Online judgment (OJ) systems are capable of evaluating program results by automatically executing test cases, significantly improving the efficiency of traditional guidance approaches. Moreover, existing studies attempt to assist novices through automated fault localization techniques to provide feedback to novices, which can help them quickly find the location of faulty statements. Among them, spectrum‐based fault localization (SBFL) techniques have been widely used for their lightweight and efficiency, which only requires coverage information and test results of test cases to conduct fault localization. However, manually constructing high‐quality test cases for a large number of OJ questions is tough work to complete. To solve this problem, we propose the novice program‐orientedMulti‐Objective Optimization‐BasedFault Localization‐OrientedTestCaseGeneration (MFTCG) for automatically generating test inputs. Specifically, we use multi‐objective optimization algorithms to evolve the test case in terms of both fault localization and faulty code detection capability. We conduct experiments with 8911 programs from the well‐known public OJ platform AtCoder. The results show that our proposed approach MFTCG can achieve the best fault localization performance compared with existing automated test case generation approaches in most cases and can achieve the similar faulty code detection capability compared to manually designed test cases.

MBFL (Mutation-Based Fault Localization) is one of the most commonly studied fault localization techniques due to its promising fault localization effectiveness. However, MBFL incurs a high execution cost as it needs to execute the test suite on a large number of mutants. While previous studies have proposed mutant reduction methods for FOMs (First-Order Mutants) to help alleviate the cost of MBFL, the reduction of HOMs (Higher-Order Mutants) has not been thoroughly investigated. In this study, we propose SGS (Statement Granularity Sampling), a method which conducts HOMs reduction for HMBFL (Higher-Order Mutation-Based Fault Localization). Considering the relationship between HOMs and statements, we sample HOMs at the statement level to ensure each statement has corresponding HOMs. We empirically evaluate the fault localization effectiveness of HMBFL using SGS on 237 multiple-fault programs taken from the SIR and Codeflaws benchmarks. The experimental results show that (1) The best sampling ratio for HMBFL with SGS is 20%, which preserves the performance and reduces execution costs by 80% ; (2) The fault localization accuracy of HMBFL with SGS outperforms the state-of-the-art SBFL (Spectrum-Based Fault Localization) and MBFL techniques by 20%.

First-order mutants (FOMs) have been widely used in mutation-based fault localization (MBFL) approaches and have achieved promising results in single-fault localization scenarios (SFL-scenario). Higher-order mutants (HOMs) are proposed to simulate complex faults and can be applied in MBFL theoretically for multiple-fault localization scenarios (MFL-scenario). However, whether HOMs can improve MBFL’s performance is not investigated and the effectiveness is not thoroughly evaluated. In this empirical study, we investigate the impact of HOMs on the performance of MBFL in SFL-scenario and MFL-scenario. The experiments on two real-world benchmarks reveal that 1) 2-HOMs can help improve the MBFL performance in SFL-scenarios; 2) in MFL-scenarios, both 2-HOMs and 3-HOMs can achieve better performance than FOMs; and 3) huge computational cost cannot be ignored in the practice of HOMs. Therefore, effective methods to reduce the number of HOMs for future MBFL studies should be considered.