Multiple fault localization (MFL) identifies the positions of multiple faults (i.e., more than one fault) residing in a buggy program. It is notably more difficult as compared with single fault localization (SFL) which aims to locate a single fault (i.e., one fault) in a buggy program. Clustering-based multiple fault localization (CBMFL) is amongst the most popular MFL approaches, showing promising results in multiple fault localization. The requisite of launching CBMFL depends on test oracles to acquire the test results (i.e., a pass or a failure). In practice, test oracles are commonly not available known as the oracle problem, and CBMFL becomes infeasible in these cases. Inspired by metamorphic testing in solving the oracle problem, we attempt to combine this technique into CBMFL to broaden its application scope. Thus, we propose MetaMFL:MetamorphicMultipleFaultLocalization, which leverages metamorphic testing to extend CBMFL to the cases where test oracles are not available. Specifically, MetaMFL uses metamorphic testing groups as minimum units of testing. It defines metamorphic features for representing those that have violated metamorphic relations. Using these features, CBMFL can perform clustering to support parallel debugging, thus achieving MFL without test oracles. The large-scale experiments show that MetaMFL largely retains the effectiveness of CBMFL even though test oracles are not available.

Fault localization (FL) analyzes the execution information of a test suite to pinpoint the root cause of a failure. The class imbalance of a test suite, i.e., the imbalanced class proportion between passing test cases (i.e., majority class) and failing ones (i.e., minority class), adversely affects FL effectiveness.To mitigate the effect of class imbalance in FL, we propose CGAN4FL: a data augmentation approach using Context-aware Generative Adversarial Network for Fault Localization. Specifically, CGAN4FL uses program dependencies to construct a failure-inducing context showing how a failure is caused. Then, CGAN4FL leverages a generative adversarial network to analyze the failure-inducing context and synthesize the minority class of test cases (i.e., failing test cases). Finally, CGAN4FL augments the synthesized data into original test cases to acquire a class-balanced dataset for FL. Our experiments show that CGAN4FL significantly improves FL effectiveness, e.g., promoting MLP-FL by 200.00%, 25.49%, and 17.81% under the Top-1, Top-5, and Top-10 respectively.

Abstract Deep‐Learning‐based Fault Localisation (DLFL) leverages deep neural networks to learn the relationship between statement behaviour and program failures, showing promising results. However, since DLFL uses program failures as labels to conduct supervised learning, a labelled dataset is a requisite of applying DLFL. A failure is detected by comparing program output with a test oracle which is the standard answer for the given input. The problem is, test oracles are often difficult, or even impossible to acquire in real life, and that has severely restricted the application of DLFL since we have only unlabelled datasets in most cases. Thus, MetaFL: Metamorphic Fault Localisation Using Weakly Supervised Deep Learning is proposed, to provide a weakly supervised learning solution for DLFL. Instead of using test oracles, MetaFL uses metamorphic relations to prescribe expected behaviour of a program, and defines labels of metamorphic testing groups by verifying integrity in each group of test cases. Hence, a coarse‐grained labelled dataset can be built from the originally unlabelled one, with which DLFL can work now, utilising a weakly supervised learning paradigm. The experiments show that MetaFL yields a performance comparable to plain DLFL under ideal condition (i.e. the labels of datasets are available). MetaFL successfully extends the methodology of DLFL from supervised learning to weakly supervised learning, and a fully labelled dataset is no longer mandatory for applying DLFL.

Automated fault localization techniques collect runtime information as input data to identify suspicious statement potentially responsible for program failures. To discover the statistical coincidences between test results (i.e., failing or passing) and the executions of the different statements of a program (i.e., executed or not executed), researchers developed a suspiciousness methodology (e.g., spectrum-based formulas and deep neural network models). However, the occurrences of coincidental correctness (CC) which means the faulty statements were executed but the output of the program was right affect the effectiveness of fault localization. Many researchers seek to identify CC tests using cluster analysis. However, the high-dimensional data containing too much noise reduce the effectiveness of cluster analysis.