Identifying the cause of a software bug is a difficult and costly task that requires a detailed understanding of codebase structures. We argue that a smaller target space can reduce the developer's investigation effort, thus it should be utilized in fault localization. In this paper, we propose a new spectrum-based fault location (SBFL) approach based on the test call-structure information, called Test Call-Structure-Based (TCS) fault location. We evaluate the effectiveness of our approach using the Barinel formula on the Defects4J bug benchmark. The results show that our approach can achieve an average rank improvement in 75% of the projects in Defects4J. Furthermore, our approach can place more buggy methods among the Top-10 most suspicious elements. Moreover, our approach identifies more bugs that were previously unlikely to be found using the hit-based SBFL method.

Spectrum-based fault localization is an automated technique that helps developers identify and isolate the origin of suspicious errors during software development. Despite being a well-researched topic, it is rarely used in the industry. The primary reason is that, in its basic version, it uses only local information on the coverage of a program element to estimate its probability of failure, rarely utilizing additional contextual information on the element or the test cases. Other researchers have tried solving the problem using contextual information with varying success. In this paper, we enhance the approach called Call Frequency-based Fault Localization, which analyzes the method's occurrence frequency in call-stack instances of failed tests. While it boosts SBFL's effectiveness, it overlooks the test scope. We propose that identifying unit and unit-like tests, followed by adjusting the frequency of the method by test type, can further enhance the fault localization ability of FL techniques. We empirically evaluated our method's effectiveness with the Defects4J benchmark. We found that utilizing weights in Call Frequency-based Fault Localization often ranks faulty methods higher, increasing the number of items in the top-10 positions.

Many Spectrum-based Fault Localization (SBFL) techniques have been published over the years, but they are still rarely used in the industry. Traditional SBFL methods may be effective in certain situations but may fall short in others due to a lack of contextual information for the algorithm to function properly. I have made efforts to improve SBFL algorithms by investigating how SBFL aligns bug-fix patterns with high-ranked elements, creating an improved algorithm that uses call frequencies, examining how division by zero affects SBFL’s efficiency, and proposing ways to avoid it. I have also developed a user feedback-based SBFL tool called CharmFL, which is helpful for Python programmers. I consider all of these things as contexts that can be used to create a more context-aware SBFL approach. In this doctoral symposium, I present my research and outline my plans for the remainder of my Ph.D. program.

Spectrum-Based Fault Localization (SBFL) is based on risk formulas to rank program elements, which work generally well in various situations. However, it cannot be ruled out that zero division might happen during score calculation, which has negative consequences, e.g., essential elements will not be in the top part of the rank list. The literature has given several strategies to tackle the problem, although there is little knowledge on which one to use. In our work, we performed mathematical analysis and an empirical study to find out how this phenomenon affects SBFL. Results show that division by zero happens in many cases, and the strategies can mitigate their consequences with varying success. Thus, we propose a combined method to avoid the threat of division by zero and improve the trustworthiness of SBFL. Our proposals should be taken into consideration whenever a formula is being used or a new one is proposed.

In traditional Spectrum-Based Fault Localization (SBFL), hit-based spectrum is used to estimate a program element’s suspiciousness to contain a fault, i.e., only the binary information is used if the code element was executed by the test case or not. Count-based spectra can potentially improve the localization effectiveness due to the number of executions also being available. In this work, we use function-level granularity and define count-based spectra which use function call frequencies. We investigate the naïve approach, which simply counts the function call instances. We also define a novel method which is based on counting the different function call contexts, i.e., the frequency of the investigated function occurring in unique call stack instances during test execution. The basic intuition is that if a function is called in many different contexts during a failing test case, it will be more probable to be accountable for the fault. We empirically evaluated the fault localization capability of different variations of the approach and compared them to 9 traditional SBFL techniques using the Defects4J benchmark. We show that: (i) naïve counts result in worse rank positions than the hit-based approach, but (ii) unique counts produce better rank positions with some of the algorithm variants.

Fault localization is one of the most time-consuming and error-prone parts of software debugging. There are several tools for helping developers in the fault localization process, however, they mostly target programs written in Java and C/C++ programming languages. While these tools are splendid on their own, we must not look over the fact that Python is a popular programming language, and still there are a lack of easy-to- use and handy fault localization tools for Python developers. In this paper, we present a tool called “CharmFL” for software fault localization as a plug-in for PyCharm IDE. The tool employs Spectrum-based fault localization (SBFL) to help Python developers automatically analyze their programs and generate useful data at run-time to be used, then to produce a ranked list of potentially faulty program elements (i.e., statements, functions, and classes). Thus, our proposed tool supports different code coverage types with the possibility to investigate these types in a hierarchical approach. The applicability of our tool has been presented by using a set of experimental use cases. The results show that our tool could help developers to efficiently find the locations of different types of faults in their programs.

Spectrum-Based Fault Localization (SBFL), in its basic form, uses only local information about a program element’s (such as a method’s) coverage to predict its faultiness, and rarely is any additional (contextual) information leveraged about the element itself, nor the test cases. As such an additional context, in the presented approach, we rely on the frequency of the investigated method occurring in call stack instances during the course of executing the failing test cases. The basic intuition is that if a method is called in many different contexts during a failing test case, it will be more probable to be accountable for the fault compared to other methods. We empirically evaluated the fault localization capability of the approach compared to five traditional SBFL techniques using the bug benchmark Defects4J. We found that the new algorithms (i) find the location of bugs at higher rank positions more often, (ii) can achieve 38%–52% rank position improvement compared to the baseline algorithms with statistical significance, and (iii) place more items at the top-10 positions of the suspiciousness ranking.

Spectrum-Based Fault Localization (SBFL) is a well-understood statistical approach to software fault localization, and there have been numerous studies performed that tackle its effectiveness. However, mostly Java and C/C++ programs have been addressed to date. We performed an empirical study on SBFL for JavaScript programs using a recent bug benchmark, BugsJS. In particular, we examined (1) how well some of the most popular SBFL algorithms, Tarantula, Ochiai and DStar, can predict the faulty source code elements in these JavaScript programs, (2) whether there is a significant difference between the effectiveness of the different SBFL algorithms, and (3) whether there is any relationship between the bug-fix types and the performance of SBFL methods. For the latter, we performed a manual classification of each benchmark bug according to an existing classification scheme. Results show that the performance of the SBFL algorithms is similar but there are some notable differences among them as well, and that certain bug-fix types can be significantly differentiated from the others (in both positive and negative direction) based on the fault localization effectiveness of the investigated algorithms.