Mispredictions by machine learning components can have severe consequences, especially in safety-critical and mission-critical software systems. Therefore, understanding and debugging these mispredictions is a crucial part of the development process for systems that use machine learning components. Previous research has successfully applied methods that identify when a model’s predictions may be unreliable by generating a rule set that links feature values to prediction errors. However, current state-of-the-art rule set approaches require significant computational resources, particularly for large data setsTo address these high computational demands, we propose a strategy to identify and focus only on the most influential features that lead to mispredictions. Additionally, to improve the accuracy of mispredictions diagnosis, we replace traditional rule-based approaches with decision tree learning. We evaluate our tool MMDFAST across 11 diverse real-world data sets. The results show that focusing on influential features with decision trees improves the accuracy of misprediction explanations, while significantly reducing computational demands in all scenarios. Thus, MMDFAST produces better results much faster, making it more efficient and effective for generating misprediction explanations.

Program fixes must preserve passing tests while fixing failing ones. Validating these properties requires test oracles that distinguish passing from failing runs.We introduce BASHIRI, a tool that learns failure oracles from test suites with labeled outcomes using execution features. BASHIRI leverages execution-feature-driven debugging to collect program execution features and trains interpretable models as testing oracles. Our evaluation shows that BASHIRI predicts test outcomes with 95% accuracy, effectively identifying failing runs. BASHIRI is available as an open-source tool at https://github.com/smythi93/bashiriA demonstration video is available at https://youtu.be/D2mJkCtSXtM

In recent years, automatic program repair has gained much attention in the research community. Generally, program repair approaches consider a faulty program and a test suite that captures the program's intended behavior. The goal is automatically generating a patch that corrects the fault by identifying the faulty code locations, suggesting a candidate fix, and validating it against the provided tests. However, most existing program repair tools focus on Java or C programs, while Python, one of the most popular programming languages, lacks approaches that work on it.

Why does my program fail? We present a novel and general technique to automatically determine failure causes and conditions, using logical properties over input elements: “The program fails if and only if int( ) > len( ) holds—that is, the given is larger than the length.” Our AVICENNA prototype uses modern techniques for inferring properties of passing and failing inputs and validating and refining hypotheses by having a constraint solver generate supporting test cases to obtain such diagnoses. As a result, AVICENNA produces crisp and expressive diagnoses even for complex failure conditions, considerably improving over the state of the art with diagnoses close to those of human experts.

Programs fail. But which part of the input is responsible for the failure? To resolve the issue, developers must first understand how and why the program behaves as it does, notably when it deviates from the expected outcome. A program’s behavior is essentially the set of all its executions. This set is usually diverse, unpredictable, and generally unbounded. A pathological program behavior occurs once the actual outcome does not match the expected behavior. Consequently, developers must fix these issues to ensure the built system is the desired software. In our upcoming research, we want to focus on providing developers with a detailed description of the root causes that resulted in the program’s unwanted behavior. Thus, we aim to automatically produce explanations that capture the circumstances of arbitrary program behavior by correlating individual input elements (features) and their corresponding execution outcome. To this end, we use the scientific method and combine generative and predictive models, allowing us (i) to learn the statistical relations between the features of the inputs and the program behavior and (ii) to generate new inputs to refine or refute our current explanatory prediction model.