VLDB 2026 Research / reviewers in the wild / expert
David Clark 0001
dblp:40/4629-1
· DBLP profile ↗
49ranked-venue papers
9as first author
18since 2021 · last 2025
0000-0002-7004-934XORCID · conflict
Domains — the database's venue-derived domains; a paper can count in several
Software engineering, systems software and programming languages · 32 · 3 first-author · 13 since 2021Theory of computation · 9 · 5 first-authorArtificial intelligence and machine learning · 8 · 4 since 2021Security and privacy · 4 · 1 first-author · 1 since 2021Databases, data management, data science and information retrieval · 3 · 2 first-author
| Year | Publication | Venue | Position |
|---|---|---|---|
| 2025 | Population Diversity, Information Theory and Genetic Improvement
William B. Langdon, David Clark 0001 |
EuroGP | 2 |
| 2025 | Deep imperative mutations have less impactabstractAbstract Information theory and entropy loss predict deeper more hierarchical software will be more robust. Suggesting silent errors and equivalent mutations will be more common in deeper code, highly structured code will be hard to test, so explaining best practise preference for unit testing of small methods rather than system wide analysis. Using the genetic improvement (GI) tool MAGPIE , we measure the impact of source code mutations and how this varies with execution depth in two diverse multi-level nested software. gem5 is a million line single threaded state-of-the-art C++ discrete time VLSI circuit simulator, whilst PARSEC VIPS is a non-deterministic parallel computing multi-threaded image processing benchmark written in C. More than 28–53% of mutants compile and generate identical results to the original program. We observe 12% and 16% Failed Disruption Propagation (FDP). Excluding internal errors, exceptions and asserts, here most faults below about 30 nested function levels which are Executed and Infect data or divert control flow are not Propagated to the output, i.e. these deep PIE changes have no visible external effect. Suggesting automatic software engineering on highly structured code will be hard. William B. Langdon, David Clark 0001 |
Autom. Softw. Eng. | 2 |
| 2025 | Hyperfuzzing: black-box security hypertesting with a grey-box fuzzerabstractAbstract Despite being a severe error where programs inadvertently reveal confidential information, insecure flows rarely receive explicit attention during software testing. LeakFuzzer uses an input-output non-interference property, specialised via a security flow policy for the program under test, to advance the state of the art. It detects insecure flows by using hypertesting for violations of the program’s non-interference property. LeakFuzzer extends the capabilities of the state of the art fuzzer, AFL++, and thus inherits its advantages such as scalability, automated input generation, high coverage and low developer intervention. It can thus detect the same set of errors as AFL++, as well as being able to detect violations of secure information flow policies at small additional performance costs. This offers a significant advance in scalability and automation for the state of the art. We evaluated LeakFuzzer on a diverse set of 12 C and C++ benchmarks containing known bugs that cause confidential information to be disclosed, ranging in size from just 80 to over 900k lines of code. Nine of these are taken from real-world CVEs including Heartbleed and a recent error in PostgreSQL. Given 20 24-hour runs, LeakFuzzer can find 100% of the insecure flows in the SUTs whereas existing techniques using the CBMC model checker and AFL++ augmented with different sanitizers can only find 40% at best. Daniel Blackwell, Ingolf Becker, David Clark 0001 |
Empir. Softw. Eng. | 3 |
| 2024 | Genetic Improvement of Last Level Cache
William B. Langdon, David Clark 0001 |
EuroGP | 2 |
| 2024 | Mining for Mutation Operators for Reduction of Information Flow Control ViolationsabstractThe unintentional flow of confidential data to unauthorised users is a serious software security vulnerability. Detection and repair of such errors is a non-trivial task that has been worked on by the security community for many years. More recently, dynamic approaches, such as HyperGI, have been introduced that use hypertesting and genetic improvement to not only detect, but also provide a patch that reduces such information flow control violations. However, empirical studies performed so far have used mostly generic mutation operators, potentially limiting the strength of this approach. In this new ideas paper we mine the National Vulnerabilities Database to find repairs of information leaks. Of 636 issues initially identified, we found 73 fixes that relate to information leaks and come with open source patches to the code. From these, we identified 10 types of mutation operators with potential to fix such issues. Six of these have so far never been used to fix information leaks via automated mutation to the code. We propose that these could help improve effectiveness of tools using the HyperGI approach. Ilya Kosorukov, Daniel Blackwell, David Clark 0001, Myra B. Cohen, Justyna Petke |
ASE | 3 |
| 2023 | Genetic Improvement of LLVM Intermediate Representation
William B. Langdon, Afnan A. Al-Subaihin, Aymeric Blot, David Clark 0001 |
EuroGP | 4 |
| 2023 | June: A Type Testability Transformation for Improved ATG PerformanceabstractStrings are universal containers: they are flexible to use, abundant in code, and difficult to test. String-controlled programs are programs that make branching decisions based on string input. Automatically generating valid test inputs for these programs considering only character sequences rather than any underlying string-encoded structures, can be prohibitively expensive. We present June, a tool that enables Java developers to expose any present latent string structure to test generation tools. June is an annotation-driven testability transformation and an extensible library, JuneLib, of structured string definitions. The core JuneLib definitions are empirically derived and provide templates for all structured strings in our test set. June takes lightly annotated source code and injects code that permits an automated test generator (ATG) to focus on the creation of mutable substrings inside a structured string. Using June costs the developer little, with an average of 2.1 annotations per string-controlled class. June uses standard Java build tools and therefore deploys seamlessly within a Java project. By feeding string structure information to an ATG tool, June dramatically reduces wasted effort; branches are effortlessly covered that would otherwise be extremely difficult, or impossible, to cover. This waste reduction both increases and speeds coverage. EvoSuite, for example, achieves the same coverage on June-ed classes in 1 minute, on average, as it does in 9 minutes on the un-June-ed class. These gains increase over time. On our corpus, June-ing a program compresses 24 hours of execution time into ca. 2 hours. We show that many ATG tools can reuse the same June-ed code: a few June annotations, a one-off cost, benefit many different testing regimes. Dan Bruce, David A. Kelly, Héctor D. Menéndez 0001, Earl T. Barr, David Clark 0001 |
ISSTA | 5 |
| 2023 | Verifying Opacity Properties in Security SystemsabstractWe delineate a methodology for the specification and verification of flow security properties expressible in the opacity framework. We propose a logic,${{\sf OpacTL}}$, for straightforwardly expressing such properties in systems that can be modelled as partially observable labelled transition systems. We develop verification techniques for analysing property opacity with respect to observation notions. Adding a probabilistic operator to the specification language enables quantitative analysis and verification. This analysis is implemented as an extension to the PRISM model checker and illustrated via a number of examples. Finally, an alternative approach to quantifying the opacity property based on entropy is sketched. Chunyan Mu, David Clark 0001 |
IEEE Trans. Dependable Secur. Comput. | 2 |
| 2023 | PopArt: Ranked Testing EfficiencyabstractToo often, programmers are under pressure to maximize their confidence in the correctness of their code with a tight testing budget. Should they spend some of that budget on finding “interesting” inputs or spend their entire testing budget on test executions? Work on testing efficiency has explored two competing approaches to answer this question: systematic partition testing (ST), which defines a testing partition and tests its parts, and random testing (RT), which directly samples inputs with replacement. A consensus as to which is better when has yet to emerge. We present Probability Ordered Partition Testing (PopArt), a new systematic partition-based testing strategy that visits the parts of a testing partition in decreasing probability order and in doing so leverages any non-uniformity over that partition. We show how to construct a homogeneous testing partition, a requirement for systematic testing, by using an executable oracle and the path partition. A program's path partition is a naturally occurring testing partition that is usually skewed for the simple reason that some paths execute more frequently than others. To confirm this conventional wisdom, we instrument programs from the Codeflaws repository and find that 80% of them have a skewed path probability distribution.PopArtvisits the parts of a testing partition in decreasing probability order. We then comparePopArtwithRTto characterise the configuration space in which each is more efficient. We show that, when simulating Codeflaws,PopArtoutperformsRTafter$100{,}000$executions. Our results reaffirmRT's power for very small testing budgets but also show that for any application requiring high (above 90%) probability-weighted coveragePopArtshould be preferred. In such cases, despite paying more for each test execution, we prove thatPopArtoutperformsRT: it traverses parts whose cumulative probability bounds that of random testing, showing that sampling without replacement pays for itself, given a nonuniform probability over a testing partition. Iason Papapanagiotakis-Bousy, Earl T. Barr, David Clark 0001 |
IEEE Trans. Software Eng. | 3 |
| 2022 | Measuring failed disruption propagation in genetic programmingabstractInformation theory explains the robustness of deep GP trees, with on average up to 83.3% of crossover run time disruptions failing to propagate to the root node, and so having no impact on fitness, leading to phenotypic convergence. Monte Carlo simulations of perturbations covering the whole tree demonstrate a model based on random synchronisation of the evaluation of the parent and child which cause parent and offspring evaluations to be identical. This predicts the effectiveness of fitness measurement grows slowly as O(log(n)) with number n of test cases. This geometric distribution model is tested on genetic programming symbolic regression. William B. Langdon, Afnan A. Al-Subaihin, David Clark 0001 |
GECCO | 3 |
| 2022 | Keeping Secrets: Multi-objective Genetic Improvement for Detecting and Reducing Information LeakageabstractInformation leaks in software can unintentionally reveal private data, yet they are hard to detect and fix. Although several methods have been proposed to detect leakage, such as static verification-based approaches, they require specialist knowledge, and are time-consuming. Recently, we introduced HyperGI, a dynamic, hypertest-based approach that can detect and produce potential fixes for hyperproperty violations. In particular, we focused on violations of the noninterference property, as it results in information flow leakage. Our instantiation of HyperGI was able to detect and reduce leakage in three small programs. Its fitness function tried to balance information leakage and program correctness but, as we pointed out, there may be tradeoffs between keeping program semantics and reducing information leakage that require developer decisions. Ibrahim Mesecan, Daniel Blackwell, David Clark 0001, Myra B. Cohen, Justyna Petke |
ASE | 3 |
| 2022 | An information theoretic notion of software testability
Krishna Patel, Robert M. Hierons, David Clark 0001 |
Inf. Softw. Technol. | 3 |
| 2022 | Output Sampling for Output Diversity in Automatic Unit Test GenerationabstractDiverse test sets are able to expose bugs that test sets generated with structural coverage techniques cannot discover. Input-diverse test set generators have been shown to be effective for this, but also have limitations: e.g., they need to be complemented with semantic information derived from the Software Under Test. We demonstrate how to drive the test set generation process with semantic information in the form of output diversity. We present the first totally automatic output sampling for output diversity unit test set generation tool, called OutGen. OutGen transforms a program into an SMT formula in bit-vector arithmetic. It then applies universal hashing in order to generate an output-based diverse set of inputs. The result offers significant diversity improvements when measured as a high output uniqueness count. It achieves this by ensuring that the test set’s output probability distribution is uniform, i.e., highly diverse. The use of output sampling, as opposed to any of input sampling, CBMC, CAVM, behaviour diversity or random testing improves mutation score and bug detection by up to 4150 and 963 percent respectively on programs drawn from three different corpora: the R-project, SIR and CodeFlaws. OutGen test sets achieve an average mutation score of up to 92 percent, and 70 percent of the test sets detect the defect. Moreover, OutGen is the only automatic unit test generation tool that is able to detect bugs on the real number C functions from the R-project. Héctor D. Menéndez 0001, Michele Boreale, Daniele Gorla, David Clark 0001 |
IEEE Trans. Software Eng. | 4 |
| 2022 | Hashing Fuzzing: Introducing Input Diversity to Improve Crash DetectionabstractThe utility of a test set of program inputs is strongly influenced by its diversity and its size. Syntax coverage has become a standard proxy for diversity. Although more sophisticated measures exist, such as proximity of a sample to a uniform distribution, methods to use them tend to be type dependent. We use r-wise hash functions to create a novel, semantics preserving, testability transformation for C programs that we call HashFuzz. Use of HashFuzz improves the diversity of test sets produced by instrumentation-based fuzzers. We evaluate the effect of the HashFuzz transformation on eight programs from the Google Fuzzer Test Suite using four state-of-the-art fuzzers that have been widely used in previous research. We demonstrate pronounced improvements in the performance of the test sets for the transformed programs across all the fuzzers that we used. These include strong improvements in diversity in every case, maintenance or small improvement in branch coverage – up to 4.8 perent improvement in the best case, and significant improvement in unique crash detection numbers – between 28 to 97 perent increases compared to test sets for untransformed programs. Héctor D. Menéndez 0001, David Clark 0001 |
IEEE Trans. Software Eng. | 2 |
| 2021 | HyperGI: Automated Detection and Repair of Information Flow LeakageabstractMaintaining confidential information control in soft-ware is a persistent security problem where failure means secrets can be revealed via program behaviors. Information flow control techniques traditionally have been based on static or symbolic analyses — limited in scalability and specialized to particular languages. When programs do leak secrets there are no approaches to automatically repair them unless the leak causes a functional test to fail. We present our vision for HyperGI, a genetic improvement framework that detects, localizes and repairs information leakage. Key elements of HyperGI include (1) the use of two orthogonal test suites, (2) a dynamic leak detection approach which estimates and localizes potential leaks, and (3) a repair component that produces a candidate patch using genetic improvement. We demonstrate the successful use of HyperGI on several programs with no failing functional test cases. We manually examine the resulting patches and identify trade-offs and future directions for fully realizing our vision. Ibrahim Mesecan, Daniel Blackwell, David Clark 0001, Myra B. Cohen, Justyna Petke |
ASE | 3 |
| 2021 | Software robustness: a survey, a theory, and prospectsabstractIf a software execution is disrupted, witnessing the execution at a later point may see evidence of the disruption or not. If not, we say the disruption failed to propagate. One name for this phenomenon is software robustness but it appears in different contexts in software engineering with different names. Contexts include testing, security, reliability, and automated code improvement or repair. Names include coincidental correctness, correctness attraction, transient error reliability. As witnessed, it is a dynamic phenomenon but any explanation with predictive power must necessarily take a static view. As a dynamic/static phenomenon it is convenient to take a statistical view of it which we do by way of information theory. We theorise that for failed disruption propagation to occur, a necessary condition is that the code region where the disruption occurs is composed with or succeeded by a subsequent code region that suffers entropy loss over all executions. The higher is the entropy loss, the higher the likelihood that disruption in the first region fails to propagate to the downstream observation point. We survey different research silos that address this phenomenon and explain how the theory might be exploited in software engineering. Justyna Petke, David Clark 0001, William B. Langdon |
ESEC/SIGSOFT FSE | 2 |
| 2021 | Diversifying Focused Testing for Unit TestingabstractSoftware changes constantly, because developers add new features or modifications. This directly affects the effectiveness of the test suite associated with that software, especially when these new modifications are in a specific area that no test case covers. This article tackles the problem of generating a high-quality test suite to cover repeatedly a given point in a program, with the ultimate goal of exposing faults possibly affecting the given program point. Both search-based software testing and constraint solving offer ready, but low-quality, solutions to this: Ideally, a maximally diverse covering test set is required, whereas search and constraint solving tend to generate test sets with biased distributions. Our approach, Diversified Focused Testing (DFT), uses a search strategy inspired by GödelTest. We artificially inject parameters into the code branching conditions and use a bi-objective search algorithm to find diverse inputs by perturbing the injected parameters, while keeping the path conditions still satisfiable. Our results demonstrate that our technique, DFT, is able to cover a desired point in the code at least 90% of the time. Moreover, adding diversity improves the bug detection and the mutation killing abilities of the test suites. We show that DFT achieves better results than focused testing, symbolic execution, and random testing by achieving from 3% to 70% improvement in mutation score and up to 100% improvement in fault detection across 105 software subjects. Héctor D. Menéndez 0001, Gunel Jahangirova, Federica Sarro, Paolo Tonella, David Clark 0001 |
ACM Trans. Softw. Eng. Methodol. | 5 |
| 2021 | An Empirical Validation of Oracle ImprovementabstractWe propose a human-in-the-loop approach for oracle improvement and analyse whether the proposed oracle improvement process is helping developers to create better oracles. For this, we conducted two human studies with 68 participants overall: an oracle assessment study and an oracle improvement study. Our results show that developers exhibit poor performance (29 percent accuracy) when manually assessing whether an assertion oracle contains a false positive, a false negative or none of the two. This shows that automated detection of these oracle deficiencies is beneficial for the users. Our tool OASIs (Oracle ASsessment and Improvement) helps developers produce assertions with higher quality. Participants who used OASIs in the improvement study were able to achieve 33 percent of full and 67 percent of partial correctness as opposed to participants without the tool who achieved only 21 percent of full and 43 percent of partial correctness. Gunel Jahangirova, David Clark 0001, Mark Harman, Paolo Tonella |
IEEE Trans. Software Eng. | 2 |
| 2019 | Directing a Search Towards Execution Properties with a Learned Fitness FunctionabstractSearch based software testing is a popular and successful approach both in academia and industry. SBST methods typically aim to increase coverage whereas searching for executions with specific properties is largely unresearched. Fitness functions for execution properties often possess search landscapes that are difficult or intractable. We demonstrate how machine learning techniques can convert a property that is not searchable, in this case crashes, into one that is. Through experimentation on 6000 C programs drawn from the Codeflaws repository, we demonstrate a strong, program independent correlation between crashing executions and library function call patterns within those executions as discovered by a neural net. We then exploit the correlation to produce a searchable fitness landscape to modify American Fuzzy Lop, a widely used fuzz testing tool. On a test set of previously unseen programs drawn from Codeflaws, a search strategy based on a crash targeting fitness function outperformed a baseline in 80.1% of cases. The experiments were then repeated on three real world programs: the VLC media player, and the libjpeg and mpg321 libraries. The correlation between library call traces and crashes generalises as indicated by ROC AUC scores of 0.91, 0.88 and 0.61. The produced search landscape however is not convenient due to plateaus. This is likely because these programs do not use standard C libraries as often as do those in Codeflaws. This limitation can be overcome by considering a more powerful observation domain and a broader training corpus in future work. Despite limited generalisability of the experimental setup, this research opens new possibilities in the intersection of machine learning, fitness functions, and search based testing in general. Leonid Joffe, David Clark 0001 |
ICST | 2 |
| 2019 | Dorylus: An Ant Colony Based Tool for Automated Test Case Generation
Dan Bruce, Héctor D. Menéndez 0001, David Clark 0001 |
SSBSE | 3 |
| 2019 | Constructing Search Spaces for Search-Based Software Testing Using Neural Networks
Leonid Joffe, David Clark 0001 |
SSBSE | 2 |
| 2019 | The arms race: Adversarial search defeats entropy used to detect malwareabstractMalware creators have been getting their way for too long now. String-based similarity measures can leverage ground truth in a scalable way and can operate at a level of abstraction that is difficult to combat from the code level. At the string level, information theory and, specifically, entropy play an important role related to detecting patterns altered by concealment strategies, such as polymorphism or encryption. Controlling the entropy levels in different parts of a disk resident executable allows an analyst to detect malware or a black hat to evade the detection. This paper shows these two perspectives into two scalable entropy-based tools: EnTS and EEE. EnTS, the detection tool, shows the effectiveness of detecting entropy patterns, achieving 100% precision with 82% accuracy. It outperforms VirusTotal for accuracy on combined Kaggle and VirusShare malware. EEE, the evasion tool, shows the effectiveness of entropy as a concealment strategy, attacking binary-based state of the art detectors. It learns their detection patterns in up to 8 generations of its search process, and increments their false negative rate from range 0–9%, up to the range 90–98.7%. Héctor D. Menéndez 0001, Sukriti Bhattacharya, David Clark 0001, Earl T. Barr |
Expert Syst. Appl. | 3 |
| 2019 | Normalised Squeeziness and Failed Error Propagation
David Clark 0001, Robert M. Hierons, Krishna Patel |
Inf. Process. Lett. | 1 |
| 2018 | OASIs: oracle assessment and improvement toolabstractThe oracle problem remains one of the key challenges in software testing, for which little automated support has been developed so far. We introduce OASIs, a search-based tool for Java that assists testers in oracle assessment and improvement. It does so by combining test case generation to reveal false positives and mutation testing to reveal false negatives. In this work, we describe how OASIs works, provide details of its implementation, and explain how it can be used in an iterative oracle improvement process with a human in the loop. Finally, we present a summary of previous empirical evaluation showing that the fault detection rate of the oracles after improvement using OASIs increases, on average, by 48.6%. Gunel Jahangirova, David Clark 0001, Mark Harman, Paolo Tonella |
ISSTA | 2 |
| 2018 | A comparison of code similarity analysersabstractCopying and pasting of source code is a common activity in software engineering. Often, the code is not copied as it is and it may be modified for various purposes; e.g. refactoring, bug fixing, or even software plagiarism. These code modifications could affect the performance of code similarity analysers including code clone and plagiarism detectors to some certain degree. We are interested in two types of code modification in this study: pervasive modifications, i.e. transformations that may have a global effect, and local modifications, i.e. code changes that are contained in a single method or code block. We evaluate 30 code similarity detection techniques and tools using five experimental scenarios for Java source code. These are (1) pervasively modified code, created with tools for source code and bytecode obfuscation, and boiler-plate code, (2) source code normalisation through compilation and decompilation using different decompilers, (3) reuse of optimal configurations over different data sets, (4) tool evaluation using ranked-based measures, and (5) local + global code modifications. Our experimental results show that in the presence of pervasive modifications, some of the general textual similarity measures can offer similar performance to specialised code similarity tools, whilst in the presence of boiler-plate code, highly specialised source code similarity detection techniques and tools outperform textual similarity measures. Our study strongly validates the use of compilation/decompilation as a normalisation technique. Its use reduced false classifications to zero for three of the tools. Moreover, we demonstrate that optimal configurations are very sensitive to a specific data set. After directly applying optimal configurations derived from one data set to another, the tools perform poorly on the new data set. The code similarity analysers are thoroughly evaluated not only based on several well-known pair-based and query-based error measures but also on each specific type of pervasive code modification. This broad, thorough study is the largest in existence and potentially an invaluable guide for future users of similarity detection in source code. Chaiyong Ragkhitwetsagul, Jens Krinke, David Clark 0001 |
Empir. Softw. Eng. | 3 |
| 2018 | Picking on the family: Disrupting android malware triage by forcing misclassificationabstractMachine learning classification algorithms are widely applied to different malware analysis problems because of their proven abilities to learn from examples and perform relatively well with little human input. Use cases include the labelling of malicious samples according to families during triage of suspected malware. However, automated algorithms are vulnerable to attacks. An attacker could carefully manipulate the sample to force the algorithm to produce a particular output. In this paper we discuss one such attack on Android malware classifiers. We design and implement a prototype tool, called IagoDroid, that takes as input a malware sample and a target family, and modifies the sample to cause it to be classified as belonging to this family while preserving its original semantics. Our technique relies on a search process that generates variants of the original sample without modifying their semantics. We tested IagoDroid against RevealDroid, a recent, open source, Android malware classifier based on a variety of static features. IagoDroid successfully forces misclassification for 28 of the 29 representative malware families present in the DREBIN dataset. Remarkably, it does so by modifying just a single feature of the original malware. On average, it finds the first evasive sample in the first search iteration, and converges to a 100% evasive population within 4 iterations. Finally, we introduce RevealDroid*, a more robust classifier that implements several techniques proposed in other adversarial learning domains. Our experiments suggest that RevealDroid* can correctly detect up to 99% of the variants generated by IagoDroid. Alejandro Calleja, Alejandro Martín, Héctor D. Menéndez 0001, Juan Tapiador, David Clark 0001 |
Expert Syst. Appl. | 5 |
| 2016 | Test Set Diameter: Quantifying the Diversity of Sets of Test CasesabstractA common and natural intuition among software testers is that test cases need to differ if a software system is to be tested properly and its quality ensured. Consequently, much research has gone into formulating distance measures for how test cases, their inputs and/or their outputs differ. However, common to these proposals is that they are data type specific and/or calculate the diversity only between pairs of test inputs, traces or outputs. We propose a new metric to measure the diversity of sets of tests: the test set diameter (TSDm). It extends our earlier, pairwise test diversity metrics based on recent advances in information theory regarding the calculation of the normalized compression distance (NCD) for multisets. A key advantage is that TSDm is a universal measure of diversity and so can be applied to any test set regardless of data type of the test inputs (and, moreover, to other test-related data such as execution traces). But this universality comes at the cost of greater computational effort compared to competing approaches. Our experiments on four different systems show that the test set diameter can help select test sets with higher structural and fault coverage than random selection even when only applied to test inputs. This can enable early test design and selection, prior to even having a software system to test, and complement other types of test automation and analysis. We argue that this quantification of test set diversity creates a number of opportunities to better understand software quality and provides practical ways to increase it. Robert Feldt, Simon M. Poulding, David Clark 0001, Shin Yoo |
ICST | 3 |
| 2016 | Test oracle assessment and improvementabstractWe introduce a technique for assessing and improving test oracles by reducing the incidence of both false positives and false negatives. We prove that our approach can always result in an increase in the mutual information between the actual and perfect oracles. Our technique combines test case generation to reveal false positives and mutation testing to reveal false negatives. We applied the decision support tool that implements our oracle improvement technique to five real-world subjects. The experimental results show that the fault detection rate of the oracles after improvement increases, on average, by 48.6% (86% over the implicit oracle). Three actual, exposed faults in the studied systems were subsequently confirmed and fixed by the developers. Gunel Jahangirova, David Clark 0001, Mark Harman, Paolo Tonella |
ISSTA | 2 |
| 2016 | Similarity of Source Code in the Presence of Pervasive ModificationsabstractSource code analysis to detect code cloning, code plagiarism, and code reuse suffers from the problem of pervasive code modifications, i.e. transformations that may have a global effect. We compare 30 similarity detection techniques and tools against pervasive code modifications. We evaluate the tools using two experimental scenarios for Java source code. These are (1) pervasive modifications created with tools for source code and bytecode obfuscation and (2) source code normalisation through compilation and decompilation using different decompilers. Our experimental results show that highly specialised source code similarity detection techniques and tools can perform better than more general, textual similarity measures. Our study strongly validates the use of compilation/decompilation as a normalisation technique. Its use reduced false classifications to zero for six of the tools. This broad, thorough study is the largest in existence and potentially an invaluable guide for future users of similarity detection in source code. Chaiyong Ragkhitwetsagul, Jens Krinke, David Clark 0001 |
SCAM | 3 |
| 2015 | Information Transformation: An Underpinning Theory for Software EngineeringabstractSoftware engineering lacks underpinning scientific theories both for the software it produces and the processes by which it does so. We propose that an approach based on information theory can provide such a theory, or rather many theories. We envision that such a benefit will be realised primarily through research based on the quantification of information involved and a mathematical study of the limiting laws that arise. However, we also argue that less formal but more qualitative uses for information theory will be useful. The main argument in support of our vision is based on the fact that both a program and an engineering process to develop such a program are fundamentally processes that transform information. To illustrate our argument we focus on software testing and develop an initial theory in which a test suite is input/output adequate if it achieves the channel capacity of the program as measured by the mutual information between its inputs and its outputs. We outline a number of problems, metrics and concrete strategies for improving software engineering, based on information theoretical analyses. We find it likely that similar analyses and subsequent future research to detail them would be generally fruitful for software engineering. David Clark 0001, Robert Feldt, Simon M. Poulding, Shin Yoo |
ICSE (2) | 1 |
| 2015 | A semiring-based trace semantics for processes with applications to information leakage analysisabstractWe propose a framework for reasoning about program security building on language-theoretic and coalgebraic concepts. The behaviour of a system is viewed as a mapping from traces of high (unobservable) events to low (observable) events: the less the degree of dependency of low events on high traces, the more secure the system. We take the abstract view that low events are drawn from a generic semiring, where they can be combined using product and sum operations; throughout the paper, we provide instances of this framework, obtained by concrete instantiations of the underlying semiring. We specify systems via a simple process calculus, whose semantics is given as the unique homomorphism from the calculus into the set of behaviours, i.e. formal power series, seen as a final coalgebra. We provide a compositional semantics for the calculus in terms of rational operators on formal power series and show that the final and the compositional semantics coincide. This compositional, syntax-driven framework lays a foundation for automation and abstraction of a quantified approach to flow security of system specifications. Michele Boreale, David Clark 0001, Daniele Gorla |
Math. Struct. Comput. Sci. | 2 |
| 2014 | An analysis of the relationship between conditional entropy and failed error propagation in software testingabstractFailed error propagation (FEP) is known to hamper software testing, yet it remains poorly understood. We introduce an information theoretic formulation of FEP that is based on measures of conditional entropy. This formulation considers the situation in which we are interested in the potential for an incorrect program state at statement s to fail to propagate to incorrect output. We define five metrics that differ in two ways: whether we only consider parts of the program that can be reached after executing s and whether we restrict attention to a single program path of interest .We give the results of experiments in which it was found that on average one in 10 tests suffered from FEP, earlier studies having shown that this figure can vary significantly between programs. The experiments also showed that our metrics are well-correlated with FEP. Our empirical study involved 30 programs, for which we executed a total of 7,140,000 test cases. The results reveal that the metrics differ in their performance but the Spearman rank correlation with failed error propagation is close to 0.95 for two of the metrics. These strong correlations in an experimental setting, in which all information about both FEP and conditional entropy is known, open up the possibility in the longer term of devising inexpensive information theory based metrics that allow us to minimise the effect of FEP. Kelly Androutsopoulos, David Clark 0001, Haitao Dan, Robert M. Hierons, Mark Harman |
ICSE | 2 |
| 2013 | Fault localization prioritization: Comparing information-theoretic and coverage-based approachesabstractTest case prioritization techniques seek to maximize early fault detection. Fault localization seeks to use test cases already executed to help find the fault location. There is a natural interplay between the two techniques; once a fault is detected, we often switch focus to fault fixing, for which localization may be a first step. In this article we introduce the Fault Localization Prioritization (FLP) problem, which combines prioritization and localization. We evaluate three techniques: a novel FLP technique based on information theory, FLINT (Fault Localization using INformation Theory), that we introduce in this article, a standard Test Case Prioritization (TCP) technique, and a “test similarity technique” used in previous work. Our evaluation uses five different releases of four software systems. The results indicate that FLP and TCP can statistically significantly reduce fault localization costs for 73% and 76% of cases, respectively, and that FLINT significantly outperforms similarity-based localization techniques in 52% of the cases considered in the study. Shin Yoo, Mark Harman, David Clark 0001 |
ACM Trans. Softw. Eng. Methodol. | 3 |
| 2013 | Amorphous Slicing of Extended Finite State MachinesabstractSlicing is useful for many software engineering applications and has been widely studied for three decades, but there has been comparatively little work on slicing extended finite state machines (EFSMs). This paper introduces a set of dependence-based EFSM slicing algorithms and an accompanying tool. We demonstrate that our algorithms are suitable for dependence-based slicing. We use our tool to conduct experiments on 10 EFSMs, including benchmarks and industrial EFSMs. Ours is the first empirical study of dependence-based program slicing for EFSMs. Compared to the only previously published dependence-based algorithm, our average slice is smaller 40 percent of the time and larger only 10 percent of the time, with an average slice size of 35 percent for termination insensitive slicing. Kelly Androutsopoulos, David Clark 0001, Mark Harman, Robert M. Hierons, Zheng Li 0002, Laurence Tratt |
IEEE Trans. Software Eng. | 2 |
| 2012 | Squeeziness: An information theoretic measure for avoiding fault masking
David Clark 0001, Robert M. Hierons |
Inf. Process. Lett. | 1 |
| 2011 | Model projection: simplifying models in response to restricting the environmentabstractThis paper introduces Model Projection. Finite state models such as Extended Finite State Machines are being used in an ever increasing number of software engineering activities. Model projection facilitates model development by specializing models for a specific operating environment. A projection is useful in many design-level applications including specification reuse and property verification. Kelly Androutsopoulos, Dave W. Binkley, David Clark 0001, Nicolas E. Gold, Mark Harman, Kevin Lano, Zheng Li 0002 |
ICSE | 3 |
| 2011 | Foreword: programming language interference and dependenceabstractInterference and dependence are closely related concepts: interference being the observable phenomenon connected with dependence. Essentially, interference means that the behaviour of some parts of a dynamic system may influence the behaviour of other parts of the same system, while dependence specifies how the semantics of sub-components of a dynamic system are related. Identifying, measuring and controlling interference is essential in many aspects of modern computer science, in particular, in security, program analysis and verification, debugging, systems specification, model checking, program manipulation, program slicing, reverse engineering, data mining, distributed databases and systems biology. In all these fields, dependency and interference play a key role in designing suitable abstractions or in partitioning complex systems into simpler ones. Reasoning about dependency and interference requires theories, models and semantics, as well as algorithms and tools for their analysis. Beginning in 2004, the series of Programming Language Interference and Dependence (PLID) workshops has been devoted to promoting and spreading cutting-edge research in this field, with a particular emphasis on unpublished results with great impact on the theoretical basis. PLID2007, which was held at the The Technical University of Denmark on 21 August 2007, was particularly successful, and constituted the ideal forum for announcing a call for papers for a special journal issue on programming language interference and dependence, which would not necessarily be restricted to PLID2007 contributions. From the many expressions of interest, we selected six contributions by leading researchers in the field, some of which had been presented at the PLID2007 workshop. The selected papers focus on foundational aspects of dependency and interference, with applications in language-based security, data-base management systems and program slicing. David Clark 0001, Roberto Giacobazzi, Chunyan Mu |
Math. Struct. Comput. Sci. | 1 |
| 2010 | An alternative characterization of weak order dependence
Torben Amtoft, Kelly Androutsopoulos, David Clark 0001, Mark Harman, Zheng Li 0002 |
Inf. Process. Lett. | 3 |
| 2009 | Quantitative Analysis of Secure Information Flow via Probabilistic SemanticsabstractWe present an automatic analyzer for measuring information flow within software systems. In this paper, we quantify leakage in terms of information theory and incorporate this computation into probabilistic semantics. Our semantic functions provide information flow measurement for programs given secure inputs under any probability distribution. The major contribution is an automatic quantitative analyzer based on the leakage definition for such a language. While-loops are handled by applying entropy of generalized distributions and related properties in order to provide the analysis with the ability to incorporate the observation of elapsed time. Chunyan Mu, David Clark 0001 |
ARES | 2 |
| 2009 | Control Dependence for Extended Finite State Machines
Kelly Androutsopoulos, David Clark 0001, Mark Harman, Zheng Li 0002, Laurence Tratt |
FASE | 2 |
| 2007 | A static analysis for quantifying information flow in a simple imperative languageabstractWe propose an approach to quantify interference in a simple imperative language that includes a looping construct. In this paper we focus on a particular case of this definition of interference: leakage of information from private variables to public ones via a Trojan Horse attack. We quantify leakage in terms of Shannon’s information theory and we motivate our definition by proving a result relating this definition of leakage and the classical notion of programming language interference. The major contribution of the paper is a quantitative static analysis based on this definition for such a language. The analysis uses some non-trivial information theory results like Fano’s inequality and the [Formula: see text] inequality to provide reasonable bounds for conditional statements. While-loops are handled by integrating a qualitative flow-sensitive dependency analysis into the quantitative analysis. David Clark 0001, Sebastian Hunt, Pasquale Malacaria |
J. Comput. Secur. | 1 |
| 2005 | Quantitative Information Flow, Relations and Polymorphic TypesabstractThis paper uses Shannon's information theory to give a quantitative definition of information flow in systems that transform inputs to outputs. For deterministic systems, the definition is shown to specialize to a simpler form when the information source and the known inputs jointly determine all inputs uniquely. For this special case, the definition is related to the classical security condition of non-interference and an equivalence is established between non-interference and independence of random variables. Quantitative information flow for deterministic systems is then presented in relational form. With this presentation, it is shown how relational parametricity can be used to derive upper and lower bounds on information flows through families of functions defined in the second-order lambda calculus. David Clark 0001, Sebastian Hunt, Pasquale Malacaria |
J. Log. Comput. | 1 |
| 2004 | UML to B: Formal Verification of Object-Oriented Models
Kevin Lano, David Clark 0001, Kelly Androutsopoulos |
IFM | 2 |
| 2002 | From Implicit Specifications to Explicit Designs in Reactive System Development
Kevin Lano, David Clark 0001, Kelly Androutsopoulos |
IFM | 2 |
| 2002 | Safety and Security Analysis of Object-Oriented Models
Kevin Lano, David Clark 0001, Kelly Androutsopoulos |
SAFECOMP | 2 |
| 2002 | Information flow for Algol-like languages
David Clark 0001, Chris Hankin, Sebastian Hunt |
Comput. Lang. Syst. Struct. | 1 |
| 2000 | Structuring and Design of Reactive Systems Using RSDS and B
Kevin Lano, Kelly Androutsopoulos, David Clark 0001 |
FASE | 3 |
| 2000 | Safety of Strictness Analysis via Term Graph Rewriting
David Clark 0001, Chris Hankin, Sebastian Hunt |
SAS | 1 |
| 1996 | Event Structures and Non-Orthogonal Term Graph RewritingabstractWe show that for every term graph in a left-linear but non-orthogonal term graph rewrite system, one can construct an event structure that represents all the possible reductions that can occur in reduction sequences starting from that term graph. Every finite reduction sequence from that graph corresponds to a configuration of the event structure, and Lévy-equivalent sequences correspond to the same configuration. Garbage collection is modelled in the event structure by an ‘erases’ relation. The asymmetric conflicts that arise in non-orthogonal rewrite systems are modelled by introducing a ‘prevents’ relation. The configurations of the event structure then form the state space of an event automaton. Taking the directed completion of this space yields a prime algebraic domain. David Clark 0001, Richard Kennaway |
Math. Struct. Comput. Sci. | 1 |