Policy as Code (PaC) is an emerging DevOps practice that enables teams to specify organisational and technical policies, such as regulatory compliance, security requirements, and resource limits, through machine-enforceable declarative code. As PaC gains prominence, practitioners face difficulties in adopting PaC while there remains a limited empirical understanding of how these policies are introduced, what types can be expressed, and how they are maintained in practice.

Web GUI testing is a quality assessment practice aimed at evaluating the functionality of web applications from the perspective of its end users. While prior studies have explored the technical challenges of automated Web GUI testing, fewer works have explored how this practice is applied in real-world web apps. This study aims to investigate the adoption, characteristics, and maintenance of automated web GUI testing practices in open-source web applications, focusing on identifying trends and providing actionable insights for researchers and practitioners. We conducted a large-scale empirical analysis of 472 web applications on the GitHub platform, developed in Java , JavaScript , Python , and TypeScript . These projects use popular browser automation frameworks like Selenium , Playwright , Cypress , and Puppeteer . The study involved examining project characteristics and analyzing the co-evolution and maintenance of automated web GUI tests over time. Our findings empirically document automated web GUI testing adoption patterns in open-source projects, providing insights into the practical drivers behind both initial framework adoption and migration between different testing frameworks. Projects incorporating these tests generally show higher community engagement and consistent maintenance efforts. The analysis reveals that Web GUI tests tend to co-evolve with the underlying applications, reflecting their integration into the development lifecycle. The study provides valuable insights into the prevalence and maintenance of Web GUI testing, highlighting practical implications for improving testing practices. Our findings can guide further research on the matter and support practitioners in enhancing their testing strategies.

Micro-service architectures remain a popular choice for cloud applications. However, micro-service applications are subject to faults such as dropped requests between services which are difficult to catch during development in testing environments. Automated testing tools have been proposed to test the resilience of micro-service applications against faults in testing, but these require manually specifying execution scenarios. In this paper, we propose an approach to resilience testing which integrates fault injection into the fuzzing process. In this way, system executions and faults can be simultaneously explored, allowing fully automated resilience testing. We describe our approach as well as a prototype implementation called Onweer, and evaluate its performance on a standard micro-service benchmark application.

Performance testing is crucial to ensuring that web applications meet user expectations under varying workloads. Activities such as stress, load, and smoke testing are designed to simulate different kinds of simultaneous user interactions and assess system behavior. Despite its recognized importance in quality assurance of large-scale web-based systems, witnessed by numerous studies proposing solutions to support these activities, the real-world adoption and evolutionary dynamics of performance tests have received limited attention in the literature. To fill this gap, we analyzed 77 open-source web projects using Apache JMETER and LOCUST. Our study investigates how performance tasks are performed (adoption time, load design, types of tasks), the characteristics of projects that adopt them, and their longterm maintenance. Our findings reveal that performance tests in open-source projects are simple, with a focus on singleuser behaviors and minimal requests, and most tests have low concurrency. Load tests are the most common, followed by smoke and stress tests. Projects with performance tests tend to be larger and more actively maintained. However, tests are mostly long-lived but rarely updated, suggesting potential risks to their relevance and coverage over time. Finally, by creating a taxonomy of performance test changes, we observe recurring patterns of modifications, including workload adjustments, network request changes, and updates to system monitoring.

End-to-end (E2E) testing is a software validation approach that simulates realistic user scenarios throughout the entire workflow of an application. In the context of web applications, E2E testing involves two activities: Graphic User Interface (GUI) testing, which simulates user interactions with the web app’s GUI through web browsers, and performance testing, which evaluates system workload handling. Despite its recognized importance in delivering high-quality web applications, the availability of large-scale datasets featuring real-world E2E web tests remains limited, hindering research in the field.To address this gap, we present E2EGit, a comprehensive dataset of non-trivial open-source web projects collected on GitHub that adopt E2E testing. By analyzing over 5,000 web repositories across popular programming languages (Java, JavaScript, TypeScript and Python), we identified 472 repositories implementing 43,670 automated Web GUI tests with popular browser automation frameworks (Selenium, Playwright, Cypress, Puppeteer), and 84 repositories that featured 271 automated performance tests implemented leveraging the most popular open-source tools (JMeter, LoCust). Among these, 13 repositories implemented both types of testing for a total of 786 Web GUI tests and 61 performance tests. The dataset is available on Zenodo (DOI: 10.5281/zenodo.14234731).

The digital infrastructures supporting modern software have grown too complex to manage by hand. Therefore, Infrastructure as Code (IaC) has become a widely adopted practice to programmatically automate deploying such infrastructures. As infrastructure code may rely on third-party libraries and packages, understanding the software supply chains generated by these deployment dependencies is crucial to ensure reproducibility and security of software deployments. Nonetheless, deployment software supply chains remain an understudied topic. This paper aims to bridge this gap by first investigating which types of third-party software IaC may depend upon, then building an automated mechanism to identify such dependencies from infrastructure implementations. We focus our investigation on Ansible, one of the most popular IaC tools, and its plugins, which implement the interactions with the deployment platforms under configuration. From a manual analysis of 266 documented third-party requirements of Ansible plugins, we construct a taxonomy of 7 types of third-party software dependencies and their properties. We also found that a plugin's dependencies are typically only described informally in the plugin's documentation, which may be unstructured, incorrect, or incomplete, which encumbers the automatic generation of Software Bills of Materials (SBOMs) for deployment code. Therefore, we design an automated Software Composition Analysis (SCA) that extracts these dependencies from an Ansible plugin's implementation, leveraging 5 dependency implementation patterns identified in our manual analysis. This approach achieves a recall of 61 %-77% and a precision of 74%-95%. Finally, we apply the SCA in a large-scale quantitative experiment on 11,241 plugins, and find that 38% have third-party dependencies. The taxonomy presented in this paper can serve as a reference to design deployment SBOMs for these plugins, whereas our SCA forms a first step towards automatically generating such SBOMs.

Infrastructure as Code is an emerging paradigm to automate the configuration of cloud infrastructures. Infrastructure code often processes secret information, such as passwords or private keys. Mishandling such secrets can lead to information disclosure vulnerabilities, yet existing efforts to detect them rely on pattern matching of parameter and variable names, causing false positives and negatives due to suboptimal string patterns.This paper aims to address these limitations by assessing the effectiveness of traditional Machine Learning (ML) and transformer-based Language Model (LM) classifiers to predict sensitive module parameters in Ansible, one of the most popular IaC tools. We collect a dataset of over 160,000 Ansible module parameters and their documentation, containing more than 16,000 parameters that expect secret data. Then, we train several ML algorithms and find that the Random Forest algorithm performs best, achieving 93.5% precision but limited recall (72.7%). In parallel, we evaluate multiple pretrained zero-shot language models, which achieve a recall of up to 90.4% at the expense of a lower precision of up to 88.5%. We subsequently fine-tune the language models, resulting in nearly perfect precision (99.8%) and recall (99.8%) on the ground truth dataset.We compare the best performing ML and LM classifiers to two baselines that use string patterns. We find that the ML classifier achieves a performance comparable to the two baselines, while the fine-tuned LM outperforms all approaches. A qualitative comparison reveals that the approaches are complementary to the baselines, motivating future work to use prediction models to reduce false positives in reports generated by inexpensive baselines. However, we also find that the fine-tuned LM misses several secrets caused by noise in the dataset, highlighting the importance of fine-tuning on a high-quality ground truth.

Nowadays, many developers heavily rely on feedback from bug-detection tools to help ensure the quality of the code they produce. Such tools are underlain by static analysis. It is, however, critical for analysis results to be produced fast. To this end, incremental static analysis can be used. Upon a program change, an incremental analysis updates the previous results rather than recomputing the results from scratch.Incremental static analyses may suffer from cyclic reinforcement of lattice values, where the computation of some values within the analysis relies on the values themselves, due to the abstractions made by the analysis. This can cause the incremental analysis to produce less precise results, reducing its usability.In this work, we provide a solution to cyclic reinforcement of lattice values for incremental dependency-driven analyses. We compute the information flow within the analysis and show how this information flow can be used to detect cyclic reinforcements. We establish a criterion to detect when a cyclic reinforcement contains outdated information that needs to be removed, and show how precision can be regained. Our results show that using our method, an incremental analysis produces results matching a from-scratch analysis for all but one benchmark program, at the cost of a performance hit in some cases.

The WebAssembly standard aims to form a portable compilation target, enabling the cross-platform distribution of programs written in a variety of languages. This paper introduces and evaluates novel slicing approaches for WebAssembly, including dynamic and hybrid approaches. Given a program and a location in that program, a program slice is a reduced program that preserves the behavior at the given location. A static slice does so for all possible inputs, while a dynamic slice does so for a fixed set of inputs. Hybrid slicing is a combination of static and dynamic slicing. We build on Observational-Based Slicing (ORBS), where we explore the design space for instantiating ORBS for WebAssembly. For example, ORBS can be applied to the whole program or to only the function containing the slicing criterion, and it can be applied before compilation to WebAssembly or afterwards. We evaluate the slices produced using various options quantitatively and qualitatively. Our evaluation reveals that dynamic slicing at the level of a function from a WebAssembly binary finds a sweet spot in terms of slice time and slice size, and that a combination of static and dynamic slicers achieves the best trade-off in terms of slicing time and slice size.

In property-based testing (PBT), developers specify properties that they expect the system under test to hold. The PBT tool generates random inputs for the system and tests for each of these inputs whether the given property holds. An advantage of this approach over testing a set of manually defined example inputs is that it enables a higher code coverage. Machine learning (ML) projects, however, often have to process large amounts of diverse data, both for training a model and afterwards, when the trained model is deployed. Generating a sufficient amount of diverse data for the property-based tests is therefore challenging. In this paper, we present the results of a preliminary study in which we examined a dataset of 58 open-source ML projects that have dependencies on the popular PBT library Hypothesis, to identify issues faced by developers writing property-based tests. For a subset of 28 open-source ML projects, we study the property-based tests in detail and report on the part of the ML project that is being tested as well as on the adopted data generation strategies. This way, we aim to identify issues in porting current PBT techniques to ML projects so that they can be addressed in the future.

Abstract “In Silico” research drives the world around us, as illustrated by the way our society handles climate change, controls the COVID-19 pandemic and governs economic growth. Unfortunately, the code embedded in the underlying data processing is mostly written by scientists lacking formal training in software engineering. The resulting code is vulnerable, suffering from what is known as threats to instrument validity. This position paper aims to understand and remedy threats to instrument validity in current “in silico” research. To achieve this goal, we specify a research agenda listing how recent software engineering achievements may improve “in silico” research (SE4Silico) and, conversely, how software engineering may strengthen its applicability (Silico4SE).

Actor languages model concurrency as processes that communicate through asynchronous message sends. Unfortunately, as the complexity of these systems increases, it becomes more difficult to compose and integrate their components. This is because of assumptions made by components about their communication partners which may not be upheld when they remain implicit. In this paper, we bring design-by-contract programming to actor programs through a contract system that enables expressing constraints on receiver-related properties. Expressing properties about the expected receiver of a message, and about this receiver’s communication behavior, requires two novel types of contracts. Through their recursive structure, these contracts can govern entire communication chains. We implement the contract system for an actor extension of Scheme, describe it formally, and show how to assign blame in case of a contract violation. Finally, we prove our contract system and its blame assignment correct by formulating and proving a blame correctness theorem.

Progress has recently been made on specifying instruction set architectures (ISAs) in executable formalisms rather than through prose. However, to date, those formal specifications are limited to the functional aspects of the ISA and do not cover its security guarantees. We present a novel, general method for formally specifying an ISA's security guarantees to (1) balance the needs of ISA implementations (hardware) and clients (software), (2) can be semi-automatically verified to hold for the ISA operational semantics, producing a high-assurance mechanically-verifiable proof, and (3) support informal and formal reasoning about security-critical software in the presence of adversarial code. Our method leverages universal contracts: software contracts that express bounds on the authority of arbitrary untrusted code. Universal contracts can be kept agnostic of software abstractions, and strike the right balance between requiring sufficient detail for reasoning about software and preserving implementation freedom of ISA designers and CPU implementers. We semi-automatically verify universal contracts against Sail implementations of ISA semantics using our Katamaran tool; a semi-automatic separation logic verifier for Sail which produces machine-checked proofs for successfully verified contracts. We demonstrate the generality of our method by applying it to two ISAs that offer very different security primitives: (1) MinimalCaps: a custom-built capability machine ISA and (2) a (somewhat simplified) version of RISC-V with PMP. We verify a femtokernel using the security guarantee we have formalized for RISC-V with PMP.

Information Flow Control is important for securing applications, primarily to preserve the confidentiality and integrity of applications and the data they process.Statically determining the flows of information for security purposes helps to secure applications early in the development pipeline.However, a sound and precise static analysis is difficult to scale.Modular static analysis is a technique for improving the scalability of static analysis.In this paper, we present an approach for constructing a modular static analysis for performing Information Flow Control for higher-order, imperative programs.A modular analysis requires information about data dependencies between modules.These dependencies arise as a result of information flows between modules, and therefore we piggy-back an Information Flow Control analysis on top of an existing modular analysis.Additionally, the resulting modular Information Flow Control analysis retains the benefits of its modular character.We validate our approach by performing an Information Flow Control analysis on 9 synthetic benchmark programs that contain both explicit and implicit information flows.

The recently introduced WebAssembly standard aims to form a portable compilation target, enabling the cross-platform distribution of programs written in a variety of languages. In this paper, we propose and investigate the first dynamic slicing approaches for WebAssembly. Given a program and a location in that program, a program slice is a reduced version of the program that preserves the behavior at the given location. Slicing has numerous applications in software maintenance and evolution, including reverse engineering, code comprehension, and quality assurance.Our dynamic approaches are built on Observational-Based Slicing (ORBS). We explore the design space for instantiating ORBS for WebAssembly: for example, it can be applied to the whole program or to only the function containing the slicing criterion, and it can be applied before compilation to WebAssembly or afterwards. We evaluate the slices produced quantitatively and qualitatively, and compare them to those obtained by a state-of-the-art static slicer for WebAssembly. Our evaluation reveals that dynamic slicing at the level of a function from a WebAssembly binary finds a sweet spot in terms of slice time and slice size.

Infrastructure as Code is the practice of developing and maintaining computing infrastructure through executable source code. Unfortunately, IaC has also brought about new cyber attack vectors. Prior work has therefore proposed static analyses that detect security smells in Infrastructure as Code files. However, they have so far remained at a shallow level, disregarding the control and data flow of the scripts under analysis, and may lack awareness of specific syntactic constructs. These limitations inhibit the quality of their results. To address these limitations, in this paper, we present GASEL, a novel security smell detector for the Ansible IaC language. It uses graph queries on program dependence graphs to detect 7 security smells. Our evaluation on an oracle of 243 real-world security smells and comparison against two state-of-the-art security smell detectors shows that awareness of syntax, control flow, and data flow enables our approach to substantially improve both precision and recall. We further question whether the additional effort required to develop and run such an approach is justified in practice. To this end, we investigate the prevalence of indirection through control and data flow in security smells across more than 15 000 Ansible scripts. We find that over 55% of security smells contain data-flow indirection, and over 32% require a whole-project analysis to detect. These findings motivate the need for deeper static analysis tools to detect security vulnerabilities in IaC.

Using Kubernetes for the deployment, management and scaling of containerized applications has become a common practice. To facilitate the installation and management of these applications, practitioners can use the Helm package manager to assemble their configuration files into charts. The latter are reusable packages of pre-configured Kubernetes resources that can be deployed as a unit. In this paper, we aim to support chart developers and users by carrying out a comprehensive study on publicly available charts. For 9,482 charts that are distributed via the Artifact Hub repository, we mine and collect the list of their metadata, versions, dependencies, maintainers and container images. Then, we carry out an empirical analysis to assess the state and evolution of charts, as well as the outdatedness and security risks of their images. We found that the ecosystem forming around Helm charts is growing fast. However, most of the charts are not official with no popularity and no license. We also observed that charts tend to release multiple versions, but around half of them are still in the initial development phase. When looking at the container images used in charts, we found that around half of them are outdated and 88.1% of them are exposed to vulnerabilities, jeopardizing 93.7% of the charts.

Collaborative software development depends on managing multiple versions of a program which requires mechanisms to merge program versions to eventually deploy a single executable. Merging program versions can be challenging as conflicts can arise. The most challenging form is a semantic conflict, which introduces unintended behaviour in the resulting executable while merging. In this paper, we develop an approach that detects such semantic merge conflicts by symbolic execution. We define the program semantics as path conditions, produced by a symbolic executor, and check whether the conditions satisfy established rules that reflect a merge conflict. Our usage of symbolic execution to check these rules is novel. We evaluate the correctness of our approach through mutation testing, and evaluate it empirically by applying the approach to real-world merges sampled from GitHub. We also discuss what challenges arise in the empirical evaluation, including the problems i) that semantic merge conflicts are rare in the wild, ii) and, even in retrospection, hard to find using standard search mechanisms. Our evaluation shows that in specific cases, our approach using symbolic execution is a promising extension to existing mechanisms to merge conflict detection.

Docker is a popular technology to containerise applications together with their dependencies into reproducible environments. In Docker, container images can depend on others through inheritance. Such inheritance can propagate bad practices and security vulnerabilities from a parent image to its children. Unfortunately, Docker Hub, the most popular online registry of images, lacks transparency about such inheritance. This obscures the software supply chain, possibly leaving image users unaware of quality or security issues caused by parent images. Nonetheless, we found inheritance on Docker Hub to be an understudied topic in academia to date. Therefore, the goal of this paper is to empirically investigate the practice of image inheritance on Docker Hub. To this end, we collect a dataset of 636,625 unique images belonging to popular Docker repositories and identify inheritance by comparing the images’ layers. We leverage the constructed inheritance network to empirically investigate three aspects, namely the structure of the inheritance network, how child images differ from their parents, and outdatedness of parent images. Our results show that most popular community Docker Hub images directly inherit from official images rather than other community ones. We also observe that community child images are often much larger than their parent, in comparison to official child images. This may indicate the existence of gaps between the features provided by official images and those required by consumers, suggesting the need for more ready-made parent images. Finally, we find that around half of the child images use an outdated parent image at the time the child is built, although time lag is usually less than a month. However, time lag becomes much larger when we compare against the latest version of the parent image available at the analysis date, with up to 70% of child images using an outdated parent image and a median of over 5 months of time lag. This indicates that users should pay attention to the lineage of the images they consume, and motivates future work on alleviating technical lag in Docker images.

Static analyses can be used by developers to compute properties of a program, enabling e.g., bug detection and program verification. However, reanalysing a program from scratch upon every change is time-consuming, especially in settings where code changes often, such as within IDEs. To avoid such full reanalyses, incremental analyses instead reuse parts of the previous analysis result, and reanalyse the changed code as necessary. While incrementality improves the analysis time, we introduce a complementary approach that further reduces the analysis time. A traditional incremental analysis updates previous analysis results without domain-specific knowledge. However, the effect of particular source code changes on analysis results can be predicted. Performing a traditional incremental analysis of the changed code might therefore be unnecessary. Instead, we propose to detect code change patterns of which the effect on analysis results can be predicted and to update these results accordingly, saving potentially expensive computations. In this paper, we explore the idea of adapting the analysis results for behaviour-preserving change patterns. In particular, we consider consistent renamings, inverted conditionals, and moved function definitions within Scheme programs. We implemented our approach and evaluated it on 30 programs. We show decreases in incremental analysis time between 3% and 99% on 25 programs that contain at least one behaviour-preserving change pattern.

Static analyses provide the foundation for several tools that help developers find problems before executing the program under analysis. Common applications include warning about unused code, deprecated API calls, or about potential security vulnerabilities within an IDE. A static analysis distinguishes itself from a dynamic analysis in that it is supposed to terminate even if the program under analysis does not. In many cases it is also desired for the analysis to be sound, meaning that its answers account for all possible program behavior. Unfortunately, analysis developers may make mistakes that violate these properties resulting in hard-to-find bugs in the analysis code itself. Finding these bugs can be a difficult task, especially since analysis developers have to reason about two separate code-bases: the analyzed code and the analysis implementation. The former is usually where the bug manifests itself, while the latter contains the faulty implementation. A recent survey has found that analysis developers prefer to reason about the analyzed program, indicating that debugging would be easier if debugging features such as (conditional) breakpoints and stepping were also available in the analyzed program. In this paper, we therefore propose cross-level debugging for static analysis. This novel technique moves debugging features such as stepping and breakpoints to the base-layer (i.e., analyzed program), while still making interactions with the meta-layer (i.e., analysis implementation) possible. To this end, we introduce novel conditional breakpoints that express conditions, which we call meta-predicates, about the current analysis’ state. We integrated this debugging technique in a framework for implementing modular abstract interpretation-based static analyses called MAF. Through a detailed case study on 4 real-world bugs taken from the repository of MAF, we demonstrate how cross-level debugging helps analysis developers in locating and solving bugs.

The recently introduced WebAssembly standard aims to be a portable compilation target, enabling the cross-platform distribution of programs written in a variety of languages. We propose an approach to slice WebAssembly programs in order to enable applications in reverse engineering, code comprehension, and security among others. Given a program and a location in that program, program slicing produces a minimal version of the program that preserves the behavior at the given location. Specifically, our approach is a static, intra-procedural, backward slicing approach that takes into account WebAssembly-specific dependences to identify the instructions of the slice. To do so it must correctly overcome the considerable challenges of performing dependence analysis at the binary level. Furthermore, for the slice to be executable, the approach needs to ensure that the stack behavior of its output complies with WebAssembly's validation requirements. We implemented and evaluated our approach on a suite of 8 386 real-world WebAssembly binaries, finding that the average size of the 495 204 868 slices computed is 53% of the original code, an improvement over the 60% attained by related work slicing ARM binaries. To gain a more qualitative understanding of the slices produced by our approach, we compared them to 1 956 source-level slices of benchmark C programs. This inspection helps to illustrate the slicer's strengths and to uncover potential future improvements.

Selecting a library from a vast ecosystem can be a daunting task. The libraries are not only numerous, but they also lack an enumeration of the features they offer. A feature enumeration for each library in an ecosystem would help developers select the most appropriate library for the task at hand. Within this enumeration, a library feature could take the form of a brief description together with the API references through which the feature can be reused. This paper presents LiFUSO, a tool that leverages Stack Overflow posts to compute a list of such features for a given library. Each feature corresponds to a cluster of related API references based on the similarity of the Stack Overflow posts in which they occur. Once LiFUSO has extracted such a cluster of posts, it applies natural language processing to describe the corresponding feature. We describe the engineering aspects of the tool, and illustrate its usage through a preliminary case study in which we compare the features uncovered for two competing libraries within the same domain. An executable version of the tool is available at https://github.com/softwarelanguageslab/lifuso and its demonstration video is accessible at https://youtu.be/tDE1LWa86cA.

Infrastructure as Code is the practice of automating the provisioning, configuration, and orchestration of network nodes using code in which variable values such as configuration parameters, node hostnames, etc. play a central role. Mistakes in these values are an important cause of infrastructure defects and corresponding outages. Ansible, a popular IaC language, nonetheless features semantics which can cause confusion about the value of variables.

Software frameworks define generic application blueprints which can be instantiated into an application through application-specific instantiation actions such as overriding a method or providing an object that implements an interface. In case the framework's documentation falls short, developers may use other instantiations of the same framework as a guide to the required instantiation actions. In this paper, we propose an automated approach to mining framework instantiation patterns from existing open-source instantiations. The approach leverages a graph-based representation to capture the common ways of implementing instantiation actions as well as their interplays, so called instantiation interplays. As a case study, we mined for patterns in a set of 2,028 Java projects that instantiate four of the most popular Java frameworks. We also classify the extracted interplays according to the different contexts in which they occur. We found that our approach discovers relevant practices and interplays that are not covered by previous approaches. Our results will allow developers to have a better understanding of the frameworks they instantiate.

Design-by-contract is a development best practice that requires the interactions between software components to be governed by precise specifications, called contracts. Contracts often take the form of pre- and post-conditions on function definitions, and are usually translated to (frequently redundant) run-time checks. So-called soft contract verifiers have been proposed to reduce the run-time overhead introduced by such contract checks by verifying parts of the contracts ahead of time, while leaving those that cannot be verified as residual run-time checks. In the state of the art, static analyses based on the Abstracting Abstract Machines (AAM) approach to abstract interpretation have been proposed for implementing such soft verifiers. However, these approaches result in whole-program analyses which are difficult to scale. In this paper, we propose a scalable summary-based compositional analysis for soft contract verification, which summarises both the correct behaviour and erroneous behaviour of all functions in the program using symbolic path conditions. Information from these summaries propagates backwards through the call graph, reducing the amount of redundant analysis states and improving the overall performance of the analysis. This backwards flow enables path constraints associated with erroneous program states to flow to call sites where they can be refuted, whereas in the state of the art they can only be refuted using the information available at the original location of the error. To demonstrate our improvements in both precision and performance compared to the state-of-the-art, we implemented our analysis in a framework called MAF (short for Modular Analysis Framework) — a framework for the analysis of higher-order dynamic programming languages. We conducted an empirical study and found an average performance improvement of 21%, and an average precision improvement of 38.15%.

Re-using whole repositories as a starting point for new projects is often done by maintaining a variant fork parallel to the original. However, the common artifacts between both are not always kept up to date. As a result, patches are not optimally integrated across the two repositories, which may lead to sub-optimal maintenance between the variant and the original project. A bug existing in both repositories can be patched in one but not the other (we see this as a missed opportunity) or it can be manually patched in both probably by different developers (we see this as effort duplication). In this paper we present a tool (named PaReCo) which relies on clone detection to mine cases of missed opportunity and effort duplication from a pool of patches. We analyzed 364 (source to target) variant pairs with 8,323 patches resulting in a curated dataset containing 1,116 cases of effort duplication and 1,008 cases of missed opportunities. We achieve a precision of 91%, recall of 80%, accuracy of 88%, and F1-score of 85%. Furthermore, we investigated the time interval between patches and found out that, on average, missed patches in the target variants have been introduced in the source variants 52 weeks earlier. Consequently, PaReCo can be used to manage variability in “time” by automatically identifying interesting patches in later project releases to be backported to supported earlier releases.

Social coding platforms centred around git provide explicit facilities to share code between projects: forks, pull requests, cherry-picking to name but a few. Variant forks are an interesting phenomenon in that respect, as they permit for different projects to peacefully co-exist, yet explicitly acknowledge the common ancestry. Several researchers analysed forking practices on open source platforms and observed that variant forks get created frequently. However, little is known on the motivations for launching such a variant fork. Is it mainly technical (e.g., diverging features), governance (e.g., diverging interests), legal (e.g., diverging licences), or do other factors come into play? We report the results of an exploratory qualitative analysis on the motivations behind creating and maintaining variant forks. We surveyed 105 maintainers of different active open source variant projects hosted on GitHub. Our study extends previous findings, identifying a number of fine-grained common motivations for launching a variant fork and listing concrete impediments for maintaining the co-existing projects.

Selecting an appropriate library for reuse within a vast software ecosystem can be a daunting task. A list of features for each library, i.e., a short description of the functionality that can be reused with code examples that illustrate its usage, may alleviate this problem. In this paper, we propose a data-driven approach that uses both the code snippets and the accompanying natural language descriptions from Stack Overflow posts to produce a list of features of a given library. Each extracted feature corresponds to a cluster of API classes and methods considered related based on attributes of the Stack Overflow posts in which they appear. We evaluated the approach considering seven Maven libraries and compared the resulting features against library descriptions from cookbook-like tutorials. The approach achieves an average accuracy of 67% across the seven libraries for the tutorial-like features. For at least 73% of the features extracted by the approach but missing from the documentation, we found a matching library usage in a corpus of GitHub projects. These results suggest that our clusters represent library features, which paves the way to better tool support for documenting software libraries and for selecting a library in an ecosystem.

The practice of backporting aims to bring the benefits of a bug or vulnerability fix from a higher to a lower release of a software package. When such a package adheres to semantic versioning, backports can be recognised as new releases in a lower major train. This is particularly useful in case a substantial number of software packages continues to depend on that lower major train. In this article, we study the backporting practices in four popular package distributions, namelyCargo,npm,PackagistandRubyGems. We observe that many dependent packages could benefit from backports provided by their dependencies. In particular, we find that a majority of security vulnerabilities affect more than one major train but are only fixed in the highest one, letting thousands of dependent packages exposed to the vulnerability. Despite that, we find that backporting updates is quite infrequent, and mostly practised by long-lived and more active packages for a variety of reasons.

Cloud-native applications increasingly provision infrastructure resources programmatically through Infrastructure as Code (IaC) scripts. These scripts have in turn become the subject of empirical software engineering research. However, an often-overlooked part are the software ecosystems that have grown around the IaC languages. For example, Galaxy is an ecosystem for the popular Ansible IaC language. Galaxy features a large number of so-called "roles", which are reusable collections of Ansible code akin to libraries for general-purpose languages. In contrast to, and despite their similarities, such IaC ecosystems have enjoyed far less attention in the literature than library ecosystems for general-purpose languages. In this data showcase paper, we present Andromeda, the first dataset capturing the Ansible Galaxy ecosystem, its roles, and their evolution. Andromeda provides structural representations of more than 125 000 role versions, and upwards of 800 000 concrete changes between such versions extracted from the underlying git repositories. Andromeda aims to provide an extensive view of the contributor side of the Galaxy ecosystem, which we hope will stimulate additional research on IaC ecosystems.

Stack Overflow is a popular question and answer platform where developers share technical issues in the hope of receiving answers with potential solutions. The latter may include code snippets making use of library versions that have long since been succeeded by newer ones. Other developers finding such a snippet at a later point in time may be unaware of its outdatedness unless mentioned in a comment. Furthermore, it can be difficult to integrate the snippet without knowing the exact version of the library it is referencing. In this paper, we propose an automated approach to identifying ranges of Maven library versions that might have been used in a Java snippet on Stack Overflow. We use a prototype implementation of the approach to assess the overall outdatedness of Stack Overflow snippets with respect to the latest version of each referenced library available from Maven. We found a considerable number of snippets that use outdated library versions, which suggests that developers should be careful when adopting solutions from Stack Overflow.

WebAssembly is increasingly used as the compilation target for cross-platform applications. In this paper, we investigate whether one can rely on the security measures enforced by existing C compilers when compiling C programs to WebAssembly. We compiled 4,469 C programs with known buffer overflow vulnerabilities to x86 code and to WebAssembly, and observed the outcome of the execution of the generated code to differ for 1,088 programs. Through manual inspection, we identified that the root cause for these is the lack of security measures such as stack canaries in the generated WebAssembly: while x86 code crashes upon a stack-based buffer overflow, the corresponding WebAssembly continues to be executed. We conclude that compiling an existing C program to WebAssembly without additional precautions may hamper its security, and we encourage more research in this direction.

Program slicing aims to reduce a program to a minimal form that produces the same output for a given slicing criterion. Program slicing approaches divide into static and dynamic approaches: whereas static approaches generate an over-approximation of the slice that is valid for all possible program inputs, dynamic approaches rely on executing the program and thus generate an under-approximation of the slice that is valid for only a subset of the inputs. An important limitation of static approaches is that they often do not generate an executable program, but rather identify only those program components upon which the slicing criterion depends (referred to as a closure slice). In order to overcome this limitation, we propose a novel approach that combines static and dynamic slicing. We rely on observation-based slicing, a dynamic approach, but protect all statements that have been identified as part of the static slice by the static slicer CodeSurfer. As a result, we obtain slices that cover at least the behavior of the static slice, but that can be compiled and executed. We evaluated this new approach on a set of 57 C programs and report our preliminary findings.

When migrating legacy systems to more contemporary platforms, it is not always feasible to migrate their external library dependencies as well. A workaround can be to partially reimplement the library dependencies on the new platform, limiting oneself to the features that are used by the migrated system. For contemporary programming languages, several approaches to mining for library usage patterns have been proposed. For legacy programming languages, in contrast, such tools are lacking. This encumbers establishing what parts of a library need to be rewritten during a migration project, especially when considering large-scale systems.In this industry track paper, we present an approach to mining library usage patterns in COBOL code. We describe a library usage extractor for COBOL, which produces graphs that capture the control and data flow involved in library calls. The extractor supports legacy control flow features, such as GO TO. We use these graphs as input to two state-of-the-art frequent subgraph mining algorithms, and report on the scalability of their use for mining common library usage patterns in two industrial COBOL systems. The mined library usage patterns can help assess and subsequently steer the migration effort.

Ansible, a popular Infrastructure-as-Code platform, provides reusable collections of tasks called roles. Roles are often contributed by third parties, and like general-purpose libraries, they evolve. Therefore, new releases of roles need to be tagged with version numbers, for which Ansible recommends adhering to the semantic versioning format. However, roles significantly differ from general-purpose libraries, and it is not yet known what constitutes a breaking change or the addition of a feature to a role. Consequently, this can cause confusion for clients of a role and new role contributors. To alleviate this issue, we perform an empirical study on semantic versioning in Ansible roles to uncover the types of changes that trigger certain types of version bumps. Our dataset consists of over 81 000 version increments spanning upwards of 8 500 Ansible roles. We design a novel structural model for these roles, and implement a domain-specific structural change extraction algorithm to calculate structural difference metrics. Afterwards, we quantitatively investigate the state of semantic versioning in Ansible roles and identify the most commonly changed elements. Then, using the structural difference metrics, we train a Random Forest classifier to predict applicable version bumps for Ansible role releases. Finally, we confirm our empirical findings with a developer survey. Our observations show that although most Ansible role developers follow the semantic versioning format, it appears that they do not always consistently follow the same rules when selecting the version bump to apply. Moreover, we find that the distinction between patch and minor increments is often unclear. Therefore, we use the gained insights to formulate a number of guidelines to apply semantic versioning on Ansible roles. These guidelines can be used by role developers to ensure a clear interpretation of the version increments.

A modular static analysis decomposes a program's analysis into analyses of its parts, or components. An intercomponent analysis instructs an intra-component analysis to analyse each component independently of the others. Additional analyses are scheduled for newly discovered components, and for dependent components that need to account for newly discovered component information. Modular static analyses are scalable, can be tuned to a high precision, and support the analysis of programs that are highly dynamic, featuring e.g., higher-order functions or dynamically allocated processes.In this paper, we present the engineering aspects of MAF, a static analysis framework for implementing modular analyses for higher-order languages. For any such modular analysis, the framework provides a reusable inter-component analysis and it suffices to implement its intra-component analysis. The intracomponent analysis can be composed from several interdependent and reusable Scala traits. This design facilitates changing the analysed language, as well as the analysis precision with minimal effort. We illustrate the use of MAF through its instantiation for several different analyses of Scheme programs.

One way to speed up static program analysis is to make use of today's multi-core CPUs by parallelising the analysis. Existing work on parallel analysis usually targets traditional data-How analyses for static, first-order languages such as C. Less attention has been given so far to the parallelisation of more general analyses that can also target dynamic, higherorder languages such as JavaScript. These are significantly more challenging to parallelise, as dependencies between analysis results are only discovered during the analysis itself. State-ofthe-art parallel analyses for such languages are therefore usually limited, both in their applicability and performance gains. In this work, we propose the parallelisation of modular analyses. Modular analyses compute different parts of the analysis in isolation of one another, and therefore offer inherent opportunities for parallelisation that have not been explored so far. In addition, they can be used to develop a general class of analysers for dynamic, higher-order languages. We present a parallel variant of the worklist algorithm that is used to drive such modular analyses. To further speed up its convergence, we show how this algorithm can exploit the monotonicity of the analysis. Existing modular analyses can be parallelised without additional effort by instead employing this parallel worklist algorithm. We demonstrate this for MODF, an inter-procedural modular analysis, and for MODCONC, an inter-process modular analysis. For MODCONC, we reveal an additional opportunity to exploit even more parallelism in the analysis. Our parallel worklist algorithm is implemented and integrated into MAF, a framework for modular program analysis. Using a set of Scheme benchmarks for MODF, we usually observe speedups between 3× and 8× when using 4 workers, and speedups between 8× and 32× when using 16 workers. For MODCONC, we achieve a maximum speedup of 15×.

Ansible, a popular Infrastructure-as-Code platform, provides reusable collections of tasks called roles. Roles are often contributed by third parties, and like general-purpose libraries, they evolve. As such, new releases of roles need to be tagged with version numbers, for which Ansible recommends adhering to the semantic versioning format. However, roles significantly differ from general-purpose libraries, and it is not yet known what constitutes a breaking change or the addition of a feature to a role. Consequently, this can cause confusion for clients of a role and new role contributors. To alleviate this issue, we perform an empirical study on semantic versioning in Ansible roles to uncover the types of changes that trigger certain types of version bumps. We collect a dataset of over 70000 version increments spanning upwards of 7800 Ansible roles. Moreover, we design a novel structural model for these roles, and implement a domainspecific structural change extraction algorithm to calculate structural difference metrics. Afterwards, we quantitatively investigate the state of semantic versioning in Ansible roles and identify the most commonly changed components. Then, using the structural difference metrics, we train a Random Forest classifier to predict applicable version bumps for Ansible role releases. Lastly, we confirm our empirical findings with a developer survey. Our observations show that although most Ansible role developers follow the semantic versioning format, it appears that they do not always consistently follow the same rules when selecting the version bump to apply.

Static analyses are used to gain more confidence in changes made by developers. To be of most use, such analyses must deliver feedback fast. Therefore, incremental static analyses update previous results rather than entirely recompute them. This reduces the analysis time upon a program change, and makes the analysis well-suited for environments where the code base is frequently updated, such as in IDEs and CI pipelines.In this work, we present a general approach to render a modular static analysis for highly dynamic programs incremental, by exploiting dependencies between intermediate analysis results. Modular analyses divide a program in interdependent parts that are analysed in isolation. The dependencies between these parts stem, for example, from the use of shared variables within the program. Our incrementalisation approach leverages the modularity of the analysis together with the dependencies that it reifies to compute and bound the impact of changes. This way, only the affected parts of the result need to be reanalysed, and unnecessary recomputations are avoided.We apply our approach to both a function-modular and a thread-modular analysis and evaluate it by comparing an incremental update of an existing result to a full reanalysis. We find reductions of the analysis time from 6% to 99% on 14 out of 16 benchmark programs, and on most programs the impact on precision is limited. On 7 of the programs, reanalysis time is reduced by more than 75%, showing that our approach results in fast incremental updates.

WebAssembly is a new W3C standard, providing a portable target for compilation for various languages. All major browsers can run WebAssembly programs, and its use extends beyond the web: there is interest in compiling cross-platform desktop applications, server applications, IoT and embedded applications to WebAssembly because of the performance and security guarantees it aims to provide. Indeed, WebAssembly has been carefully designed with security in mind. In particular, WebAssembly applications are sandboxed from their host environment. However, recent works have brought to light several limitations that expose WebAssembly to traditional attack vectors. Visitors of websites using WebAssembly have been exposed to malicious code as a result. In this paper, we propose an automated static program analysis to address these security concerns. Our analysis is focused on information flow and is compositional. For every WebAssembly function, it first computes a summary that describes in a sound manner where the information from its parameters and the global program state can flow to. These summaries can then be applied during the subsequent analysis of function calls. Through a classical fixed-point formulation, one obtains an approximation of the information flow in the WebAssembly program. This results in the first compositional static analysis for WebAssembly. On a set of 34 benchmark programs spanning 196kLOC of WebAssembly, we compute at least 64% of the function summaries precisely in less than a minute in total.

Recent studies show that the Maven ecosystem alone already contains over 2 million library artefacts including their source code, byte code, and documentation. To help developers cope with this information, several websites overlay configurable views on the ecosystem. For instance, views in which similar libraries are grouped into categories or views showing all libraries that have been tagged with tags corresponding to coarse-grained library features. The MVNRepository overlay website offers both category-based and tag-based views. Unfortunately, several libraries have not been categorised or are missing relevant tags. Some initial approaches to the automated categorisation of Maven libraries have already been proposed. However, no such approach exists for the problem of tagging of libraries in a multi-label setting.This paper proposes MUTAMA, a multi-label classification approach to the Maven library tagging problem based on information extracted from the byte code of each library. We analysed 4088 randomly selected libraries from the Maven software ecosystem. MUTAMA trains and deploys five multi-label classifiers using feature vectors obtained from class and method names of the tagged libraries. Our results indicate that classifiers based on ensemble methods achieve the best performances. Finally, we propose directions to follow in this area.

Code smells can compromise software quality in the long term by inducing technical debt. For this reason, many approaches aimed at identifying these design flaws have been proposed in the last decade. Most of them are based on heuristics in which a set of metrics is used to detect smelly code components. However, these techniques suffer from subjective interpretations, a low agreement between detectors, and threshold dependability. To overcome these limitations, previous work applied Machine-Learning that can learn from previous datasets without needing any threshold definition. However, more recent work has shown that Machine-Learning is not always suitable for code smell detection due to the highly imbalanced nature of the problem. In this study, we investigate five approaches to mitigate data imbalance issues to understand their impact on Machine Learning-based approaches for code smell detection in Object-Oriented systems and those implementing the Model-View-Controller pattern. Our findings show that avoiding balancing does not dramatically impact accuracy. Existing data balancing techniques are inadequate for code smell detection leading to poor accuracy for Machine-Learning-based approaches. Therefore, new metrics to exploit different software characteristics and new techniques to effectively combine them are needed.

Abstract garbage collection is the application of garbage collection to an abstract interpreter. Existing work has shown that abstract garbage collection can improve both the interpreter’s precision and performance. Current approaches rely on heuristics to decide when to apply abstract garbage collection. Garbage will build up and impact precision and performance when the collection is applied infrequently, while too frequent applications will bring about their own performance overhead. A balance between these tradeoffs is often difficult to strike. We propose a new approach to cope with the buildup of garbage in the results of an abstract interpreter. Our approach is able to eliminate all garbage, therefore obtaining the maximum precision and performance benefits of abstract garbage collection. At the same time, our approach does not require frequent heap traversals, and therefore adds little to the interpreters’s running time. The core of our approach uses reference counting to detect and eliminate garbage as soon as it arises. However, reference counting cannot deal with cycles, and we show that cycles are much more common in an abstract interpreter than in its concrete counterpart. To alleviate this problem, our approach detects cycles and employs reference counting at the level of strongly connected components. While this technique in general works for any system that uses reference counting, we argue that it works particularly well for an abstract interpreter. In fact, we show formally that for the continuation store, where most of the cycles occur, the cycle detection technique only requires O(1) amortized operations per continuation push. We present our approach formally, and provide a proof-of-concept implementation in the Scala-AM framework. We empirically show our approach achieves both the optimal precision and significantly better performance compared to existing approaches to abstract garbage collection.

Test smells are, analogously to code smells, defined as the characteristics exhibited by poorly designed unit tests. Their negative impact on test effectiveness, understanding, and maintenance has been demonstrated by several empirical studies. However, the scope of these studies has been limited mostly to Java in combination with the JUnit testing framework. Results for other language and framework combinations are --despite their prevalence in practice-- few and far between, which might skew our understanding of test smells. The combination of Scala and ScalaTest, for instance, offers more comprehensive means for defining and reusing test fixtures, thereby possibly reducing the diffusion and perception of fixture-related test smells. This paper therefore reports on two empirical studies conducted for this combination. In the first study, we analyse the tests of 164 open-source Scala projects hosted on GitHub for the diffusion of test smells. This required the transposition of their original definition to this new context, and the implementation of a tool (SOCRATES) for their automated detection. In the second study, we assess the perception and the ability of 14 Scala developers to identify test smells. For this context, our results show (i) that test smells have a low diffusion across test classes, (ii) that the most frequently occurring test smells are LazyTest, EagerTest, and AssertionRoulette, and (iii) that many developers were able to perceive but not to identify the smells.

To use a framework, developers often need to hook into and customise some of its functionality. For example, a common way of customising a framework is to subclass a framework type and to override some of its methods. Recently, Asaduzzaman et al. defined these customisations as extension points and proposed a new approach to mine large amounts of Java code examples and recommend the most frequently used example, so called extension patterns. Indeed, recommending extension patterns that frequently occur at such extension points can help developers to adopt a new framework correctly and to fully exploit it. In this paper, we present a differentiated replication study of the work by Asaduzzaman et al. on Java frameworks. Our aim is to replicate the work in order to analyse extension points and extension patterns in the context of Scala frameworks. To this aim, we propose SCALA-XP-MINER, a tool for mining extension patterns in Scala software systems to empirically investigate our hypotheses. Our results show that the approach proposed by the reference work is also able to mine extension patterns for Scala frameworks and that our tool is able to achieve similar Precision, Recall and F-measure compared to FEMIR. Despite this, the distribution of the extension points by category is different and most of the patterns are rather simple. Thus, the challenge of recommending more complex patterns to Scala developers is still an open problem.

Developing JavaScript and web applications with confidentiality and integrity guarantees is challenging. Information flow control enables the enforcement of such guarantees. However, the integration of this technique into software tools used by developers in their workflow is missing. In this paper we present GUARDIAML, a machine learning-assisted dynamic information flow control tool for JavaScript web applications. GUARDIAML enables developers to detect unwanted information flow from sensitive sources to public sinks. It can handle the DOM and interaction with internal and external libraries and services. Because the specification of sources and sinks can be tedious, GUARDIAML assists in this process by suggesting the tagging of sources and sinks via a machine learning component.

Dynamic analyses are commonly implemented by instrumenting the program under analysis. Examples of such analyses for JavaScript range from checkers of user- defined invariants to concolic testers. For a full-stack JavaScript program, these analyses would benefit from reasoning about the state of the client-side and server-side processes it is comprised of. Lifting a dynamic analysis so that it supports full-stack programs can be challenging. It involves distributed communication to maintain the analysis state across all processes, which has to be deadlock-free. In this paper, we advocate maintaining distributed analysis state in a centralized analysis process instead — which is communicated with from the processes under analysis. The approach is supported by a dynamic analysis platform that provides abstractions for this communication. We evaluate the approach through a case study. We use the platform to build a distributed origin analysis, capable of tracking the expressions from which values originate from across process boundaries, and deploy it on collaborative drawing application. The results show that our approach greatly simplifies the lifting process at the cost of a computational overhead. We deem this overhead acceptable for analyses intended for use at development time.

Composite commits are a common mistake in the use of version control software. A composite commit groups many unrelated tasks, rendering the commit difficult for developers to understand, revert, or integrate and for empirical researchers to analyse. We propose an algorithmic foundation for tool support to identify such composite commits. Our algorithm computes both a program dependence graph and the changes to the abstract syntax tree for the files that have been changed in a commit. Our algorithm then groups these fine-grained changes according to the slices through the dependence graph they belong to. To evaluate our technique, we analyse and refine an established dataset of Java commits, the results of which we also make available. We find that our algorithm can determine whether or not a commit is composite. For the majority of commits, this analysis takes but a few seconds. The parts of a commit that our algorithm identifies do not map directly to the commit's tasks. The parts tend to be smaller, but stay within their respective tasks.

The popularity of super repositories such as Maven Central and the CRAN is a testament to software reuse activities in both open-source and commercial projects alike. However, several studies have highlighted the risks and dangers brought about by application developers keeping dependencies on outdated library versions. Intelligent mining of super repositories could reveal hidden trends within the corresponding software ecosystem and thereby provide valuable insights for such dependency-related decisions. In this paper, we propose the Software Universe Graph (SUG) Model as a structured abstraction of the evolution of software systems and their library dependencies over time. To demonstrate the SUG's usefulness, we conduct an empirical study using 6,374 Maven artifacts and over 6,509 CRAN packages mined from their real-world ecosystems. Visualizations of the SUG model such as `library coexistence pairings' and `dependents diffusion' uncover popularity, adoption and diffusion patterns within each software ecosystem. Results show the Maven ecosystem as having a more conservative approach to dependency updating than the CRAN ecosystem.

Properties such as the absence of errors or bounds on mailbox sizes are hard to deduce statically for actor-based programs. This is because actor-based programs exhibit several sources of unboundedness, in addition to the non-determinism that is inherent to the concurrent execution of actors. We developed a static technique based on abstract interpretation to soundly reason in a finite amount of time about the possible executions of an actor-based program. We use our technique to statically verify the absence of errors in actor-based programs, and to compute upper bounds on the actors' mailboxes. Sound abstraction of these mailboxes is crucial to the precision of any such technique. We provide several mailbox abstractions and categorize them according to the extent to which they preserve message ordering and multiplicity of messages in a mailbox. We formally prove the soundness of each mailbox abstraction, and empirically evaluate their precision and performance trade-offs on a corpus of benchmark programs. The results show that our technique can statically verify the absence of errors for more benchmark programs than the state-of-the-art analysis.

Software developers often need to repeat similar modifications in multiple different locations of a system's source code. These repeated similar modifications, or systematic edits, can be both tedious and error-prone to perform manually. While there are tools that can be used to assist in automating systematic edits, it is not straightforward to find out where the occurrences of a systematic edit are located in an existing system. This knowledge is valuable to help decide whether refactoring is needed, or whether future occurrences of an existing systematic edit should be automated. In this paper, we tackle the problem of finding unknown systematic edits using a closed frequent itemset mining algorithm, operating on sets of distilled source code changes. This approach has been implemented for Java programs in a tool called SysEdMiner. To evaluate the tool's precision and scalability, we have applied it to an industrial use case.

Integrating code from different sources can be an error-prone and effort-intensive process. While an integration may appear statically sound, unexpected errors may still surface at run time. The industry practice of continuous integration aims to detect these and other run-time errors through an extensive pipeline of successive tests. Using data from a continuous integration service, Travis CI, we look into the prevalence of integration errors. We find code integration causes failure less often than regular commits. Repairing is usually done the same day and takes less than ten lines of code, largely in the source code. These results indicate that applying proper practices mitigates many issues associated with code integration.

Change distilling algorithms compute a sequence of fine-grained changes that, when executed in order, transform a given source AST into a given target AST. The resulting change sequences are used in the field of mining software repositories to study source code evolution. Unfortunately, detecting and specifying source code evolutions in such a change sequence is cumbersome. We therefore introduce a tool-supported approach that identifies minimal executable subsequences in a sequence of distilled changes that implement a particular evolution pattern, specified in terms of intermediate states of the AST that undergoes each change. This enables users to describe the effect of multiple changes, irrespective of their execution order, while ensuring that different change sequences that implement the same code evolution are recalled. Correspondingly, our evaluation is two-fold. Using examples, we demonstrate the expressiveness of specifying source code evolutions through intermediate ASTs. We also show that our approach is able to recall different implementation variants of the same source code evolution in open-source histories.

This paper presents the implementation of an efficient interpreter for a Scheme-like language using manually written asm.js code. The asm.js specification defines an optimizable subset of JavaScript that has already served well as a compilation target for web applications where performance is critical. However, its usage as a human-writable language that can be integrated into existing projects to improve performance has remained largely unexplored. We therefore apply this strategy to optimize the implementation of an interpreter. We also discuss the feasibility of this approach, as writing asm.js by hand is generally not its recommended use-case. We therefore present a macro system to solve the challenges we encounter. The resulting interpreter is compared to the original C implementation and its compiled equivalent in asm.js. This way, we evaluate whether manual integration with asm.js provides the necessary performance to bring larger applications and runtimes to the web. We also refactor our implementation to assess how more JavaScript code can cohabit with asm.js code, improving maintainability of the implementation while preserving near-native performance. In the case of our interpreter, this improved maintainability enables adding more complex optimizations. We investigate the addition of function inlining , for which we validate the performance gain.

Abstract We present a static analysis for determining whether and to what extent functions in JavaScript programs are pure. To this end, the analysis classifies functions as pure functions, observers, or procedures. A function is pure if none of its executions generate or depend upon externally observable side effects. A function is an observer as soon as one of its executions depends on an external side effect, but none of its executions generate observable side effects. Otherwise, the function is classified as a procedure. Function executions and associated callers are found by traversing all reachable function execution contexts on the call stack at the point where an effect occurs. Our approach is based on a flow analysis that, in addition to computing traditional control and value flow, keeps track of read and write effects. To increase the precision of our purity analysis, we combine it with an intraprocedural analysis that determines freshness of variables and objects. We formalize the core aspects of our technique and discuss its implementation and results on common JavaScript benchmarks. Results show that our approach is capable of determining function purity in the presence of higher‐order functions, dynamic property expressions, and prototypal inheritance. When compared with existing purity analyses, we find that our approach is as precise or more precise than the existing analyses.

In today’s web applications asynchronous requests to remote services using callbacks or futures are omnipresent. The continuation of such a non-blocking task is represented as a callback function that will later be called with the result of the request. This style of programming where the remainder of a computation is captured in a continuation function is called continuation-passing style (CPS). This style of programming can quickly lead to a phenomenon called “call- back hell”, which has a negative impact on the maintain- ability of applications that employ this style. Several alter- natives to callbacks are therefore gaining traction within the web domain. For example, there are a number of frameworks that rely on automatically transforming sequential style code into the continuation-passing style. However, these frame- works often employ a conservative approach in which each function call is transformed into CPS. This conservative approach can sequentialise requests that could otherwise be run in parallel. So-called delimited continuations can remedy, but require special marks that have to be manually inserted in the code for marking the beginning and end of the continuation. In this paper we propose an alternative strategy in which we apply a delimited CPS transformation that operates on a Program Dependence Graph instead to find the limits of each continuation.We implement this strategy in JavaScript and demonstrate its applicability to various web programming scenarios.

We present Scala-AM, a modular framework for static analysis based on systematic abstraction of abstract machines. Scala-AM achieves modularity by separating operational semantics, abstract values and machine abstraction concerns via actions, thus enabling language designers, static analysis developers and machine abstraction experts to combine their efforts. This modularity enables us to support semantics for multiple languages, and to include multiple machine abstractions in our framework. Different operational semantics and machine abstractions can then be combined with minimal effort. We evaluate our framework by demonstrating how it enables implementing machine abstractions independently of the operational semantics, and by instantiating the framework for a static taint analysis of Scheme.

We present Linvail, a novel instrumentation platform for developing dynamic analyses of JavaScript programs. Linvail is particularly well-suited to implementing shadow executions which involve tagging runtime values with analysis-specific data. In contrast to existing instrumentation platforms, Linvail is capable of tracking both tagged objects and tagged primitive values during their entire life-time in a behavior-preserving manner. To demonstrate the expressiveness of our platform, we present the implementation of several state-of-the-art analyses. Our experiments demonstrate that Linvail's accuracy comes at the price of a performance overhead, but we believe that real-world applications will remain usable under analysis.

Code templates are an intuitive means to specify source code snippets of interest, such as all instances of a bug, groups of snippets that need to be refactored or transformed, or instances of design patterns. While intuitive, it is not always straightforward to write a template that produces only the desired matches. A template could produce either more snippets than desired, or too few. To assist the users of EKEKO/X, our template-based search and transformation tool for Java, we have extended it with two components: The first is a suite of mutation operators that simplifies the process of modifying templates. The second is a system that can automatically suggest a sequence of mutations to a given template, such that it matches only with a set of desired snippets. In this tool paper, we highlight the key design decisions in implementing these two components of EKEKO/X, and demonstrate their use by walking through an example sequence of mutations suggested by the system.

JavaScript has become a popular programming language. However, its highly dynamic nature encumbers static analysis for quality assurance purposes. Only dynamic techniques such as concolic testing seem to cope. Often, these involve an instrumentation phase in which source code is extended with analysis-specific concerns. The corresponding implementations represent a duplication of engineering efforts. To facilitate developing dynamic analyses for JavaScript, we introduce Aran; a general-purpose JavaScript instrumenter that takes advantage of proxies, a recent addition to the JavaScript reflection APIs.

Whereas "responsive" web applications already offered a more desktop-like experience, there is an increasing user demand for "rich" web applications (RIAs) that offer collaborative and even off-line functionality. Realizing these qualities requires distributing previously centralized application logic and state vertically from the server to a client tier (e.g., for desktop-like and off-line client functionality), and horizontally between instances of the same tier (e.g., for collaborative client functionality and for scaling of resource-starved services). Both bring about the essential complexity of distributing application assets and maintaining their consistency, along with the accidental complexity of reconciling a myriad of heterogenous tier-specific technology. Tierless programming languages enable developing web applications as a single artefact that is automatically split in tier-specific code - resulting in a development process akin to that of a desktop application. This relieves developers of distribution and consistency concerns, as well as the need to align different tier-specific technologies. However, programmers still have to adopt a new and perhaps esoteric language. We therefore advocate developing tierless programs in a general-purpose language instead. In this poster, we introduce our approach to tierless programming in JavaScript. We expand upon our previous work by identifying development challenges arising from this approach that could be resolved through tool support.

Few static analyses support concurrent higher-order programs. Tools for detecting concurrency bugs such as deadlocks and race conditions are nonetheless invaluable to developers. Concurrency can be implemented using a variety of models, each supported by different synchronization primitives. Using this poster, we present an approach for analyzing concurrent higher-order programs in a precise manner through abstract interpretation. We instantiate the approach for two static analyses that are capable of detecting deadlocks and race conditions in programs that rely either on compare-and-swap (cas), or on conventional locks for synchronization. We observe few false positives and false negatives on a corpus of small concurrent programs, with better results for the lock-based analyses. We also observe that these programs lead to a smaller state space to be explored by the analyses. Our results show that the choice of synchronization primitives supported by an abstract interpreter has an important impact on the complexity of the static analyses performed with this abstract interpreter.

Manually detecting bugs in concurrent programs is hard due to the myriad of thread interleavings that needs to be accounted for. Higher-order programming features only exacerbate this difficulty. The need for tool support therefore increases as these features become more widespread. We investigate the P(CEK*)S abstract machine as the foundation for tool support for detecting concurrency bugs. This abstract interpreter analyzes multi-threaded, higher-order programs with shared-store concurrency and a compare-and-swap synchronization primitive. In this paper, we evaluate two different approaches to reduce the size of the state space explored by the abstract interpreter. First, we integrate abstract garbage collection into the abstract interpreter, and we observe that it does not reduce the state space as expected. We then evaluate the impact of adding first-class support for locks on the machine's client analyses. To this end, we compare a cas-based and a lock-based formulation of race condition and deadlock detection analyses. We show that adding first-class support for locks not only significantly reduces the number of abstract program states that need to be explored, but also simplifies formulating the client analyses.

We present an approach to detect function purity in JavaScript. A function is pure if none of its applications cause observable side-effects. The approach is based on a pushdown flow analysis that besides traditional control and value flow also keeps track of write effects. To increase the precision of our purity analysis, we combine it with an intraprocedural analysis to determine freshness of variables and object references. We formalize the core aspects of our analysis, and discuss our implementation used to analyze several common JavaScript benchmarks. Experiments show that our technique is capable of detecting function purity, even in the presence of higher-order functions, dynamic property expressions, and prototypal inheritance.

Functional testing requires executing particular sequences of user actions. Test automation tools enable scripting user actions such that they can be repeated more easily. SELENIUM, for instance, enables testing Web applications through scripts that interact with a Web browser and assert properties about its observable state. However, little is known about how common such tests are in practice. We therefore present a cross-sectional quantitative study of the prevalence of SELENIUM-based tests among open-source Web applications, and of the extent to which such tests are used within individual applications. Automating functional tests also brings about the problem of maintaining test scripts. As the system under test evolves, its test scripts are bound to break. Even less is known about the way test scripts change over time. We therefore also present a longitudinal quantitative study of whether and for how long test scripts are maintained, as well as a longitudinal qualitative study of the kind of changes they undergo. To the former's end, we propose two new metrics based on whether a commit to the application's version repository touches a test file. To the latter's end, we propose to categorize the changes within each commit based on the elements of the test upon which they operate. As such, we are able to identify the elements of a test that are most prone to change.

We present the QwalKeko meta-programming library for Clojure that enables querying the history of versioned software projects in a declarative manner. Unique to this library is its support for regular path expressions within history queries. Regular path expressions are akin to regular expressions, except that they match a sequence of successive snapshots of a software project along which user-specified logic conditions must hold. Such logic conditions can concern the source code within a snapshot, versioning information associated with the snapshot, as well as patterns of source code changes with respect to other snapshots. We have successfully used the resulting multi-faceted queries to detect refactorings in project histories. In this paper, we discuss how applicative logic meta-programming enabled combining the heterogenous components of QwalKeko into a uniform whole. We focus on the applicative logic interface to a new implementation of a well-known change distilling algorithm. We use the problem of detecting and categorizing changes made to Selenium-based test scripts for illustration purposes.

Developers often reuse source files developed for another project. In order to update a reused file to a newer version released by the original project, developers have to track which revision of a file was reused and how its content was modified. However, such tracking is tedious for developers. Many projects keep older versions of files whose bugs are already fixed in the original project. In this paper, we propose a technique to automatically identify source code reuse relationships between two repositories. Using a similarity metric based on longest common subsequence, we identify pairs of similar revisions of files across the repositories. To evaluate our approach, we have analyzed eight project pairs of open source software projects and compared the result with the recorded information in the repositories. As a result, we have identified 1394 file revisions as instances of source code reuse. While 75.3% of the instances are recorded in the repositories, 20.1% of the instances are unrecorded but recovered by our approach.

Developers often need to perform repetitive changes to source code. For instance, to repair several instances of a bug or to update all clients of a library to a newer version. Manually performing such changes is laborious and error-prone. Program transformation tools enable automating changes, but specifying changes as a program transformation requires significant expertise. Code templates are often touted as a remedy, yet have never been endorsed wholeheartedly. Their use is mostly limited to expressing the syntactic characteristics of the intended change subjects. Less familiar means have to be resorted to for expressing their structural, control flow, and data flow characteristics. In this tool paper, we introduce a decidedly template-driven program transformation tool called Ekeko/X. Its specifications feature templates for specifying all of the aforementioned characteristics of its subjects. To this end, developers can associate different directives with individual components of a template. Each matching directive imposes particular constraints on the matches for the component it is associated with. Rewriting directives, on the other hand, determine how each match should be changed. We develop Ekeko/X from the ground up, starting from its applicative logic meta-programming foundation. We highlight the key choices in this implementation and demonstrate its use through two example program transformations.

System maintainers face several challenges stemming from a system and its library dependencies evolving separately. Novice maintainers may lack the historical knowledge required to efficiently manage an inherited system. While some libraries are regularly updated, some systems keep a dependency on older versions. On the other hand, maintainers may be unaware that other systems have settled on a different version of a library. In this paper, we visualize how the dependency relation between a system and its dependencies evolves from two perspectives. Our system-centric dependency plots (SDP) visualize the successive library versions a system depends on over time. The radial layout and heat-map metaphor provide visual clues about the change in dependencies along the system's release history. From this perspective, maintainers can navigate to a library-centric dependants diffusion plot (LDP). The LDP is a time-series visualization that shows the diffusion of users across the different versions of a library. We demonstrate on real-world systems how maintainers can benefit from our visualizations through four case scenarios.

This paper is concerned with understanding API usage in a systematic, explorative manner for the benefit of both API developers and API users. There exist complementary, less explorative methods, e.g., based on code search, code completion, or API documentation. In contrast, our approach is highly interactive and can be seen as an extension of what IDEs readily provide today. Exploration is based on multiple dimensions: i) the hierarchically organized scopes of projects and APIs; ii) metrics of API usage (e.g., number of project classes extending API classes); iii) metadata for APIs; iv) project- versus API-centric views. We also provide the QUAATLAS corpus of Java projects which enhances the existing QUALITAS corpus to enable API-usage analysis. We implemented the exploration approach in an open-source, IDE-like, Web-enabled tool EXAPUS.

To be able to modularize crosscutting concerns, aspects introduce new programming language features, often in a new language, with a specific syntax. These new features lead to new needs for source code analysis tools, resulting in the requirement for a general-purpose aspectual source code analysis tool. Ignoring this requirement has led to a nontrivial duplication of effort in the aspect-oriented software development community. This is because all code analysis efforts that we are aware of have either built ad-hoc analysis tools or were performed manually. In this paper we present Gasr: a source code analysis tool in the tradition of logic program querying that reasons over AspectJ source code. By hooking into the IDE plugins for AspectJ, Gasr provides a library of predicates that can be used to query aspectual source code. We demonstrate the use of Gasr by automating the recognition of a number of previously identified aspectual source code assumptions. We then detect assumption instances on two well-known case studies that were manually investigated in the earlier work. In addition to finding the already known aspect assumptions, Gasr encounters assumption instances that were overlooked before.

Coupling in an object-oriented context is often defined in terms of access to instance variables and methods of other classes. JavaScript, however, lacks static type information and classes, and instead features a flexible object system with prototypal inheritance. In order to determine coupling in JavaScript, we infer object types based on abstract interpretation of a program. Type inference depends on both structure and behavior of objects, and common patterns for expressing classes and modules are supported. We approximate a set of accessed types per function, and classify every access as either local or foreign. Examples demonstrate that our object type inference, together with some additional heuristics concerning property access, enable determining coupling in JavaScript in a meaningful way.

Current-day programming languages include constructs to embed meta-data in a program's source code in the form of annotations. More than mere documentation, these annotations are used in modern frameworks to map source-level entities to domain-specific ones. A common example being the Hibernate Object-Relational Mapping framework that relies on annotations to declare persistence configurations. While the presence of annotations extends the base semantics of the language, it also imposes restrictions on the annotated program. In this paper we consider the manner in which annotations affect automated refactorings, and in particular how they break their behavior preservation. As refactorings, during their condition checking phase, ignore the annotation's restrictions they can no longer guarantee the preservation of the domain-specific mappings. To address this problem, we propose to make the restrictions of the annotations explicit, and use them to steer the refactoring process. A prototype extension of the Eclipse IDE's refactoring engine is used to demonstrate our approach on three annotation libraries: Java Persistence API, AspectJ5 and Simple XML serialization.

The need to identify source code that exhibits particular characteristics is essential to program comprehension. In this paper we introduce Arabica, a tool for querying Java code using UML class and sequence diagrams. Our use of UML diagrams avoids the need for developers to familiarize themselves with yet another language. In contrast to tools that rely on dedicated query languages, Arabica encodes querying semantics in a dedicated, minimal UML profile. Stereotyped class and sequence diagrams, characterizing structural and behavioral properties respectively, are translated into logic program queries. Using examples from the JHotDraw framework, we illustrate the utility of Arabica in validating design invariants, finding design pattern implementations and exploring extension points. We present a pre/post-test quasi experiment as a preliminary assessment of our approach.

Extracting information from the source code of a program is an important step in the way to program understanding, manipulation, development and maintenance. To this end, logic-based query languages provide a declarative manner in which to identify program elements of interest. In this paper we present BARISTA, a tool-suite for querying Java programs based on the Smalltalk Open Unification Language (SOUL). BARISTA offers programmers an advanced IDE to write queries and navigate their results. Tool builders can benefit from SOUL querying facilities by exploiting the on demand code querying and query scheduling services offered by BARISTA.

This paper summarizes the doctoral dissertation in which we introduced an example-driven approach to pattern detection. This approach enables specifying pattern characteristics in a familiar language: through a code excerpt that corresponds to their prototypical implementation. Such excerpts are matched against the program under investigation according to various matching strategies that vary in leniency. Each match is quantified by the extent to which it exhibits the exemplified characteristics. The smaller this extent, the more likely the match is a false positive - thus establishing a ranking which facilitates assessing a large amount of matches. Unique to the matching process is that it incorporates whole-program analyses in its comparison of individual program elements. This way, we are able to recall implicit implementation variants (i.e., those implied by the semantics of the programming language) of a pattern of which only the prototypical implementation has been exemplified.

The multi-core revolution heralds a challenging era for software maintainers. Manually parallelizing large sequential code bases is often infeasible. In this paper, we present a program transformation that automatically parallelizes real-life Scheme programs. The transformation has to be instantiated with an interprocedural dependence analysis that exposes parallelization opportunities in a sequential program. To this end, we extended a state-of-the art analysis that copes with higher-order procedures and side effects. Our parallelizing transformation exploits all opportunities for parallelization that are exposed by the dependence analysis. Experiments demonstrate that this brute-force approach realizes scalable speedups in certain benchmarks, while others would benefit from a more selective parallelization.

Program query languages and pattern-detection techniques are an essential part of program analysis and manipulation systems. Queries and patterns permit the identification of the parts of interest in a program's implementation through a representation dedicated to the intent of the system (e.g. call-graphs to detect behavioral flaws, abstract syntax trees for transformations, concrete source code to verify programming conventions, etc). This requires that developers understand and manage all the different representations and techniques in order to detect various patterns of interest. To alleviate this overhead, we present a logic-based language that allows the program's implementation to be queried using concrete source code templates. The queries are matched against a combination of structural and behavioral program representations, including call-graphs, points-to analysis results and abstract syntax trees. The result of our approach is that developers can detect patterns in the queried program using source code excerpts (embedded in logic queries) which act as prototypical samples of the structure and behavior they intend to match.

Typically, multiple developers are involved in the various stages of the software development and maintenance process. To ensure an optimal transfer of knowledge between these different peers, a reliable human-readable model of the dynamics of a software artefact is needed. Once these models become machine-verifiable, they can be used throughout an application’s lifetime to check whether the documented behavioral properties continue to hold as the application evolves. Unfortunately, most existing modeling media are inadequate to express human-readable behavioral models which are at the same time machine-verifiable. We therefore propose a declarative platform wherein behavioral program models can be expressed in terms of userdefined high-level concepts and be automatically verified against an application’s actual behavior. We demonstrate our approach by using it to both document and verify an interpreter for a garbage-collected programming language.

The quality of an application's implementation can be assured by validating the presence or absence of a set of user-prescribed software patterns such as software engineering best practices, programming conventions and indications of poor programming. Most of the existing pattern detection techniques, however, interpret pattern descriptions in an inflexible manner, leaving the quality assurance tool to approve only the most strictly adhering pattern implementations. In order to detect various concrete pattern implementations using a single pattern description, we have combined logic meta programming ---wherein patterns can be expressed as constraints over facts representing a program's source code---, fuzzy logic and static program analysis in a way that is completely transparent to the end-user. We have achieved this by having the conditions in a logic rule interpreted as constraints over the run-time behavior source code constructs give rise to instead of as constraints over the literal source code constructs themselves. This way, a pattern's abstract description often suffices to recognize various concrete implementation variants with an indication of the similarity between the recognized implementation and the abstract pattern description