Policy as Code (PaC) is an emerging DevOps practice that enables teams to specify organisational and technical policies, such as regulatory compliance, security requirements, and resource limits, through machine-enforceable declarative code. As PaC gains prominence, practitioners face difficulties in adopting PaC while there remains a limited empirical understanding of how these policies are introduced, what types can be expressed, and how they are maintained in practice.

End-to-end (E2E) testing is a software validation approach that simulates realistic user scenarios throughout the entire workflow of an application. In the context of web applications, E2E testing involves two activities: Graphic User Interface (GUI) testing, which simulates user interactions with the web app’s GUI through web browsers, and performance testing, which evaluates system workload handling. Despite its recognized importance in delivering high-quality web applications, the availability of large-scale datasets featuring real-world E2E web tests remains limited, hindering research in the field.To address this gap, we present E2EGit, a comprehensive dataset of non-trivial open-source web projects collected on GitHub that adopt E2E testing. By analyzing over 5,000 web repositories across popular programming languages (Java, JavaScript, TypeScript and Python), we identified 472 repositories implementing 43,670 automated Web GUI tests with popular browser automation frameworks (Selenium, Playwright, Cypress, Puppeteer), and 84 repositories that featured 271 automated performance tests implemented leveraging the most popular open-source tools (JMeter, LoCust). Among these, 13 repositories implemented both types of testing for a total of 786 Web GUI tests and 61 performance tests. The dataset is available on Zenodo (DOI: 10.5281/zenodo.14234731).

Infrastructure as Code is the practice of developing and maintaining computing infrastructure through executable source code. Unfortunately, IaC has also brought about new cyber attack vectors. Prior work has therefore proposed static analyses that detect security smells in Infrastructure as Code files. However, they have so far remained at a shallow level, disregarding the control and data flow of the scripts under analysis, and may lack awareness of specific syntactic constructs. These limitations inhibit the quality of their results. To address these limitations, in this paper, we present GASEL, a novel security smell detector for the Ansible IaC language. It uses graph queries on program dependence graphs to detect 7 security smells. Our evaluation on an oracle of 243 real-world security smells and comparison against two state-of-the-art security smell detectors shows that awareness of syntax, control flow, and data flow enables our approach to substantially improve both precision and recall. We further question whether the additional effort required to develop and run such an approach is justified in practice. To this end, we investigate the prevalence of indirection through control and data flow in security smells across more than 15 000 Ansible scripts. We find that over 55% of security smells contain data-flow indirection, and over 32% require a whole-project analysis to detect. These findings motivate the need for deeper static analysis tools to detect security vulnerabilities in IaC.

Using Kubernetes for the deployment, management and scaling of containerized applications has become a common practice. To facilitate the installation and management of these applications, practitioners can use the Helm package manager to assemble their configuration files into charts. The latter are reusable packages of pre-configured Kubernetes resources that can be deployed as a unit. In this paper, we aim to support chart developers and users by carrying out a comprehensive study on publicly available charts. For 9,482 charts that are distributed via the Artifact Hub repository, we mine and collect the list of their metadata, versions, dependencies, maintainers and container images. Then, we carry out an empirical analysis to assess the state and evolution of charts, as well as the outdatedness and security risks of their images. We found that the ecosystem forming around Helm charts is growing fast. However, most of the charts are not official with no popularity and no license. We also observed that charts tend to release multiple versions, but around half of them are still in the initial development phase. When looking at the container images used in charts, we found that around half of them are outdated and 88.1% of them are exposed to vulnerabilities, jeopardizing 93.7% of the charts.

Infrastructure as Code is the practice of automating the provisioning, configuration, and orchestration of network nodes using code in which variable values such as configuration parameters, node hostnames, etc. play a central role. Mistakes in these values are an important cause of infrastructure defects and corresponding outages. Ansible, a popular IaC language, nonetheless features semantics which can cause confusion about the value of variables.

Cloud-native applications increasingly provision infrastructure resources programmatically through Infrastructure as Code (IaC) scripts. These scripts have in turn become the subject of empirical software engineering research. However, an often-overlooked part are the software ecosystems that have grown around the IaC languages. For example, Galaxy is an ecosystem for the popular Ansible IaC language. Galaxy features a large number of so-called "roles", which are reusable collections of Ansible code akin to libraries for general-purpose languages. In contrast to, and despite their similarities, such IaC ecosystems have enjoyed far less attention in the literature than library ecosystems for general-purpose languages. In this data showcase paper, we present Andromeda, the first dataset capturing the Ansible Galaxy ecosystem, its roles, and their evolution. Andromeda provides structural representations of more than 125 000 role versions, and upwards of 800 000 concrete changes between such versions extracted from the underlying git repositories. Andromeda aims to provide an extensive view of the contributor side of the Galaxy ecosystem, which we hope will stimulate additional research on IaC ecosystems.

Stack Overflow is a popular question and answer platform where developers share technical issues in the hope of receiving answers with potential solutions. The latter may include code snippets making use of library versions that have long since been succeeded by newer ones. Other developers finding such a snippet at a later point in time may be unaware of its outdatedness unless mentioned in a comment. Furthermore, it can be difficult to integrate the snippet without knowing the exact version of the library it is referencing. In this paper, we propose an automated approach to identifying ranges of Maven library versions that might have been used in a Java snippet on Stack Overflow. We use a prototype implementation of the approach to assess the overall outdatedness of Stack Overflow snippets with respect to the latest version of each referenced library available from Maven. We found a considerable number of snippets that use outdated library versions, which suggests that developers should be careful when adopting solutions from Stack Overflow.

Test smells are, analogously to code smells, defined as the characteristics exhibited by poorly designed unit tests. Their negative impact on test effectiveness, understanding, and maintenance has been demonstrated by several empirical studies. However, the scope of these studies has been limited mostly to Java in combination with the JUnit testing framework. Results for other language and framework combinations are --despite their prevalence in practice-- few and far between, which might skew our understanding of test smells. The combination of Scala and ScalaTest, for instance, offers more comprehensive means for defining and reusing test fixtures, thereby possibly reducing the diffusion and perception of fixture-related test smells. This paper therefore reports on two empirical studies conducted for this combination. In the first study, we analyse the tests of 164 open-source Scala projects hosted on GitHub for the diffusion of test smells. This required the transposition of their original definition to this new context, and the implementation of a tool (SOCRATES) for their automated detection. In the second study, we assess the perception and the ability of 14 Scala developers to identify test smells. For this context, our results show (i) that test smells have a low diffusion across test classes, (ii) that the most frequently occurring test smells are LazyTest, EagerTest, and AssertionRoulette, and (iii) that many developers were able to perceive but not to identify the smells.

Software developers often need to repeat similar modifications in multiple different locations of a system's source code. These repeated similar modifications, or systematic edits, can be both tedious and error-prone to perform manually. While there are tools that can be used to assist in automating systematic edits, it is not straightforward to find out where the occurrences of a systematic edit are located in an existing system. This knowledge is valuable to help decide whether refactoring is needed, or whether future occurrences of an existing systematic edit should be automated. In this paper, we tackle the problem of finding unknown systematic edits using a closed frequent itemset mining algorithm, operating on sets of distilled source code changes. This approach has been implemented for Java programs in a tool called SysEdMiner. To evaluate the tool's precision and scalability, we have applied it to an industrial use case.

Integrating code from different sources can be an error-prone and effort-intensive process. While an integration may appear statically sound, unexpected errors may still surface at run time. The industry practice of continuous integration aims to detect these and other run-time errors through an extensive pipeline of successive tests. Using data from a continuous integration service, Travis CI, we look into the prevalence of integration errors. We find code integration causes failure less often than regular commits. Repairing is usually done the same day and takes less than ten lines of code, largely in the source code. These results indicate that applying proper practices mitigates many issues associated with code integration.