Confounders (or confounding variables) pose significant challenges to detecting reliable causal relationships in observational studies. When data are collected from naturally occurring phenomena—e.g., mining software repositories (MSR)—researchers cannot rely on randomization to control for confounders, leading to biased causal inferences. Alternative approaches are required to mitigate confounding bias when exploring causal inferences. This paper explains and exemplifies the use of matching in MSR.

Abstract Context Security issues in Open-Source (OS) software systems emerge from everyday coding activities carried out by developers. As security best practices evolve toward “shift-left” paradigms—emphasizing early and continuous integration of security into the development process—understanding how these issues are introduced and fixed has become increasingly important. Objective Our primary goal is to study the spread and evolution of security issues that lie in the source code of OS Python software systems at commit level. Method We conducted a mining study in which we quantitatively analyzed the commit histories of 361 OS Python software systems, whose repositories were publicly available on GitHub , for a total of 380,931 commits analyzed. To identify security issues at the commit level, we used SonarQube , a popular and widely used both in academic and industrial contexts Static Application Security Testing ( SAST ) tool. Results We observed that security issues are spread in OS Python software systems (on average, there are about 14 security issues per commit) and tend to survive for 11 days and 14 commits. Critical security issues, despite their high severity level, are the most spread and tend to survive the most. Furthermore, we noticed that 55 kinds of security issues—belonging to 62 OWASP Top 10 and CWE security classes—were introduced, and the top six (per number of introductions) are mostly critical and account for 77% of all introduced security issues. Conclusions Python developers need to give utmost importance to security issues, particularly critical ones. To that end, we can suggest developers that use secure coding practices, automated tools, or even DevSecOps to limit/avoid the introduction of security issues into their source code or fix them as soon as possible.

The increasing adoption of Artificial Intelligence (AI) in any kind of software has highlighted the need for greater transparency, security, and traceability within the AI supply chain. The AI Bill of Materials (AIBoM) extends the Software Bill of Materials (SBoM) concept by incorporating AI-specific components such as models, datasets, dependencies, and metadata. In this paper, we introduce ALOHA, a novel tool that automatically generates AIBoM from AI models hosted on Hugging Face (HF), leveraging the CycloneDX standard for software transparency and security. ALOHA extracts relevant metadata from model cards and maps them to a structured AIBoM format, ensuring compliance with existing SBoM frameworks. We conducted a preliminary empirical evaluation on a statistically significant sample of 312 AI models to assess ALOHA. Our initial findings indicate that while ALOHA successfully retrieves and structures essential AIBoM fields, challenges remain regarding metadata completeness and standardization of model cards. This work represents a step towards enhancing AI supply chain security and governance, providing a foundation for future advancements in AIBoM generation.

Background: Static Code Analysis (SCA) tools are widely adopted to enforce code quality standards. However, little is known about how open-source projects use and customize these tools. Aims: This paper investigates how GitHub projects use and customize a popular SCA tool, namely SonarQube Cloud. Method: We conducted a mining study of GitHub projects that are linked through GitHub Actions to SonarQube Cloud projects. Results: Among 321 GitHub projects using SonarQube Cloud, 81% of them are correctly connected to SonarQube Cloud projects, while others exhibit misconfigurations or restricted access. Among 265 accessible SonarQube Cloud projects, 75% use the organization's default quality gate, i.e., a set of conditions that deployed source code must meet to pass automated checks. While 55% of the projects use the built-in quality gate provided by SonarQube Cloud, 45% of them customize their quality gate with different conditions. Overall, the most common quality conditions align with SonarQube Cloud's “Clean as You Code” principle and enforce security, maintainability, reliability, coverage, and a few duplicates on newly added or modified source code. Conclusions: Many projects rely on predefined configurations, yet a significant portion customize their configurations to meet specific quality goals. Building on our initial results, we envision a future research agenda linking quality gate configurations to actual software outcomes (e.g., improvement of software security). This would enable evidence-based recommendations for configuring SCA tools like SonarQube Cloud in various contexts.

A software supply chain consists of anything needed to develop and deliver a software project, including (third-party) components. Software Composition Analysis (SCA) allows for managing the security of software supply chains by identifying such components and their (security) vulnerabilities. The main goal of the empirical study presented in this paper is to investigate the effects of adopting/using over time an SCA tool like OWASP Dependency-Check (OWASP DC) in the context of the security of the software supply chain. To this end, following a cohort design, we analyzed the vulnerabilities affecting the components of the open-source (OS) Java Maven projects owned by the Apache Software Foundation (ASF) and publicly hosted on GitHub. These projects could adopt (or not) OWASP DC. The results indicate that the adoption of OWASP DC appears to be causing a significant reduction in the overall number/score of vulnerabilities, including those with a high Common Vulnerability Scoring System (CVSS) severity level. The use of OWASP DC also increased the vulnerabilities with a low severity level. Our results seem to encourage practitioners to adopt SCA to improve the security of their software supply chains.

GitHub’s dependency graph is a tool that eases Software Composition Analysis (SCA), and it is leveraged not only by other tools or by practitioners in their analyses but also by researchers when conducting studies on open-source projects. However, its potential inaccuracy may seriously harm its applicability and usefulness. This paper quantitatively and qualitatively analyzes the accuracy of GitHub’s dependency graphs for Java and Python projects, how such accuracy has changed over time, and what the likely pitfalls and limitations of the dependency graph are. After creating statistically significant samples of Java and Python projects, we analyzed their dependency graph in two directions, forward (by looking at dependencies), backward (by looking at dependents), and inspected their manifest/lock files. Results indicate that in our sample, dependencies have over 27% of inaccuracy, and dependents up to 10%. Errors depend on several reasons, among others, an oversimplified processing of manifest/lock files by the dependency graph generator. Our results provide (i) guidelines for researchers to understand the threats arising in studies based on the dependency graph and (ii) insights to practitioners and tool builders to enhance their SCA, given the current limitations of the dependency graph. • We show that GitHub dependency graph is inaccurate ( ≃ 20% errors in dependencies and ≃ 10 errors in dependents). • We report a qualitative categorization of dependency graph inaccuracies root causes. • The findings of our research warn about the accuracy of tools and studies leveraging the dependency graph.

A Software Bill of Materials ( SBOM ) formally lists the open-source and proprietary components that constitute a software product, including their licenses, versions, vendors, vulnerabilities, and supply chain relationships. SBOMs enable software producers and consumers to gain visibility into the software supply chain and monitor the risks associated with software security, licensing, and more. This paper presents the results of an exploratory mining study investigating the adoption of SBOMs by open-source software projects. To that end, we mined GitHub and identified 186 public software repositories using SBOM generation tools owned by SPDX and CycloneDX . Although the adoption of SBOMs is low, it is increasing. Moreover, SBOMs are under version control or available in public release versions of less than half the software projects analyzed. Finally, only a limited fraction of SBOMs contain minimum/recommended information, and some SBOMs are also uncompliant with existing SBOM standards. Our study reveals that software producers are paying more attention to SBOMs, but even so, these may be incomplete. We urge software producers to adopt SBOMs and meet the new software supply chain standards. As for researchers, we foster further investigations on adopting SBOMs and their correct use.

GitHub’s dependency graph shows dependency relationships between repositories. This feature is leveraged by tools such as Dependabot, or GitHub’s feature to export SBOM (Software Bill of Materials) files. Also, it has been used in empirical studies. Inaccuracies in the dependency graph might negatively affect both the effectiveness of tools and the results of the conducted studies. In this paper, we present the results of a mining study to assess the accuracy of GitHub’s dependency graph in Java and Python open-source software projects. In particular, on April 16th, 2023, we randomly sampled 297 software projects developed in Java and 338 developed in Python (all hosted on GitHub), each using GitHub’s dependency graph. Then, we performed three analyses to assess how accurate GitHub’s dependency graph is: (i) backward analysis, focusing on the accuracy of the dependencies of a given repository, as reported in GitHub’s dependency graph; (ii) forward analysis, focusing on the accuracy of the dependents of a given repository, as reported in GitHub’s dependency graph; and (iii) manifest/lock file analysis, focusing on the correspondence between the dependencies reported in the dependency graph of a given repository and what was reported in the corresponding manifest/lock files. The obtained results highlight several inaccuracies in GitHub’s dependency graph, which might affect the output of tools based on GitHub’s dependency graph (e.g., Dependabot and SBOM generators) as well as the outcomes of past empirical studies. We also provide qualitative insights into these inaccuracies and implications for practitioners and researchers.

MSR4SBOM (Mining Software Repositories for enhanced Software Bills of Materials) is a project whose main goal is to deliver a framework that analyzes the content of software repositories and SBOMs to provide context-sensitive recommendations. The expected outputs are (i) a set of approaches and tools released as open-source projects, making them exploitable in industrial, academic, and open-source contexts; and (ii) replication packages of our empirical studies and repositories of datasets collected while developing, calibrating, and validating the MSR4SBOM approaches and tools.

A Software Bill of Materials (SBOM) describes, in a structured, machine-readable format, the open-source and proprietary components that constitute a software product, including their licenses, versions, vendors, vulnerabilities, and dependency relationships. SBOMs enable practitioners to gain visibility into the software supply chain and monitor any risks associated with software security, licensing, and more. In this industry paper, we present the findings of 10 semi-structured interviews with practitioners with different roles in six different software companies operating in Italy, some of which being very large multinationals. The gathered information indicates that the adoption of SBOMs is low, yet the attention of the software industry to software supply chain-related challenges is high. A possible reason behind this outcome is that the software industry has limited knowledge of SBOMs and software supply chain regulations. Although some participants showed a growing interest in SBOMs, the Italian software industry seems to respond less promptly to this technology. We plan to use these results and those from past research to design a survey with practitioners to have a complete picture of SBOM usage in the software industry.

Background: There are a number of academic courses in the Bachelor Program in Computer Science (CS) on the design of Web apps. Often the internal and external quality of the developed Web apps is not adequately taken into account. Aim: We aimed to (i) estimate the quality of Web apps developed by bachelor CS students in a Software Technologies for the Web (STW) course (a.y. 2021-22) and (ii) define a training plan (on the base of the results of the first step) for the students enrolled to this course for the a.y. 2022-23 to let them design and implement better Web apps, and (iii) experimenting the training plan by comparing the quality of Web apps developed in a.y. 2021-22 and a.y. 2022-23. Method: We designed a prospective empirical investigation to study STW with respect to the training of bachelor students with respect to the quality (internal and external) of the developed Web apps. Results: We observed that quality concerns are widespread in the code of the Web apps the STW students developed in the a.y. 2021-22. Therefore, we plan to ask the students of the a.y. 2022-23 to use in their development pipeline a Static Analysis Tool (SAT) to detect quality concerns in the developed Web apps and deal with them. This second step represents an ongoing stage of our research. Conclusions: Our preliminary outcomes suggest that students must be aware that quality is of primary relevance for the development of Web apps and prepared to use SAT in the development pipeline.

Background: Software vulnerabilities are flaws in application source code that can be exploited to cause harm, hence companies must devise strategies to manage them.Aim: We want to understand how software vulnerabilities are managed in a big IT (Information Technology) service and consulting company like NTT Data.Method: We conducted a focus group involving six software professionals working at NTT Data and analyzed the gathered data through a thematic analysis approach.Results: We found that application security standards are defined based on the needs of the clients (i.e., companies that commissioned NTT Data the software to be developed) and the projects’ nature (i.e., the development of greenfield projects vs. maintenance of existing ones). Also, to detect software vulnerabilities, SAST (Static Application Security Testing) tools are mainly used; among these, SonarLint and SonarQube appear to be the de-facto standards for NTT Data. Finally, not all software vulnerabilities are fixed; for example, the presence of some software vulnerabilities is tolerated by the clients, who take on the responsibility of not removing these vulnerabilities.Conclusions: It seems that developers and NTT Data clients are not averse to securing their code. NTT Data follows the application security standards established with their clients. To detect software vulnerabilities, SonarLint and SonarQube appear to be the de-facto standards, so explaining to some extent the increasing attention on these tools by the software engineering research community.

We conducted a large-scale fine-grained empirical study in which we quantitatively analyzed the commit histories of 200 Open-Source (OS) Python software systems, whose software repositories were publicly available on GitHub, for a total of 164,980 commits analyzed. We focused on commits—this is why our study is considered fine-grained—to investigate the spread and evolution of security concerns. To detect security concerns at a commit level, we used SonarQube, a popular Static Application Security Testing (SAST) tool. We found, among other things, that: security concerns are spread in OS Python software systems (on average, about 11 security concerns per commit) and tend to survive more than a couple of weeks and a dozen commits; and critical security concerns, despite their high severity level, are the most spread and tend to survive the most. Also, we found that 47 different kinds of security concerns were introduced into the source code of the studied software systems, and the top eight (per number of introductions) are severe and account for 87% of all introduced security concerns. Python developers should pay more attention to security concerns, especially those critical, and use secure coding practices, automated tools, or even DevSecOps to avoid the introduction of security concerns into their source code or fix them as soon as possible.

A Software Bill of Materials (SBOM) is a complete, formally structured list of all the open-source and proprietary software components present in a software product, including their licenses, versions, and vendors. SBOMs enable software creators and consumers to gain visibility into the software supply chain and monitor any risks associated with security or licensing. Thereby, the United States Government and the European Union have brought SBOMs to the forefront of digital policy. In this paper, we present the results of an exploratory mining study that aims to investigate the adoption of SBOMs by open-source software projects. To that end, we mined GitHub to identify repositories that use SBOM generation tools developed by SPDX and CycloneDX, identifying a total of 186 public repositories adopting SBOMs. We found that the adoption of SBOMs is low, yet it has an increasing trend. Moreover, SBOM files are available in the repository or published release versions in 46% of the software projects analyzed. SBOMs are getting an increasing attention from software creators and consumers. There is a pressing need for organizations to update their software to meet the new standards required for the software supply chain.