The increasing adoption of Artificial Intelligence (AI) in any kind of software has highlighted the need for greater transparency, security, and traceability within the AI supply chain. The AI Bill of Materials (AIBoM) extends the Software Bill of Materials (SBoM) concept by incorporating AI-specific components such as models, datasets, dependencies, and metadata. In this paper, we introduce ALOHA, a novel tool that automatically generates AIBoM from AI models hosted on Hugging Face (HF), leveraging the CycloneDX standard for software transparency and security. ALOHA extracts relevant metadata from model cards and maps them to a structured AIBoM format, ensuring compliance with existing SBoM frameworks. We conducted a preliminary empirical evaluation on a statistically significant sample of 312 AI models to assess ALOHA. Our initial findings indicate that while ALOHA successfully retrieves and structures essential AIBoM fields, challenges remain regarding metadata completeness and standardization of model cards. This work represents a step towards enhancing AI supply chain security and governance, providing a foundation for future advancements in AIBoM generation.

Large Language Models (LLMs) are widely used in software development tasks nowadays. Unlike reusing code taken from the Web, for LLMs' generated code, developers are concerned about its lack of trustworthiness and possible copyright or licensing violations, due to the lack of code provenance information. This paper proposes CodeGenLink, a GitHub CoPilot extension for Visual Studio Code aimed at (i) suggesting links containing code very similar to automatically generated code, and (ii) whenever possible, indicating the license of the likely origin of the code. CodeGenLink retrieves candidate links by combining LLMs with their web search features and then performs similarity analysis between the generated and retrieved code. Preliminary results show that CodeGenLink effectively filters unrelated links via similarity analysis and provides licensing information when available.Tool URL: https://github.com/danielebifolco/CodeGenLinkTool Video: https://youtu.be/M6nqjBf_pw

Large Language Models (LLMs) are currently used for various software development tasks, including generating code snippets to solve specific problems. Unlike reuse from the Web, LLMs are limited in providing provenance information about the generated code, which may have important trustworthiness and legal consequences. While LLM-based assistants may provide external links that are “related” to the generated code, we do not know how relevant such links are. This paper presents the findings of an empirical study assessing the extent to which 243 and 194 code snippets, across six programming languages, generated by Bing CoPilot and Google Gemini, likely originate from the links provided by these two LLM-based assistants. The study leverages automated code similarity assessments with thorough manual analysis. The study’s findings indicate that the LLM-based assistants provide a mix of relevant and irrelevant links having a different nature. Specifically, although 66% of the links from Bing CoPilot and 28% from Google Gemini are relevant, LLMs-based assistants still suffer from serious “provenance debt”.

GitHub’s dependency graph is a tool that eases Software Composition Analysis (SCA), and it is leveraged not only by other tools or by practitioners in their analyses but also by researchers when conducting studies on open-source projects. However, its potential inaccuracy may seriously harm its applicability and usefulness. This paper quantitatively and qualitatively analyzes the accuracy of GitHub’s dependency graphs for Java and Python projects, how such accuracy has changed over time, and what the likely pitfalls and limitations of the dependency graph are. After creating statistically significant samples of Java and Python projects, we analyzed their dependency graph in two directions, forward (by looking at dependencies), backward (by looking at dependents), and inspected their manifest/lock files. Results indicate that in our sample, dependencies have over 27% of inaccuracy, and dependents up to 10%. Errors depend on several reasons, among others, an oversimplified processing of manifest/lock files by the dependency graph generator. Our results provide (i) guidelines for researchers to understand the threats arising in studies based on the dependency graph and (ii) insights to practitioners and tool builders to enhance their SCA, given the current limitations of the dependency graph. • We show that GitHub dependency graph is inaccurate ( ≃ 20% errors in dependencies and ≃ 10 errors in dependents). • We report a qualitative categorization of dependency graph inaccuracies root causes. • The findings of our research warn about the accuracy of tools and studies leveraging the dependency graph.

GitHub’s dependency graph shows dependency relationships between repositories. This feature is leveraged by tools such as Dependabot, or GitHub’s feature to export SBOM (Software Bill of Materials) files. Also, it has been used in empirical studies. Inaccuracies in the dependency graph might negatively affect both the effectiveness of tools and the results of the conducted studies. In this paper, we present the results of a mining study to assess the accuracy of GitHub’s dependency graph in Java and Python open-source software projects. In particular, on April 16th, 2023, we randomly sampled 297 software projects developed in Java and 338 developed in Python (all hosted on GitHub), each using GitHub’s dependency graph. Then, we performed three analyses to assess how accurate GitHub’s dependency graph is: (i) backward analysis, focusing on the accuracy of the dependencies of a given repository, as reported in GitHub’s dependency graph; (ii) forward analysis, focusing on the accuracy of the dependents of a given repository, as reported in GitHub’s dependency graph; and (iii) manifest/lock file analysis, focusing on the correspondence between the dependencies reported in the dependency graph of a given repository and what was reported in the corresponding manifest/lock files. The obtained results highlight several inaccuracies in GitHub’s dependency graph, which might affect the output of tools based on GitHub’s dependency graph (e.g., Dependabot and SBOM generators) as well as the outcomes of past empirical studies. We also provide qualitative insights into these inaccuracies and implications for practitioners and researchers.

MSR4SBOM (Mining Software Repositories for enhanced Software Bills of Materials) is a project whose main goal is to deliver a framework that analyzes the content of software repositories and SBOMs to provide context-sensitive recommendations. The expected outputs are (i) a set of approaches and tools released as open-source projects, making them exploitable in industrial, academic, and open-source contexts; and (ii) replication packages of our empirical studies and repositories of datasets collected while developing, calibrating, and validating the MSR4SBOM approaches and tools.