Code translation aims to convert a program from one programming language (PL) to another. This long-standing software engineering task is crucial for modernizing legacy systems, ensuring cross-platform compatibility, enhancing performance, and more. However, automating this process remains challenging due to many syntactic and semantic differences between PLs. Recent studies show that even advanced techniques such as large language models (LLMs), especially open-source LLMs, still struggle with the task. Currently, code LLMs are trained with source code from multiple programming languages, thus presenting multilingual capabilities. In this paper, we investigate whether such capabilities can be harnessed to enhance code translation. To achieve this goal, we introduce INTERTRANS, an LLM-based automated code translation approach that, in contrast to existing approaches, leverages intermediate translations to bridge the syntactic and semantic gaps between source and target PLs. INTERTRANS contains two stages. It first utilizes a novel Tree of Code Translation (ToCT) algorithm to plan transitive intermediate translation sequences between a given source and target PL, then validates them in a specific order. We evaluate INTERTRANS with three open LLMs on three benchmarks (i.e., CodeNet, HumanEval-X, and TransCoder) involving six PLs. Results show an absolute improvement of 18.3% to 43.3% in Computation Accuracy (CA) for INTERTRANS over Direct Translation with 10 attempts. The best-performing variant of INTERTRANS (with the Magicoder LLM) achieved an average CA of 87.3%-95.4% on three benchmarks.

Foundation Models (FMs) such as Large Language Models (LLMs) are reshaping the software industry by enabling FMware, systems that integrate these FMs as core components.In this KDD 2025 tutorial, we present a comprehensive exploration of FMware that combines a curated catalogue of challenges with real-world production concerns.We first discuss the state of research and practice in building FMware.We further examine the difficulties in selecting suitable models, aligning high-quality domain-specific data, engineering robust prompts, and orchestrating autonomous agents.We then address the complex journey from impressive demos to production-ready systems by outlining issues in system testing, optimization, deployment, and integration with legacy software.Drawing on our industrial experience and recent research in the area, we provide actionable insights and a technology roadmap for overcoming these challenges.Attendees will gain practical strategies to enable the creation of trustworthy FMware in the evolving technology landscape.

Vulnerability disclosure is critical for ensuring the security and reliability of open source software (OSS). However, in practice, many vulnerabilities are reported and discussed on public platforms before being formally disclosed, posing significant risks to vulnerability management. Inadequate vulnerability disclosure can expose users to security threats and severely impact the stability and reliability of software systems. For example, prior work shows that over 21% of CVEs are publicly discussed before a patch is released. Despite its importance, we still lack clarity on the vulnerability disclosure practices adopted by open source communities and the preferences of practitioners regarding vulnerability management. To fill this gap, we analyzed the vulnerability disclosure practices of 8,073 OSS projects spanning from 2017 to 2023. We then conducted an empirical study by surveying practitioners about their preferences and recommendations in vulnerability disclosure management. Finally, we compared the survey results with the actual vulnerability practice observed within the OSS projects. Our results show that while over 80% of practitioners support Coordinated Vulnerability Disclosure (CVD), only 55% of vulnerabilities conform to CVD in practice. Although only 20% of practitioners advocate discussions before disclosure, 42% of vulnerabilities are discussed in issue reports before their disclosure. This study reveals the vulnerability management practices in OSS, provides valuable guidance to OSS owners, and highlights potential directions to improve the security of OSS platforms.

Foundation models (FM), such as large language models (LLMs), which are large-scale machine learning (ML) models, have demonstrated remarkable adaptability in various downstream software engineering (SE) tasks, such as code completion, code understanding, and software development. As a result, FM leaderboards have become essential tools for SE teams to compare and select the best third-party FMs for their specific products and purposes. However, the lack of standardized guidelines for FM evaluation and comparison threatens the transparency of FM leaderboards and limits stakeholders’ ability to perform effective FM selection. As a first step towards addressing this challenge, our research focuses on understanding how these FM leaderboards operate in real-world scenarios (“leaderboard operations”) and identifying potential pitfalls and areas for improvement (“leaderboard smells”). In this regard, we collect up to$1,045$FM leaderboards from five different sources: GitHub, Hugging Face Spaces, Papers With Code, spreadsheet and independent platform, to examine their documentation and engage in direct communication with leaderboard operators to understand their workflows. Through card sorting and negotiated agreement, we identify five distinct workflow patterns and develop a domain model that captures the key components and their interactions within these workflows. We then identify eight unique types of leaderboard smells in LBOps. By mitigating these smells, SE teams can improve transparency, accountability, and collaboration in current LBOps practices, fostering a more robust and responsible ecosystem for FM comparison and selection.

Understanding code coverage is an important precursor to software maintenance activities (e.g., better testing). Although modern code coverage tools provide key insights, they typically rely on code instrumentation, resulting in significant performance overhead. An alternative approach to code instrumentation is to process an application's source code and the associated log traces in tandem. This so-called “log-based code coverage” approach does not impose the same performance overhead as code instrumentation. Chen et al. proposedLogCoCo— a tool that implements log-based code coverage forJava. WhileLogCoCobreaks important new ground, it has fundamental limitations, namely: uncertainty due to the lack of logging statements in conditional branches, and imprecision caused by dependency injection. In this study, we proposeLog2Cov, a tool that generates log-based code coverage for programs written inPythonand addresses uncertainty and imprecision issues. We evaluateLog2Covon three large and active open-source systems. More specifically, we compare the performance ofLog2Covto that ofCoverage.py, an instrumentation-based coverage tool forPython. Our results indicate that 1)Log2Covachieves high precision without introducing runtime overhead; and 2) uncertainty and imprecision can be reduced by up to 11% by statically analyzing the program's source code and execution logs, without requiring additional logging instrumentation from developers. While our enhancements make substantial improvements, we find that future work is needed to handle conditional statements and exception handling blocks to achieve parity with instrumentation-based approaches. We conclude the paper by drawing attention to these promising directions for future work.

Programming language documentation refers to the set of technical documents that provide application developers with a description of the high-level concepts of a language (e.g., manuals, tutorials, and API references). Such documentation is essential to support application developers in effectively using a programming language. One of the challenges faced by documenters (i.e., personnel that design and produce documentation for a programming language) is to ensure that documentation has relevant information that aligns with the concrete needs of developers, defined as the missing knowledge that developers acquire via voluntary search. In this article, we present an automated approach to support documenters in evaluating the differences and similarities between the concrete information need of developers and the current state of documentation (a problem that we refer to as the topical alignment of a programming language documentation). Our approach leverages semi-supervised topic modelling that uses domain knowledge to guide the derivation of topics. We initially train a baseline topic model from a set of Rust -related Q&A posts. We then use this baseline model to determine the distribution of topic probabilities of each document of the official Rust documentation. Afterwards, we assess the similarities and differences between the topics of the Q&A posts and the official documentation. Our results show a relatively high level of topical alignment in Rust documentation. Still, information about specific topics is scarce in both the Q&A websites and the documentation, particularly related topics with programming niches such as network, game, and database development. For other topics (e.g., related topics with language features such as structs, patterns and matchings, and foreign function interface), information is only available on Q&A websites while lacking in the official documentation. Finally, we discuss implications for programming language documenters, particularly how to leverage our approach to prioritize topics that should be added to the documentation.

Dependency management bots are increasingly being used to support the software development process, for example, to automatically update a dependency when a new version is available. Yet, human intervention is often required to either accept or reject any action or recommendation the bot creates. In this article, our objective is to study the extent to which dependency management bots create additional, and sometimes unnecessary, work for their users. To accomplish this, we analyze 93,196 issue reports opened by Greenkeeper , a popular dependency management bot used in open source software projects in the npm ecosystem. We find that Greenkeeper is responsible for half of all issues reported in client projects, inducing a significant amount of overhead that must be addressed by clients, since many of these issues were created as a result of Greenkeeper taking incorrect action on a dependency update (i.e., false alarms). Reverting a broken dependency update to an older version, which is a potential solution that requires the least overhead and is automatically attempted by Greenkeeper , turns out to not be an effective mechanism. Finally, we observe that 56% of the commits referenced by Greenkeeper issue reports only change the client’s dependency specification file to resolve the issue. Based on our findings, we argue that dependency management bots should (i) be configurable to allow clients to reduce the amount of generated activity by the bots, (ii) take into consideration more sources of information than only the pass/fail status of the client’s build pipeline to help eliminate false alarms, and (iii) provide more effective incentives to encourage clients to resolve dependency issues.

Complex software systems have a network of dependencies. Developers often configure package managers (e.g., npm ) to automatically update dependencies with each publication of new releases containing bug fixes and new features. When a dependency release introduces backward-incompatible changes, commonly known as breaking changes , dependent packages may not build anymore. This may indirectly impact downstream packages, but the impact of breaking changes and how dependent packages recover from these breaking changes remain unclear. To close this gap, we investigated the manifestation of breaking changes in the npm ecosystem, focusing on cases where packages’ builds are impacted by breaking changes from their dependencies. We measured the extent to which breaking changes affect dependent packages. Our analyses show that around 12% of the dependent packages and 14% of their releases were impacted by a breaking change during updates of non-major releases of their dependencies. We observed that, from all of the manifesting breaking changes, 44% were introduced in both minor and patch releases, which in principle should be backward compatible. Clients recovered themselves from these breaking changes in half of the cases, most frequently by upgrading or downgrading the provider’s version without changing the versioning configuration in the package manager. We expect that these results help developers understand the potential impact of such changes and recover from them.

Cargo, the software packaging manager of Rust, provides a yank mechanism to support release-level deprecation, which can prevent packages from depending on yanked releases. Most prior studies focused on code-level (i.e., deprecated APIs) and package-level deprecation (i.e., deprecated packages). However, few studies have focused on release-level deprecation. In this study, we investigate how often and how the yank mechanism is used, the rationales behind its usage, and the adoption of yanked releases in the Cargo ecosystem. Our study shows that 9.6% of the packages in Cargo have at least one yanked release, and the proportion of yanked releases kept increasing from 2014 to 2020. Package owners yank releases for other reasons than withdrawing a defective release, such as fixing a release that does not follow semantic versioning or indicating a package is removed or replaced. In addition, we found that 46% of the packages directly adopted at least one yanked release and the yanked releases propagated through the dependency network, which leads to 1.4% of the releases in the ecosystem having unresolved dependencies.

The coordinated vulnerability disclosure (CVD) process is commonly adopted for open source software (OSS) vulnerability management, which suggests to privately report the discovered vulnerabilities and keep relevant information secret until the official disclosure. However, in practice, due to various reasons (e.g., lacking security domain expertise or the sense of security management), many vulnerabilities are first reported via public issue reports (IRs) before its official disclosure. Such IRs are dangerous IRs, since attackers can take advantages of the leaked vulnerability information to launch zero-day attacks. It is crucial to identify such dangerous IRs at an early stage, such that OSS users can start the vulnerability remediation process earlier and OSS maintainers can timely manage the dangerous IRs. In this paper, we propose and evaluate a deep learning based approach, namely MemVul, to automatically identify dangerous IRs at the time they are reported. MemVul augments the neural networks with a memory component, which stores the external vulnerability knowledge from Common Weakness Enumeration (CWE). We rely on publicly accessible CVE-referred IRs (CIRs) to operationalize the concept of dangerous IR. We mine 3,937 CIRs distributed across 1,390 OSS projects hosted on GitHub. Evaluated under a practical scenario of high data imbalance, MemVul achieves the best trade-off between precision and recall among all baselines. In particular, the F1-score of MemVul (i.e., 0.49) improves the best performing baseline by 44%. For IRs that are predicted as CIRs but not reported to CVE, we conduct a user study to investigate their usefulness to OSS stakeholders. We observe that 82% (41 out of 50) of these IRs are security-related and 28 of them are suggested by security experts to be publicly disclosed, indicating MemVul is capable of identifying undisclosed dangerous IRs.

Deprecation is used by developers to discourage the usage of certain features of a software system. Prior studies have focused on the deprecation of source code features, such as API methods. With the advent of software ecosystems, package managers started to allow developers to deprecate higher-level features, such as package releases. This study examines how the deprecation mechanism offered by the${\sf npm}$package manager is used to deprecate releases that are published in the ecosystem. We propose two research questions. In our first RQ, we examine how often package releases are deprecated in${\sf npm}$, ultimately revealing the importance of a deprecation mechanism to the package manager. We found that the proportion of packages that have at least one deprecated release is 3.7 percent and that 66 percent of such packages have deprecated all their releases, preventing client packages to migrate from a deprecated to a replacement release. Also, 31 percent of the partially deprecated packages do not have any replacement release. In addition, we investigate the content of the deprecation messages and identify five rationales behind the deprecation of releases, namely: withdrawal, supersession, defect, test, and incompatibility. In our second RQ, we examine how client packages adopt deprecated releases. We found that, at the time of our data collection, 27 percent of all client packages directly adopt at least one deprecated release and that 54 percent of all client packages transitively adopt at least one deprecated release. The direct adoption of deprecated releases is highly skewed, with the top 40 popular deprecated releases accounting for more than half of all deprecated releases adoption. As a discussion that derives from our findings, we highlight the rudimentary aspect of the deprecation mechanism employed by${\sf npm}$and recommend a set of improvements to this mechanism. These recommendations aim at supporting client packages in detecting deprecated releases, understanding their impact, and coping with them.

Build verifiability is a safety property for a software system which can be used to check against various security-related issues during the build process. In summary, a verifiable build generates equivalent build artifacts for every build instance, allowing independent auditors to verify that the generated artifacts correspond to their source code. Producing a verifiable build is a very challenging problem, as non-equivalences in the build artifacts can be caused by non-determinsm from the build environment, the build toolchain, or the system implementation. Existing research and practices on build verifiability mainly focus on remediating sources of non-determinism. However, such a process does not work well with large-scale commercial systems (LSCSs) due to their stringent security requirements, complex third party dependencies, and large volumes of code changes. In this paper, we present an experience report on using a unified process and a toolkit to produce verifiable builds for LSCSs. A unified process contrasts with the existing practices in which recommendations to mitigate sources of non-determinism are proposed on a case-by-case basis and are not codified in a comprehensive tool. Our approach supports the following three strategies to systematically mitigate non-equivalences in the build artifacts: remediation, controlling, and interpretation. Case study on three LSCSs within${{\sf Huawei}}$shows that our approach is able to increase the proportion of verified build artifacts from less than 50 to 100 percent. To cross-validate our approach, we successfully applied our approach to build 2,218 open source packages distributed under${{\sf CentOS}}$7.8, increasing the proportion of verified build artifacts from 85 to 99 percent with minimal human intervention. We also provide an overview of our mitigation guideline, which describes the recommended strategies to mitigate various non-equivalences. Finally, we present some discussions and open research problems in this area based on our experience and lessons learned in the past few years of applying our approach within the company. This paper will be useful for practitioners and software engineering researchers who are interested in build verifiability.

In a software ecosystem, a dependency relationship enables aclientpackage to reuse a certain version of aproviderpackage. Packages in a software ecosystem often release versions containing bug fixes, new functionalities, and security enhancements. Hence, updating the provider version is an important maintenance task for client packages. Despite the number of investigations about dependency updates, there is a lack of studies about dependency downgrades in software ecosystems. A downgrade indicates that the adopted version of a provider package is not suitable to the client package at a certain moment. In this paper, we investigate downgrades in the${\sf npm}$ecosystem. We address three research questions. In our first RQ, we provide a list of the reasons behind the occurrence of downgrades. Our manual analysis of the artifacts (e.g., release notes and commit messages) of a package code repository identified two categories of downgrades according to their rationale: reactive and preventive. The reasons behind reactive downgrades are defects in a specific version of a provider, unexpected feature changes in a provider, and incompatibilities. In turn, preventive downgrades are an attempt to avoid issues in future releases. In our second RQ, we investigate how the versioning of dependencies is modified when a downgrade occurs. We observe that 49 percent of the downgrades are performed by replacing a range of acceptable versions of a provider by a specific old version. This observation suggests that client packages have the tendency to become more conservative regarding the update of their providers after a downgrade. Also, 48 percent of the downgrades reduce the provider version by a minor level (e.g., from 2.1.0 to 2.0.0). This observation indicates that client packages in${\sf npm}$should be cautious when updating minor releases of the provider (e.g., by prioritizing tests). Finally, in our third RQ we observe that 50 percent of the downgrades are performed at a rate that is 2.6 times as slow as the median time-between-releases of their associated client packages. We also observe that downgrades that follow an explicit update of a provider package occur faster than downgrades that follow an implicit update. Explicit updates occur when the provider is updated by means of an explicit change to the versioning specification (i.e., the string used by client packages to define the provider version that they are willing to adopt). We conjecture that, due to the controlled nature of explicit updates, it is easier for client packages to identify the provider that is associated with the problem that motivated the downgrade.