High-quality labeled datasets are crucial for training and evaluating foundation models in software engineering, but creating them is often prohibitively expensive and labor-intensive. We introduce SPICE, a scalable, automated pipeline for labeling SWE-bench-style datasets with annotations for issue clarity, test coverage, and effort estimation. SPICE combines context-aware code navigation, rationale-driven prompting, and multi-pass consensus to produce labels that closely approximate expert annotations. SPICE’s design was informed by our own experience and frustration in labeling more than 800 instances from SWE-Gym. SPICE achieves strong agreement with human-labeled SWE-bench Verified data while reducing the cost of labeling 1,000 instances from around $100,000 (manual annotation) to only $5.10. These results demonstrate SPICE’s potential to enable cost-effective, large-scale dataset creation for SE-focused FMs. To support the community, we release both SPICE tool and SPICE Bench, a new dataset of 6,802 SPICE-labeled instances curated from 291 open-source projects in SWE-Gym (over 13x larger than SWE-bench Verified).

Foundation Models (FMs) such as Large Language Models (LLMs) are reshaping the software industry by enabling FMware, systems that integrate these FMs as core components.In this KDD 2025 tutorial, we present a comprehensive exploration of FMware that combines a curated catalogue of challenges with real-world production concerns.We first discuss the state of research and practice in building FMware.We further examine the difficulties in selecting suitable models, aligning high-quality domain-specific data, engineering robust prompts, and orchestrating autonomous agents.We then address the complex journey from impressive demos to production-ready systems by outlining issues in system testing, optimization, deployment, and integration with legacy software.Drawing on our industrial experience and recent research in the area, we provide actionable insights and a technology roadmap for overcoming these challenges.Attendees will gain practical strategies to enable the creation of trustworthy FMware in the evolving technology landscape.

Ethereum is one of the most popular platforms for the development of blockchain-powered applications. These applications are known as ÐApps. When engineering ÐApps, developers need to translate requests captured in the front-end of their application into one or more smart contract transactions. Developers need to pay for these transactions and, the more they pay (i.e., the higher the gas price), the faster the transaction is likely to be processed. Developing cost-effective ÐApps is far from trivial, as developers need to optimize the balance between cost (transaction fees) and user experience (transaction processing times). Online services have been developed to provide transaction issuers (e.g., ÐApp developers) with an estimate of how long transactions will take to be processed given a certain gas price. These estimation services are crucial in the Ethereum domain and several popular wallets such as Metamask rely on them. However, despite their key role, their accuracy has not been empirically investigated so far. In this article, we quantify the transaction processing times in Ethereum, investigate the relationship between processing times and gas prices, and determine the accuracy of state-of-the-practice estimation services. Our results indicate that transactions are processed in a median of 57 seconds and that 90% of the transactions are processed within 8 minutes. We also show that higher gas prices result in faster transaction processing times with diminishing returns. In particular, we observe no practical difference in processing time between expensive and very expensive transactions. With regards to the accuracy of processing time estimation services, we observe that they are equivalent. However, when stratifying transactions by gas prices, we observe that Etherscan’s Gas Tracker is the most accurate estimation service for the very cheap and cheap transactions. EthGasStation’s Gas Price API, in turn, is the most accurate estimation service for regular, expensive, and very expensive transactions. In a post-hoc study, we design a simple linear regression model with only one feature that outperforms the Gas Tracker for very cheap and cheap transactions and that performs as accurately as the EthGasStation model for the remaining categories. Based on our findings, ÐApp developers can make more informed decisions concerning the choice of the gas price of their application-issued transactions.

Programmable blockchain platforms such as Ethereum offer unique benefits to application development, including a decentralized infrastructure, tamper-proof transactions, and auditability. These benefits enable new types of applications that can bring competitive advantage to several business segments. Nonetheless, the pressure of time-to-market combined with relatively immature development technologies (e.g., the Solidity programming language), lack of high-quality training resources, and an unclear roadmap for Ethereum creates a context that favors the introduction of technical debt (e.g., code hacks, workarounds, and suboptimal implementations) into application code. In this paper, we study self-admitted technical debt (SATD) in smart contracts. SATD refers to technical debt that is explicitly acknowledged in the source code by developers via code comments. We extract 726 k real-world contracts from Ethereum and apply both quantitative and qualitative methods in order to (i) determine SATD prevalence, (ii) understand the relationship between code cloning and SATD prevalence, and (iii) uncover the different categories of SATD. Our findings reveal that, while SATD is not a widespread phenomenon (1.5% of real-world contracts contain SATD), SATD does occur in extremely relevant contracts (e.g., multi-million contracts). We also observed a strong connection between SATD prevalence and code cloning activities, leading us to conclude that the former cannot be reliably studied without taking the latter into consideration. Finally, we produced a taxonomy for SATD that consists of 6 major and 26 minor categories. We note that several minor categories are bound to the domain of blockchain and smart contracts, including gas-inefficient implementations and Solidity-induced workarounds. Based on our results, we derive a set of practical recommendations for contract developers and introduce open research questions to guide future research on the topic.

Ethereum is the most popular blockchain platform that supports smart contracts. Smart contracts are computing programs that constitute the building blocks of decentralized applications (DApps). DApps are revolutionary and have led to the creation of entirely new businesses (e.g., marketplaces for digital collectibles). Nonetheless, developing and maintaining DApps lead to entirely new research challenges. Empirical research demands high-quality data, which can be obtained by carefully mining Ethereum. In this tutorial, I will discuss best practices and pitfalls associated with mining Ethereum. The tutorial will be organized into three main parts: (i) a brief introduction to Ethereum and its fundamental concepts, (ii) a hands-on mining session, and (iii) a final Q&A session.

Deprecation is used by developers to discourage the usage of certain features of a software system. Prior studies have focused on the deprecation of source code features, such as API methods. With the advent of software ecosystems, package managers started to allow developers to deprecate higher-level features, such as package releases. This study examines how the deprecation mechanism offered by the${\sf npm}$package manager is used to deprecate releases that are published in the ecosystem. We propose two research questions. In our first RQ, we examine how often package releases are deprecated in${\sf npm}$, ultimately revealing the importance of a deprecation mechanism to the package manager. We found that the proportion of packages that have at least one deprecated release is 3.7 percent and that 66 percent of such packages have deprecated all their releases, preventing client packages to migrate from a deprecated to a replacement release. Also, 31 percent of the partially deprecated packages do not have any replacement release. In addition, we investigate the content of the deprecation messages and identify five rationales behind the deprecation of releases, namely: withdrawal, supersession, defect, test, and incompatibility. In our second RQ, we examine how client packages adopt deprecated releases. We found that, at the time of our data collection, 27 percent of all client packages directly adopt at least one deprecated release and that 54 percent of all client packages transitively adopt at least one deprecated release. The direct adoption of deprecated releases is highly skewed, with the top 40 popular deprecated releases accounting for more than half of all deprecated releases adoption. As a discussion that derives from our findings, we highlight the rudimentary aspect of the deprecation mechanism employed by${\sf npm}$and recommend a set of improvements to this mechanism. These recommendations aim at supporting client packages in detecting deprecated releases, understanding their impact, and coping with them.

Classifier specific (CS) and classifier agnostic (CA) feature importance methods are widely used (often interchangeably) by prior studies to derive feature importance ranks from a defect classifier. However, different feature importance methods are likely to compute different feature importance ranks even for the same dataset and classifier. Hence such interchangeable use of feature importance methods can lead to conclusion instabilities unless there is a strong agreement among different methods. Therefore, in this paper, we evaluate the agreement between the feature importance ranks associated with the studied classifiers through a case study of 18 software projects and six commonly used classifiers. We find that: 1) The computed feature importance ranks by CA and CS methods do not always strongly agree with each other. 2) The computed feature importance ranks by the studied CA methods exhibit a strong agreement including the features reported at top-1 and top-3 ranks for a given dataset and classifier, while even the commonly used CS methods yield vastly different feature importance ranks. Such findings raise concerns about the stability of conclusions across replicated studies. We further observe that the commonly used defect datasets are rife with feature interactions and these feature interactions impact the computed feature importance ranks of the CS methods (not the CA methods). We demonstrate that removing these feature interactions, even with simple methods like CFS improves agreement between the computed feature importance ranks of CA and CS methods. In light of our findings, we provide guidelines for stakeholders and practitioners when performing model interpretation and directions for future research, e.g., future research is needed to investigate the impact of advanced feature interaction removal methods on computed feature importance ranks of different CS methods.

Ethereum is the most popular blockchain platform for the development of blockchain-powered applications (a.k.a, ). Developing a involves translating requests captured in the frontend of an application into contract transactions. However, transactions need to be payed for. Ethereum employs the gas system to charge transaction fees. The gas system has three key components, namely gas price, gas usage, and gas limit. We refer to these components and their interplay as the gas triangle. In this paper, we claim that the inherently complex gas triangle should not be exposed to end-users. We conduct two studies that provide empirical evidence to support our claim. In light of our results, we provide a list of recommendations to novice end-users. We conclude the paper with a list of research challenges that need to be tackled in order to support the development of next-generation that completely hide the gas triangle from end-users.

Ethereum is a blockchain platform that hosts and executes smart contracts. Executing a function of a smart contract burns a certain amount of gas units (a.k.a., gas usage). The total gas usage depends on how much computing power is necessary to carry out the execution of the function. Ethereum follows a free-market policy for deciding the transaction fee for executing a transaction. More specifically, transaction issuers choose how much they are willing to pay for each unit of gas (a.k.a., gas price). The final transaction fee corresponds to the gas price times the gas usage. Miners process transactions to gain mining rewards, which come directly from these transaction fees. The flexibility and the inherent complexity of the gas system pose challenges to the development of blockchain-powered applications. Developers of blockchain-powered applications need to translate requests received in the frontend of their application into one or more smart contract transactions. Yet, it is unclear how developers should set the gas parameters of these transactions given that (i) miners are free to prioritize transactions whichever way they wish and (ii) the gas usage of a contract transaction is only known after the transaction is processed and included in a new block. In this article, we analyze the gas usage of Ethereum transactions that were processed between Oct. 2017 and Feb. 2019 (the Byzantium era). We discover that (i) most miners prioritize transactions based on their gas price only, (ii) 25% of the functions that received at least 10 transactions have an unstable gas usage (coefficient of variation = 19%), and (iii) a simple prediction model that operates on the recent gas usage of a function achieves an R-Squared of 0.76 and a median absolute percentage error of 3.3%. We conclude that (i) blockchain-powered application developers should be aware that transaction prioritization in Ethereum is frequently done based solely on the gas price of transactions (e.g., a higher transaction fee does not necessarily imply a higher transaction priority) and act accordingly and (ii) blockchain-powered application developers can leverage gas usage prediction models similar to ours to make more informed decisions to set the gas price of their transactions. Lastly, based on our findings, we list and discuss promising avenues for future research.

In a software ecosystem, a dependency relationship enables aclientpackage to reuse a certain version of aproviderpackage. Packages in a software ecosystem often release versions containing bug fixes, new functionalities, and security enhancements. Hence, updating the provider version is an important maintenance task for client packages. Despite the number of investigations about dependency updates, there is a lack of studies about dependency downgrades in software ecosystems. A downgrade indicates that the adopted version of a provider package is not suitable to the client package at a certain moment. In this paper, we investigate downgrades in the${\sf npm}$ecosystem. We address three research questions. In our first RQ, we provide a list of the reasons behind the occurrence of downgrades. Our manual analysis of the artifacts (e.g., release notes and commit messages) of a package code repository identified two categories of downgrades according to their rationale: reactive and preventive. The reasons behind reactive downgrades are defects in a specific version of a provider, unexpected feature changes in a provider, and incompatibilities. In turn, preventive downgrades are an attempt to avoid issues in future releases. In our second RQ, we investigate how the versioning of dependencies is modified when a downgrade occurs. We observe that 49 percent of the downgrades are performed by replacing a range of acceptable versions of a provider by a specific old version. This observation suggests that client packages have the tendency to become more conservative regarding the update of their providers after a downgrade. Also, 48 percent of the downgrades reduce the provider version by a minor level (e.g., from 2.1.0 to 2.0.0). This observation indicates that client packages in${\sf npm}$should be cautious when updating minor releases of the provider (e.g., by prioritizing tests). Finally, in our third RQ we observe that 50 percent of the downgrades are performed at a rate that is 2.6 times as slow as the median time-between-releases of their associated client packages. We also observe that downgrades that follow an explicit update of a provider package occur faster than downgrades that follow an implicit update. Explicit updates occur when the provider is updated by means of an explicit change to the versioning specification (i.e., the string used by client packages to define the provider version that they are willing to adopt). We conjecture that, due to the controlled nature of explicit updates, it is easier for client packages to identify the provider that is associated with the problem that motivated the downgrade.

Data science pipelines are a sequence of data processing steps that aim to derive knowledge and insights from raw data. Data science pipeline tools simplify the creation and automation of data science pipelines by providing reusable building blocks that users can drag and drop into their pipelines. Such a graphical, model-driven approach enables users with limited data science expertise to create complex pipelines. However, recent studies show that there exist several data science pitfalls that can yield spurious results and, consequently, misleading insights. Yet, none of the popular pipeline tools have built-in quality control measures to detect these pitfalls. Therefore, in this paper, we propose an approach called Pitfalls Analyzer to detect common pitfalls in data science pipelines. As a proof-of-concept, we implemented a prototype of the Pitfalls Analyzer for KNIME, which is one of the most popular data science pipeline tools. Our prototype is model-driven, since the detection of pitfalls is accomplished using pipelines that were created with KNIME building blocks. To showcase the effectiveness of our approach, we run our prototype on 11 pipelines that were created by KNIME experts for 3 Internet-of-Things (IoT) projects. The results indicate that our prototype flags all and only those instances of the pitfalls that we were able to flag while manually inspecting the pipelines.

Real world object-oriented systems are composed of hundreds or even thousands of classes that are structurally interconnected in many different ways. In this highly complex scenario, it is unclear how changes propagate. Given the high maintenance cost brought by change propagation, these questions become particularly relevant in practice. In this paper, we set out to investigate the influence of structural dependencies on change propagation. We historically analyzed thousands of code snapshots coming from 4 open-source Java projects of different sizes and domains. Our results indicated that, in general, it is more likely that two artifacts will not co-change just because one depends on the other. However, the rate with which an artifact co-changes with another is higher when the former structurally depends on the latter. This rate becomes higher if we track down dependencies to the low-level entities that are changed in commits. This implies, for instance, that developers should be aware of dependencies on methods that are added or changed, as these dependencies tend to propagate changes more often. Finally, we also found several cases where software changes could not be justified using structural dependencies, meaning that co-changes might be induced by other subtler kinds of relationships.

Large and complex workflow repositories include a series of interdependent workflows. In this scenario, it becomes hard to estimate the effort required to accomplish changes to workflows. Furthermore, ad-hoc changes may induce side and ripple effects, which ultimately hamper the reliability of the repository. In this paper, we introduce a static dependency-centric change impact analysis approach for workflow repository management. The approach relies on metrics and visualizations that makes it easy and quick to estimate change impact. We implemented the approach, incorporated it into HP Operations Orchestration (HP OO), and conducted an exploratory study in which we thoroughly analyzed the workflow repository of 8 HP OO customers. Besides being able to characterize and compare the repositories against each other, we found that while the out-of-the-box repository provided by HP OO has 10 flows with high change impact, 5 customer repositories had higher values that ranged from 11 (+10%) to 35 (+250%).