The performance assessment of blockchain applications holds significant challenges due to their decentralized architecture, immutable smart contracts, distributed ledgers, and operational costs such as gas fees. Existing blockchain benchmarks often either fail to fully capture blockchain-specific behaviors or offer limited configurability and metric reporting. In this paper, we present a new and comprehensive benchmark designed explicitly for blockchain applications, named bBench. Building on established principles from traditional benchmarking and by specializing them in the blockchain context and supported by customized blockchain tools (i.e., Hyperledger Caliper, web3.eth, and node-os-utils), bBench characterizes blockchain application performance in four dimensions: network performance, resource utilization, storage usage, and operational cost. We demonstrate the effectiveness of our benchmark through a case study involving 12 smart contract applications with varying performance demands, some of which hold known vulnerabilities. The results show the benchmark’s ability to quantify performance deviations across different applications, as well as those caused by the activation of specific vulnerabilities.

Blockchain has recently become very popular due to its use in cryptocurrencies and potential application in various domains (e.g., retail, healthcare, and insurance). The smart contract is a key part of blockchain systems and specifies an agreement between transaction participants. Nowadays, smart contracts are being deployed to carry residual faults, including severe vulnerabilities that lead to different types of failures at runtime. Fault detection tools can be used to detect faults that may then be removed from the code before deployment. However, in the case of smart contracts, the common opinion is that tools are immature and ineffective. In this work, we carry out a fault injection campaign to empirically analyze the runtime impact that realistic faults present in smart contracts may have on the reliability of blockchain systems. We pay particular attention to the faults that elude popular smart contract verification tools and show if and in which ways the faults lead the blockchain system to fail at runtime. We map the observations to the fault detection capabilities of three state-of-the-art fault detection tools, namely Mythril, Slither, and Securify. The results show that the tools individually have poor detection capabilities (e.g., Securify with 6.4% accuracy and Mythril with 60% accuracy) or tend to generate false alerts (i.e., only 1.74% of Slither's alerts are correct). The results also show several elusive faults responsible for severe blockchain failures, such as A_MCV, which impacts the integrity of the ledger, and I_MVMSV, which causes gas depletion, just to name a few.

Abstract Smart contracts are nowadays at the core of most blockchain systems. Like all computer programs, smart contracts are subject to the presence of residual faults, including severe security vulnerabilities. However, the key distinction lies in how these vulnerabilities are addressed. In smart contracts, when a vulnerability is identified, the affected contract must be terminated within the blockchain, as due to the immutable nature of blockchains, it is impossible to patch a contract once deployed. In this context, research efforts have been focused on proactively preventing the deployment of smart contracts containing vulnerabilities, mainly through the development of vulnerability detection tools. Along with these efforts, several heterogeneous vulnerability classification schemes appeared (e.g., most notably DASP and SWC). At the time of writing, these are mostly outdated initiatives, even though new smart contract vulnerabilities are consistently uncovered. In this paper, we propose OpenSCV, a new and Open hierarchical taxonomy for Smart Contract vulnerabilities, which is open to community contributions and matches the current state of the practice while being prepared to handle future modifications and evolution. The taxonomy was built based on the analysis of the existing research on vulnerability classification, community-maintained classification schemes, and research on smart contract vulnerability detection. We show how OpenSCV covers the announced detection ability of the current vulnerability detection tools and highlight its usefulness in smart contract vulnerability research. To validate OpenSCV, we performed an expert-based analysis wherein we invited multiple experts engaged in smart contract security research to participate in a questionnaire. The feedback from these experts indicated that the categories in OpenSCV are representative, clear, easily understandable, comprehensive, and highly useful. Regarding the vulnerabilities, the experts confirmed that they are easily understandable.