Blockchains promise to make online services more fault tolerant because they are replicated on a distributed system of nodes. Their nodes typically run different implementations of the same protocol across different geo-distributed regions, making the protocol supposedly tolerant to various failures including isolated crashes, transient failures, network partitions or attacks. Unfortunately, their fault tolerance has never been compared.

With the recent advent of blockchains, we have witnessed a plethora of blockchain proposals. These proposals range from using work to using time, storage or stake in order to select blocks to be appended to the chain. As a drawback it makes it difficult for the application developer to choose the right blockchain to support their applications. In particular, the scalability and performance one can obtain from a specific blockchain is typically unknown. The claimed results are often obtained in isolation by the developers of the blockchain themselves. The experimental conditions corresponding to these results are generally missing and the lack of details make these results irreproducible.