State Machine Replication (SMR) is a foundational technique for building fault-tolerant distributed systems. It underpins infrastructure across cloud platforms, databases, and microservices, yet remains surprisingly difficult to implement efficiently in real world. While prior works in both academia and industry technical blogs have explored individual components, such as consensus protocols or deployment techniques, there is still no clear, integrated guide for building high-performance SMR systems end-to-end.

Cloud storage systems often adopt state machine replication (SMR) to ensure reliability and availability. Most SMR systems use "full-copy" replication across all nodes, which leads to degraded performance for data-intensive workloads, due to high disk and network I/O costs. Erasure coding has recently been integrated with leader-based SMR systems to reduce the costs, e.g., RS-Paxos, CRaft, HRaft, and FRaft. However, these systems still have bottlenecks at the leader, limiting their performance when handling large datasets.

Cloud storage systems often use state machine replication (SMR) to ensure reliability and availability. Erasure coding has recently been integrated with SMR to reduce disk and network I/O costs. This brief announcement shares our experience in developing a leaderless erasure coding SMR system. We integrate our system Racos with etcd, a distributed key-value storage that powers Kubernetes. Racos outperforms competitors by up to 3.36x in throughput.