In modern datacenters, memory disaggregation un-packs monolithic servers to build independent network-connected compute and memory pools, greatly improving resource uti-lization. Existing memory-disaggregated key-value stores adopt ownership sharing or ownership partitioning for request scheduling and they all show poor performance and scalability. The former generates multiple cache misses and multiple network round trips due to poor cache locality, while the latter fails to fully use resources on all Compute Nodes (CNs) due to load imbalance. In this paper, we propose Seastar, a fast and scalable key-value store that achieves high cache locality as well as load balancing. To enable high concurrency and ensure linearizability, Seastar exploits a self-verification version chain to provide out-of-place atomic updates and ensure consistent reads. To improve cache locality, Seastar leverages a pair-based ownership assigning scheme to assign the processing permission of requests corre-sponding to each key to a CN pair through two independent hash functions. To ensure load balancing among CNs, Seastar proposes a decentralized power-of-two-choices routing scheme to route the requests to the less-loaded CNs in CN pairs for handling. Experimental results demonstrate that Seastar improves the throughput by up to 6.7x and significantly reduces the latency compared with state-of-the-art memory-disaggregated key-value stores.

Disaggregated memory architecture has risen in popularity for large datacenters with the advantage of improved resource utilization, failure isolation, and elasticity. Replicated state machines (RSMs) have been extensively used for reliability and consistency. In traditional RSM protocols, each replica stores replicated data and has the computing power to participate in some part of the protocols. However, traditional RSM protocols fail to work in the disaggregated memory architecture due to asymmetric resources on CPU nodes and memory nodes. This article proposes ECHO, a fast one-sided RDMA-based RSM protocol with lightweight log replication and remote applying, efficient linearizability guarantee, and fast coordinator failure recovery. ECHO enables all operations in the protocol to be efficiently executed using only one-sided RDMA, without the participation of any computing resource in the memory pool. To provide lightweight log replication and remote applying, ECHO couples the replicated log and the state machine to avoid dual-copy and performs remote applying by updating pointers. To enable efficient remote log state management, ECHO leverages a hitchhiked log state updating scheme to eliminate extra network round trips. To provide efficient linearizability guarantee, ECHO performs immediate remote applying after log replication and leverages the local locks at the coordinator to ensure linear consistency. Moreover, ECHO adopts a commit-aware log cache to make data visible immediately after being committed. To achieve fast failure recovery, ECHO leverages a commit point identification scheme to reduce the overhead of log consistency recovery. Experimental results demonstrate that ECHO outperforms the state-of-the-art RSM protocol (namely Sift) in multiple scenarios. For example, ECHO achieves 27%–52% higher throughput on typical write-intensive workloads. Moreover, ECHO reduces the consistency recovery time by three orders of magnitude for coordinator failure.

For performance benefits, recent trends of modern data centers tend to use NVM (Non-volatile Memory) as storage and utilize one-sided primitives of RDMA (Remote Direct Memory Access) to directly access NVM for I/O requests. However, one-sided RDMA doesn’t have durability semantics currently, and data may partially exist in the NVM when crashes happen, thus causing unexpected data inconsistency.