How Much Can RocksDB Chew? Achieving Near-Zero Write Stalls with Sustainable RocksDB

vldb26-2999 · Regular Research · Hojin Shin, Yongmin Lee, Seehwan Yoo, Jongmoo Choi
Abstract

Modern data-intensive applications, from microservices to real-time AI serving, demand consistently low tail latency from backend storage. However, Log-Structured Merge-tree (LSM)-based Key-Value Stores (KVSs) like RocksDB are structurally prone to unpredictable write stalls. These stalls stem from a fundamental architectural decoupling of foreground write ingress and background data reorganization. By design, the system absorbs foreground writes at maximum speed without monitoring its actual time-varying compaction capacity. As a result, it accumulates internal pressure until rigid capacity thresholds are breached, triggering reactive safeguards that abruptly freeze all foreground writes. Relying on this reactive ``stop-and-go'' approach induces a persistent limit-cycle behavior, undermining long-run predictability and strict latency guarantees. We reframe write stalls as a continuous control problem. S-RocksDB is a sustainable admission controller that regulates foreground ingress to match the system's time-varying compaction capacity. Since this capacity varies at runtime, S-RocksDB employs online reinforcement learning to discover a sustainable admission rate. To ensure safe learning, a three-state operational model (SAFE, SEMI-SAFE, UNSAFE) confines exploration to stable conditions and deploys deterministic guardrails before stalls can occur. In 24-hour evaluations, S-RocksDB reduces over 64.3 million stall events to just 69, bounds P99.99 tail latency to sub-0.11 ms, and delivers predictable throughput with the lowest resource footprint among all compared systems.

Assigned reviewers

No reviewers assigned yet.

Candidates from the panel ranked by taxonomy affinity

#ReviewerMatchLoadWhy