Elastic Block Storage (EBS) has a pivotal role in modern data center infrastructure, providing reliable, high-performance and flexible block storage service to users. In Alibaba Cloud, EBS is the most widely used service and has been supporting the operation of millions of virtual disks. However, even with layers of load balancing and caching, we still observe significant traffic skewness across the EBS stack. This motivates us to comprehensively investigate symptoms and root causes behind the traffic patterns and, more importantly, explore the fixes for the identified issues.

In this paper, we qualitatively and quantitatively discuss the design choices, production experience, and lessons in building the Elastic Block Storage ( EBS ) at Alibaba Cloud over the past decade. To cope with hardware advancement and users’ demands, we shift our focus from design simplicity in EBS1 to high performance and space efficiency in EBS2 , and finally reducing network traffic amplification in EBS3 . In addition to the architectural evolutions, we also summarize development lessons and experiences as four topics, including: (i) achieving high elasticity in latency, throughput, IOPS, and capacity; (ii) improving availability by minimizing the blast radius of individual, regional, and global failure events; (iii) identifying the motivations and key tradeoffs in various hardware offloading solutions; and (iv) identifying the pros/cons of alternative solutions and explaining why seemingly promising ideas would not work in practice.

Solid-state drives (SSDs) are widely used in current storage systems. SSD manufacturers usually aggregate physical blocks across planes into one superblock to improve SSD performance. However, due to the wear endurance variations among blocks and pages, the standard superblock scheme leads to lifetime and space waste of SSDs. Superblock granularity also causes higher and more uncertain garbage collection overhead. To address these problems, this work proposes endurance-driven dynamic superblock management calledEddySuperblockto provide a longer lifetime and higher space utilization with a faster and deterministic response time, which designs a dynamic superblock organization strategy that links free strong blocks into one superblock according to the real-time wear index of each block and introduces page-level wear management to further unlock the lifetime potential of pages. To reduce the high garbage collection overhead caused by superblocks, a nonblocking parallel garbage collection policy is proposed, which can reclaim space and serve requests simultaneously with low overhead. We conduct a set of experiments on a real hardware experiment platform. Both the proposedEddySuperblockand representative superblock schemes have been implemented on the experiment platform. The experimental results show that our scheme greatly reduces the average system response time by 47.6% and prolongs the lifetime by 75.6% compared with the state-of-the-art superblock scheme, and the final-state flash utilization reaches 95.4%. In the end, this article presents the application status of our work in real-world scenarios.