Cloud-native databases have become the de-facto choice for mission-critical applications on the cloud due to the need for high availability, resource elasticity, and cost efficiency. Meanwhile, driven by the increasing connectivity between data generation and analysis, users prefer a single database to efficiently process both OLTP and OLAP workloads, which enhances data freshness and reduces the complexity of data synchronization and the overall business cost. In this paper, we summarize five crucial design goals for a cloud-native HTAP database based on our experience and customers' feedback, i.e., transparency, competitive OLAP performance, minimal perturbation on OLTP workloads, high data freshness, and excellent resource elasticity. As our solution to realize these goals, we present PolarDB-IMCI, a cloud-native HTAP database system designed and deployed at Alibaba Cloud. Our evaluation results show that PolarDB-IMCI is able to handle HTAP efficiently on both experimental and production workloads; notably, it speeds up analytical queries up to ×149 on TPC-H (100GB). PolarDB-IMCI introduces low visibility delay and little performance perturbation on OLTP workloads (<5%), and resource elasticity can be achieved by scaling out in tens of seconds.

Modern fast NVMe devices with high throughput and ultra-low latency have brought new opportunities for persistent key-value (KV) engines. In this paper, we propose Limon, a persistent KV engine to exploit the performance potentials of fast NVMe devices. Limon targets three practical design aspects that existing KV engines fail to consider simultaneously: functionality, scalability, and stability. Limon carefully redesigns the index structure, on-disk KV record layout, and I/O processing of a persistent KV engine for these aspects. Specifically, Limon (i) proposes a semi-shared global index to improve scalability and range queries. (ii) employs a fast slab-based record layout with light-weight defragmentation to enable stable performance, and (iii) uses efficient asynchronous per-core I/O processing with two optimizations: DMA-backed buffer pool and page deduplication, to further improve scalability. Our evaluations with the YCSB benchmark and one production workload show that Limon outperforms state-of-the-art persistent key-value engines (i.e., SpanDB, KVell, and uDepot) by up to 1.2x to 3.8x and has the best scalability. Moreover, Limon has stable and predictable performance due to its novel record layout strategy.

The recent byte-addressable and large-capacity commercialized persistent memory (PM) is promising to drive database as a service (DBaaS) into unchartered territories. This paper investigates how to leverage PMs to revisit the conventional LSM-tree based OLTP storage engines designed for DRAM-SSD hierarchy for DBaaS instances. Specifically we (1) propose a light-weight PM allocator named Hal-loc customized for LSM-tree, (2) build a high-performance Semi-persistent Memtable utilizing the persistent in-memory writes of PM, (3) design a concurrent commit algorithm named Reorder Ring to aschieve log-free transaction processing for OLTP workloads and (4) present a Global Index as the new globally sorted persistent level with non-blocking in-memory compaction. The design of Reorder Ring and Semi-persistent Memtable achieves fast writes without synchronized logging overheads and achieves near instant recovery time. Moreover, the design of Semi-persistent Memtable and Global Index with in-memory compaction enables the byte-addressable persistent levels in PM, which significantly reduces the read and write amplification as well as the background compaction overheads. The overall evaluation shows that the performance of our proposal over PM-SSD hierarchy outperforms the baseline by up to 3.8x in YCSB benchmark and by 2x in TPC-C benchmark.