Many cloud databases provide fine-grained regular snapshots and sparsely deleted snapshots based on importance, and dynamically maintain large-scale snapshots to ensure data security and mine the value of cold data. However, in existing snapshot technologies, the write amplification feature of Copy-on-Write (CoW) introduces additional expensive I/O operations in a cloud environment. In Redirect-on-Write (RoW), the modified data blocks are scattered among the snapshots, resulting in a dependency between the snapshots, which seriously affects the recovery performance. In this paper, we observed that access to snapshots has the characteristics of locality and continuity. We therefore propose an efficient Batch-Based Snapshot index, called BBS, which batches snapshot indexes according to database workload and access behavior of snapshots. Specifically, we use two key techniques: Shared-Subtrees Indexing and Batch-Based Dividing, to perform split dependency of the snapshot index. The snapshot index dependency chain is divided into batches, and there is no dependency on snapshot indexes between batches. In-batch snapshot indexes reduce memory overhead by sharing subtrees. The index can directly locate data blocks instead of iterative traversal. At the same time, the design of the snapshot index deletion method is adapted to the snapshot sparse deletion model. We have implemented a working system in Ceph. Evaluation results on datasets demonstrate that, compared with existing techniques, BBS can effectively balance the overhead between index memory capacity and recovery time.

Emerging persistent memory (PM) technologies offer data persistence at close-to-DRAM latency, causing the legacy multi-layer storage stack to become a bottleneck. On the other hand, PM improves I/O efficiency while also introducing unique hardware features that require software modifications to exploit them. In this paper, we propose HYTE, a native table storage engine for hybrid memory, which abstracts the data service for the database’s table. HYTE performs a cross-layer design across the file system and storage engine to avoid the pitfalls of layered abstraction and provides SQL-compatible APIs. Meanwhile, it co-designs with PM’s properties to unlock the hardware’s performance potential. Furthermore, HYTE is hotness-aware and equipped with a suite of lightweight, customized fetching and eviction strategies. Our evaluation shows that HYTE can perform up to 1.4-7.2× better than existing state-of-the-art PM-based storage systems in industry and academia.

Distributed in-memory computing systems have dramatic performance improvement over traditional disk-based systems, which makes them widely used in large-scale data processing applications. Unfortunately, uneven and unpredictable data distributions caused by data skew have a significant impact on the performance. In Spark, when data skew happens, some tasks will process much more data than other tasks and become the performance bottleneck. The traditional approaches to handling data skew are based on sampling and repartitioning, which incur additional overhead. In this paper, we divide data skew in distributed data processing systems into intra-node and inter-node skew. Based on data stealing, we proposed DS2to handle both intra-node and inter-node data skew. It aims to improve the performance under data skew, without involving additional overhead. DS2first balances the skewed data distribution in the local and then handles the inter-node skew by RDMA during execution. It achieves up to 2.96× speedup on the aggregation operator and 2.81× speedup on the join operator.