Buffer management is critical for DBMSs but often suffers from scalability bottlenecks and poor cache locality, which stems from centralized reference counting in page access and intensive locking in page-to-buffer translation. However, prior radical approaches like pointer swizzling or optimistic lock can hardly be adopted in production-grade DBMSs due to its inherent complexity and incompatibility. This paper proposes ScaleCache , a scalable, highly-efficient and production-grade buffer management system with three key designs. ScaleCache first incorporates a novel compact per-group buffer reference counting technique, which enables scalable buffer pinning and unpinning by concurrent threads on many-core servers. It then devised a novel read-write lock based on copy-on-write and per-group reference counting, which is suitable for B-link tree. At last, we propose an optimistic, CPU-cache friendly and SIMD-accelerated hash table for fast and scalable page-to-buffer translation, which eliminates most contention on modern many-core hardware. ScaleCache has been adopted in Huawei GaussDB , a commercial high-performance DBMS. Evaluation on a 128-core server demonstrates that ScaleCache exhibits near-linear scalability and can significantly improve index query throughput of both classic B-link tree index and complex graph-based vector index.

Cloud computing is on the rise, which promotes new breeds of database systems to accommodate the cloud environment. The development of cloud-native databases reveals three trends. One is the adoption of multi-datacenter (DC) deployment to survive the downtime of any single site. Another is the separation of computation and storage resources to achieve higher elasticity and scalability. The last is the support of HTAP to eliminate data redundancy and system complexity from heterogeneous databases. To cater to these trends, we design a distributed relational database called PolarDB-X, which is built on top of the cloud-native database PolarDB. It hence inherits many cloud-native features, such as multi-datacenter deployment and elasticity. To achieve cross-DC capability, it leverages Paxos and hybrid logical clock to achieve durability and snapshot-isolation consistency with low coordination costs. For resource elasticity, since the underlying PolarDB supports rapid migration of tenants between nodes, PolarDB-X can quickly scale the cluster to cope with a sudden traffic increase. For HTAP support, with the help of read replicas and a HTAP executor, PolarDB-X can improve the latency and parallelism of analytical queries without impacting concurrently-running TP workloads. Using its MPP engine and an in-memory column index, the efficiency of analytical queries can be further enhanced. PolarDB-X is now a cloud database service at Alibaba Cloud. We have learned many useful lessons from its development and operation, and have incorporated those into our design and analysis.

Shared-nothing, distributed databases scale transactional and analytical processing over a large data volume by spreading data across servers. However, static sharding of data across nodes makes such systems fail to timely adapt to changing workloads and struggle to obey the cloud pay-as-you-go model. Migrating shards between nodes online is a key technique to react to dynamic changes of workloads for cloud elasticity. Existing approaches introduce severely degraded performance and service interruption, resulting in SLA violation on the cloud; or they are tailor-made to deterministic databases. In this paper, we propose Remus, a new live migration approach for shared-nothing, distributed databases with snapshot isolation. Remus migrates shards between nodes with zero service interruption and minimal performance impact. This is achieved by an efficient unidirectional dual execution during migration. We implement Remus on a shared-nothing, distributed version of PolarDB-PG and evaluate it against state-of-the-art approaches using standard OLTP workloads TPC-C and YCSB, and hybrid workloads consisting of long-lived and short transactions. The results demonstrate Remus is the only effective approach to achieve the goal of zero transaction interruption, zero downtime and marginal performance impact, paving the way for applying the shared-nothing architecture to a cloud database which needs to provide elasticity while guaranteeing strict SLAs.

With the rapid development of fast and byte-addressable non-volatile memories (NVMs), hybrid NVM/DRAM storage systems become promising for computer systems. Existing NVM file systems have already been optimized around the NVM properties. However, they inherit some design choices of block-oriented storage devices that lead to scalability bottlenecks and data copy overhead for ensuring data consistency. In this paper, we present noseFS, a highly non-volatile memory scalable and efficient File System. It is designed to achieve high performance through a bundle of novel techniques: (1) a scalable lightweight naming integrating VFS with the underlying file system namespace, (2) a fine-grained byte-unit file index tree avoiding redundant copy overhead introduced by Copy-On-Write, (3) a lightweight journaling providing atomicity and scalability on many-core platforms, and (4) a lightweight atomic-mmap providing strong consistency guarantee with low overhead by tracking dirty pages. Experimental results show that noseFS performs much better than the state-of-the-art file systems with equally strong data consistency guarantees, and achieves near-linear scalability on a 40-core machine.

OS-level virtualization is often used for server consolidation in data centers because of its high efficiency. However, the sharing of storage stack services among the colocated containers incurs contention on shared kernel data structures and locks within I/O stack, leading to severe performance degradation on manycore platforms incorporating fast storage technologies (e.g., SSDs based on nonvolatile memories). This article presents MultiLanes, a virtualized storage system for OS-level virtualization on manycores. MultiLanes builds an isolated I/O stack on top of a virtualized storage device for each container to eliminate contention on kernel data structures and locks between them, thus scaling them to manycores. Meanwhile, we propose a set of techniques to tune the overhead induced by storage-device virtualization to be negligible, and to scale the virtualized devices to manycores on the host, which itself scales poorly. To reduce the contention within each single container, we further propose SFS, which runs multiple file-system instances through the proposed virtualized storage devices, distributes all files under each directory among the underlying file-system instances, then stacks a unified namespace on top of them. The evaluation of our prototype system built for Linux container (LXC) on a 32-core machine with both a RAM disk and a modern flash-based SSD demonstrates that MultiLanes scales much better than Linux in micro- and macro-benchmarks, bringing significant performance improvements, and that MultiLanes with SFS can further reduce the contention within each single container.

Resource view is the user defined table to provide specific view on resource status in cloud computing environment. It provides a convenient way to retrieve resource data for applications at infrastructure, platform and service layers. But the description and maintenance of these diverse resource views are inconvenient and dramatically difficult due to massive, heterogeneous and dynamic characteristics of the cloud resources involved. In this paper we present a resource view description scheme RQL and the corresponding system Cloud View to address these difficulties. RQL provides users a scheme to specify the data processing flow from resource raw data collected to resource view data objected. By constructing data processing a cyclic graph based on view definitions and using basic routines, view maintenance mechanism update user defined resource views automatically and periodically. Cloud View use a centralized scheduler to distribute maintenance jobs to a set of scalable worker nodes. It leverages distributed key-value database to store view data. Compared to related resource monitoring and discovering systems, Cloud View is flexible in application oriented view description and maintenance. Experiments show it updates typical user defined views with desired performance.

In recent years, the Software as a Service, largely enabled by the Internet, has become an innovative software delivery model. During the streaming execution of virtualization software, the execution will wait until the missing data was downloaded, which greatly influences the user experience. In this paper, we present a block-level prefetching framework for streaming delivery of software based on N-Gram prediction model and an incremental data mining algorithm. The prefetching framework uses the historical block access logs for data mining, then dynamically updates and polishes the prefetching rules. The experimental results show that this prefetching framework achieves a launch time reduced by 10% to 50%, as well as hit rate between 81% and 97%.