In the past decade, cloud databases have experienced rapid development and growth. PolarDB, Alibaba's cloud-native OLTP database, has evolved significantly to meet the increasing demand for cloud-native architectures and now serves hundreds of thousands of customers across various industries. This paper presents PolarDB's evolution over the past eight years, with a focus on scalability, performance, and cost-efficiency. Initially, PolarDB adopted a primary-replica architecture based on disaggregated storage, with an emphasis on enhancing single-node performance for scale-up in modern many-core systems. To achieve this, we co-designed PolarDB with cutting-edge hardware, including RDMA, to improve performance. Meanwhile, we refined the internal architecture, including improvements to B+ tree concurrency control and transaction management, ensuring high scalability in scale-up scenarios. More recently, our focus has shifted to scaling out PolarDB to meet the performance and scalability needs of ultra-large-scale applications. By leveraging RDMA, we optimized distributed transaction processing, transforming PolarDB into a high-performance, high-scalability and cost-effective distributed database. In the TPC-C benchmark, PolarDB scaled out to 2340 nodes and achieved over 2 billion tpmC, with a jitter rate of no more than 0.16% during the 8-hour stress test. Compared to the second- and third-highest-performing databases in public TPC-C results, PolarDB's tpmC is 2.52× and 2.91× higher, respectively. In terms of cost-effectiveness, PolarDB's per-tpmC cost is 37% and 79.5% lower than that of the other two systems, respectively.

Cloud computing is on the rise, which promotes new breeds of database systems to accommodate the cloud environment. The development of cloud-native databases reveals three trends. One is the adoption of multi-datacenter (DC) deployment to survive the downtime of any single site. Another is the separation of computation and storage resources to achieve higher elasticity and scalability. The last is the support of HTAP to eliminate data redundancy and system complexity from heterogeneous databases. To cater to these trends, we design a distributed relational database called PolarDB-X, which is built on top of the cloud-native database PolarDB. It hence inherits many cloud-native features, such as multi-datacenter deployment and elasticity. To achieve cross-DC capability, it leverages Paxos and hybrid logical clock to achieve durability and snapshot-isolation consistency with low coordination costs. For resource elasticity, since the underlying PolarDB supports rapid migration of tenants between nodes, PolarDB-X can quickly scale the cluster to cope with a sudden traffic increase. For HTAP support, with the help of read replicas and a HTAP executor, PolarDB-X can improve the latency and parallelism of analytical queries without impacting concurrently-running TP workloads. Using its MPP engine and an in-memory column index, the efficiency of analytical queries can be further enhanced. PolarDB-X is now a cloud database service at Alibaba Cloud. We have learned many useful lessons from its development and operation, and have incorporated those into our design and analysis.

There has been an increasing interest in building cloud-native databases that decouple computation and storage for elasticity. A cloud-native database often adopts a cloud storage underneath its storage engine, leveraging another layer of virtualization and providing a high-performance and elastic storage service without exposing complex storage details. It helps reduce the maintenance cost and expedite development cycles for the database kernels. We have observed that there are significant differences between the local and the cloud storage that invalid many designs inside existing databases when they are ported to the cloud storage. In this paper, we analyze the challenges and opportunities of both B-tree and LSM-tree-based storage engines when they are deployed on a cloud storage. We propose an optimization framework that guides database developers to transform on-premise databases into their cloud-native counterparts. We use a B+-tree-based InnoDB as a demonstration vehicle where we have implemented a suite of optimizations using the proposed framework and extend such efforts to the LSM-tree-based RocksDB. On both engines, our evaluations show significant performance improvements on the cloud storage.

\beginabstract The trend in the DBMS market is to migrate to the cloud for elasticity, high availability, and lower costs. The traditional, monolithic database architecture is difficult to meet these requirements. With the development of high-speed network and new memory technologies, disaggregated data center has become a reality: it decouples various components from monolithic servers into separated resource pools (e.g., compute, memory, and storage) and connects them through a high-speed network. The next generation cloud native databases should be designed for disaggregated data centers. In this paper, we describe the novel architecture of \name, which follows thedisaggregation design paradigm: the CPU resource on compute nodes is decoupled from remote memory pool and storage pool. Each resource pool grows or shrinks independently, providing \revon-demand provisoning at multiple dimensions while improving reliability. We also design our system to mitigate the inherent penalty brought by resource disaggregation, and introduce optimizations such as optimistic locking and index awared prefetching. Compared to the architecture that uses local resources, \name achieves better dynamic resource provisioning capabilities and 5.3 times faster failure recovery speed, while achieving comparable performance. \endabstract

Racetrack Memory(RM) has attracted abundant attention of memory researchers recently. RM can achieve ultrahigh storage density, fast access velocity and non-volatility. Former research has demonstrated that RM has potential to serve as on-chip cache or main memory. However, RM has more flexibility and difficulty in design space of main memory because it has more device level design parameters. The layout of macro unit (MU) needs trade-off among area, access performance and energy consumption, and its shift operation introduces extra dimension of design space. In this paper, we explore these design parameters and analyze their relationship in memory design space in both device and system levels. Based on the results, we also propose a hybrid MU structure to further optimize read intensive applications. Experimental results demonstrated the existence of regularity between design parameters and performance features. The optimized layout of racetrack MU is suggested for application areas such as big-data and IoT which need cost-effective and energy-efficient memory respectively. Together with hybrid MU structures, RM can be designed with more flexibility so that specific structures are suitable for specific applications which make “All stack optimization” possible in memory structure level.

Quantifying the value of developers’ code contributions to a software project requires more than simply counting lines of code or commits. We define the development value of code as a combination of its structural value (the effect of code reuse) and its non-structural value (the impact on development). We propose techniques to automatically calculate both components of development value and combine them using Learning to Rank. Our preliminary empirical study shows that our analysis yields richer results than those obtained by human assessment or simple counting methods and demonstrates the potential of our approach.

As reliability requirements are increasingly important in both clusters and data centers, RAID-6, which can tolerate any two concurrent disk failures, has been widely used in modern storage systems. However, most existing RAID-6 codes cannot provide satisfied performance on both degraded reads and partial stripe writes, which are important performance metrics in storage systems. To address these problems, in this paper we propose a new RAID-6 MDS erasure code called Short Code, in order to optimize the degraded reads and partial stripe writes. In Short Code, we propose a novel short horizontal parity chain, which assures that all disks contribute to degraded reads while the continuous data elements are more likely to share the same horizontal chain for optimizing degraded reads. On the other hand, Short Code distributes all diagonal parities among disks for optimizing partial stripe writes. The proposed Short Code not only owns the optimal storage efficiency, but also keeps the optimal complexity for both encoding/decoding computations and update operations. The experiments show that Short Code achieves much higher speed on degraded reads and partial stripe writes than other popular RAID-6 codes, and provide acceptable performance on single disk failure recoveries and normal reads. Specifically, compared to RDP code, Short Code provides 6.1 to 26.3 percent higher speed on degraded reads and 36.2 to 80.3 percent higher speed on partial stripe writes with the same number of disks.

Emerging non-volatile memories (NVMs), which include PC-RAM and STT-RAM, have been proposed to replace DRAM, mainly because they have better scalability and lower standby power. However, previous research has demonstrated that these NVMs cannot completely replace DRAM due to either lifetime/performance (PCRAM) or density (STT-RAM) issues. Recently, a new type of emerging NVM, called Racetrack Memory (RM), has attracted more and more attention of memory researchers because it has ultra-high density and fast access speed without the write cycle issue. However, there lacks research on how to leverage RM for main memory. To this end, we explore main memory design based on RM technology in both circuit and architecture levels. In the circuit level, we propose the structure of the RM based main memory and investigate different design parameters. In the architecture level, we design a simple and efficient shift-sense address mapping policy to reduce 95% shift operations for performance improvement and power saving. At the same time, we analyze the efficiency of existing optimization strategies for NVM main memory. Our experiments show that RM can outperform DRAM for main memory, in respect of density, performance, and energy efficiency.

In order to satisfy the competition of multiple GPU accelerated applications and make full use of GPU resources, a lot of previous works propose spatial-multitasking to execute multiple GPGPU applications simultaneously on a single GPU device. However, when adopting the spatial-multitasking framework, the inter-application interference may slow down different applications differently, leading to the unreasonable allocation of shared resources among concurrent GPGPU applications, degrading system fairness severely and resulting in sub-optimal performance. Thus, it is imperative to develop mechanisms to control negative inter-application interactions and utilize shared resources fairly and efficiently. Quantitatively estimating application slowdowns can enable us to accurately minimize system unfairness. Although several previous works pay attention on showdown estimation for CPUs, we find that they may be inaccurate for GPUs. Therefore, we propose a novel Dynamical Application Slowdown Estimation (DASE) model to estimate application slowdowns accurately. Our evaluations show that DASE has significantly lower estimation error (only 8.8%) than the state-of-the-art estimation models (36.3% and 32.8%) across all two-application workloads. Furthermore, to verify the effectiveness of our DASE model, we leverage our model to develop an efficient fairness-oriented Streaming Multiprocessors (SM) allocation policy DASE-Fair to minimize the overall system unfairness. Compared to the even SM partition policy, DASE-Fair improves fairness dramatically by more than 16.1% on average.

Big graph computing can be performed over a single node, using recent systems such as GraphChi and XStream. Breadth-first graph search (a.k.a., BFS) has a pattern of marking each vertex only once as “visited” and then not using them in further computations. Existing single-server graph computing systems fail to take advantage of such access pattern of BFS for performance optimization, hence suffering from a lot of extra I/O latencies due to accessing no longer useful data elements of a big graph as well as wasting plenty of computing resources for processing them. In this paper, we propose FastBFS, a new approach that accelerates breadth-first graph search on a single server by leverage of the preceding access pattern during the BFS iterations over a big graph. First, FastBFS uses an edgecentric graph processing model to obtain the high bandwidth of sequential disk accesses without expensive data preprocessing. Second, with a new asynchronous trimming mechanism, FastBFS can efficiently reduce the size of a big graph by eliminating useless edges in parallel with the computation. Third, FastBFS schedules I/O streams efficiently and can attain greater parallelism if an additional disk is available. We implement FastBFS by modifying the X-Stream system developed by EPFL. Our experimental results show that FastBFS can outperform X-stream and GraphChi in the computing speed by up to 2.1 and 3.9 times faster respectively. With an additional disk, FastBFS can even outperform them by up to 3.6 and 6.5 times faster respectively.

While Phase Change Memory (PCM) has emerged as one of most promising complements or even replacements of DRAM-based memory, it has only limited write endurance. Because of uneven write distribution, PCM is highly likely to have early failures, which can spread over the chip space and leave the entire chip unusable. Wear leveling is an indispensable technique to even out wear caused by the writes. However, because of process variation early failure cannot be fully avoided. State-of-the-art wear-leveling schemes, such as Start-Gap and Security Refresh, cease to function once even a single block failure occurs because their designs require persistent writ able address space for wear leveling operations. Existent solutions attempting to address the problem demand substantial OS supports, such as explicit space allocations and data migrations. The demand on substantial OS cooperation creates a barrier to widespread adoption of the PCM technique. While fault-tolerance techniques, such as FREE-p and zombie, that remap failed blocks to inaccessible but healthy space have the potential to address the wear-leveling issue by relocating data from failed blocks to healthy ones, they cannot work together with the wear-leveling schemes as data migration may change placement of relocated data. In this paper, we propose a framework, WL-Reviver, that allows any in-PCM wear-leveling scheme to keep delivering its designed leveling service even after failures occur in its working address space. The design is unique on two aspects: (1) it leverages the fault-tolerance techniques so that they can work together with the wear leveling schemes, and (2) it requires no OS supports additional to what're available to today's DRAM-based memory system. Furthermore, WL-Reviver is a lightweight framework of very low overhead. Our extensive experiments show that WLReviver can efficiently revive a wear-leveling scheme without compromising the scheme's wear-leveling effect.