The management of mapping tables within solid-state drives (SSDs) is critical to maintaining high performance and stability. Traditional fixed-granularity mapping approaches (page-level, block-level) face significant challenges in ultra-large-scale 3D NAND flash memory, including high DRAM cache overhead resulting from page mapping and high write amplification caused by block mapping. Existing hybrid mapping solutions fail to adapt to the dynamic-varying workloads and large-capacity SSDs, resulting in unstable performance and poor scalability. In this paper, we propose a novel light-weight learning-based combination mapping technique called CoMap, which can adaptively choose the optimal mapping scheme so as to effectively prevent excessive page-mapping table sizes and poor block mapping performance in large-capacity SSDs. Comprehensive evaluations with variety of real-world workloads and enterprise-class SSD simulator show that CoMap, with its strong adaptability, can reduce 60% of latency and 30% of write amplification on average compared with several state-of-the-art methods.

For a long time, fully exploiting the multi-level parallelism of SSDs has been an important technique for optimizing SSD performance. By distributing data to different flash parallel units through address mapping policies, SSDs can achieve multi-task parallel processing, thereby significantly improving data read and write performance. However, our experiments show that when SSD blocking causes task queues to stack up, excessive pursuit of flash parallelism can cause performance interference between multiple users, thereby reducing SSD performance. Additionally, in DRAM-less SSDs, differences in data access patterns directly impact the address mapping module, thereby exacerbating the degree of performance degradation. Based on the above analysis, this paper conducts a systematic study of the impact mechanism of different data flows on address mapping policies and proposes ahybrid addressmapping allocationpolicy with dynamic switching (HMpp) for DRAM-less SSDs. This policy identifies and classifies different data access patterns to dynamically select the appropriate address mapping policy, thereby reducing multi-user interference and improving SSD performance. Experimental results based on real cloud block storage workloads show that this policy can achieve 87.2% performance improvement and 77.3% fairness optimization, while reducing the write amplification factor by 32.9% to 40.2%.

Modern SSDs achieve low latency and high throughput by utilizing multiple levels of SSD parallelism. Fairness is also a critical design consideration in cloud environments and has inspired great interest in recent years to study it. However, we find that excessive striving for SSD parallelism can inadvertently harm the SSD fairness due to the striped data layout. We observe from our experimental analysis that some common access patterns in cloud loads (e.g., high-intensity I/O) are not addressed by SSD parallelism, which can affect both SSD parallelism and fairness. In this paper, we analyze two address mapping policies with orthogonal characteristics from SSD parallelism and fairness perspectives. Based on extensive experiments and observations, we schedule a hybrid address mapping policy to achieve fairness of SSDs in cloud environments by distinguishing different I/O access modes and cold/hot data, while ensuring SSD performance. Experimental results on the latest block-level I/O traces show that our approach significantly improves performance and fairness by up to 77.3% and 62.8%, respectively, over the previous schemes with negligible cost. Meanwhile, it can also reduce the write amplification by 23.5% to 33.5%.

With the popularity of cloud services, cloud block storage (CBS) systems have been widely deployed by cloud providers. Cloud cache plays a vital role in maintaining high and stable performance in cloud block storage systems. In the past few decades, much research has been conducted on the design of cache replacement policies. Prior work frequently relies on manually-engineered heuristics to capture the most common cache access patterns, or predict the reuse distance and try to identify the blocks that are either cache-friendly or cache-averse. Researchers are now applying recent advances in machine learning to guide cache replacement policy, augmenting or replacing traditional heuristics and data structures. However, most existing approaches depend on a certain environment which restricted their application, e.g., some methods only consider the on-chip cache consisting of program counters (PCs). Moreover, those approaches with attractive hit rates are usually unable to deal with modern irregular workloads, due to the limited feature used. In contrast, we propose a cloud cache replacement framework to automatically learn the relationship between the probability distribution of different replacement policies and workload distribution by using deep reinforcement learning. We train an end-to-end cache replacement policy based on the requested address with two efficient and stable cache replacement policies. Furthermore, by using prioritized experience replay and setting parameter constraints, our framework can accelerate the offline training process without affecting the cloud application. We have evaluated our proposed framework by using block-based I/O traces collected from Alibaba Cloud and Tencent Cloud, two of the largest cloud providers in the world, and several open-source traces. Experimental results show that our method not only outperforms several state-of-the-art cache methods in hit rate, but also reduces request latency and data traffic to the backend storage.

Since cloud service providers adopt the sharing mode to improve the utilization of solid state drives (SSDs) and reduce management costs, fairness is a critical design consideration and has drawn great interest in recent years. There are many methods to achieve fairness at the SSD device level, including cache-based resource allocation and queue rescheduling of transaction scheduling unit (TSU). However, poor locality data in cloud environments makes these methods of achieving fairness through the cache level invalid. In addition, existing fairness approaches at the TSU level treat flash memory as a black box and do not consider some characteristics of flash memory, leading to room for performance improvement. In this work, we propose a novel collaborative SSD fairness scheme, named a coordinated SSD cache and TSU fair scheme (CoFS), that achieves end-to-end full path fairness at the device level. A reinforcement learning-assisted fairness management scheme is designed to coordinate the SSD cache and TSU considering both the cache space and bandwidth resources which are significant for fairness control. The key idea is to enable the front-end SSD cache to achieve workload-level fairness by recognizing workload patterns, while the back-end TSU achieves flash-level fairness by sensing SSD internal status. Finally, CoFS collaborates with them to achieve SSD device-level fairness. In addition, we have designed a flexible reward function mechanism in the cache to balance different optimization objectives and augment TSU queue scheduling to adapt to different types of SSDs. Experimental results show that CoFS improves the overall fairness and performance by 30.8% to 56.7% and 42.1% to 98.7% in different scenarios.

The address allocation policy in SSD aims to translate the logical address of I/O requests into a physical address, and the static address allocation is widely used in modern SSD. Through extensive experiments, we find that there are significant differences in the utilization of SSD parallelism among different static address allocation policies. We also observe that the fixed address allocation design prevents SSDs from continuing to meet the challenges posed by cloud workloads and misses the possibility of further optimization. These situations stem from our excessive reliance on SSD parallelism over time. In this paper, we proposeHsaP, ahybridstatic addressallocationpolicy, that adaptively chooses the best static allocation policy to meet the SSD performance at runtime.HsaPis adynamicscheduling scheme based onstaticaddress allocation policy. The static policy ensures thatHsaPhas stable performance and light-weight overhead, while dynamic scheduling can effectively combine different allocation policies, selecting the best-performing static mapping mode for a given SSD state. Meanwhile,HsaPcan further improve the read and write performance of SSDs simultaneously through plane reallocation and data rewrite. Experimental results show thatHsaPachieves significant read and write performance gain of a wide range of the latest cloud block storage traces compared to several state-of-the-art address allocation approaches.

Since multiple flows (users) compete for a single, shared solid state drives (SSDs) concurrently in cloud environments, fairness has drawn great interest in recent years. Although many fair schemes are proposed on different modules of SSDs, such as resource allocation for the cache and chip queue reordering for the transaction scheduling unit (TSU), they fail to continue to provide fairness for the worse locality of I/O requests due to a deeper cloud storage stack that makes cache performance poor. Moreover, existing methods only treat back-end flash as a black box and do not exploit the differences in flash features across requests. This paper proposes CoFS, a novel coordinated SSD cache and TSU light-weight learning-based fair scheme to achieve complete fairness considering cache and bandwidth resources which are significant for fairness control. The key idea is to make the front-end SSD cache achieves fairness at the workload-level by recognizing dynamic I/O changes, while the back-end TSU achieves fairness at the flash-level by awareness of flash page-types, and finally CoFS collaborates with them to achieve fairness at the SSD device-level. Experimental results show that CoFS achieves significant fairness and performance gain of a wide range of the latest cloud block storage workloads, with an average improvement of 46.3% and 94.5% compared to state-of-the-art fairness policies, respectively.

As a vital method for computer system design, multilevel cache technology is still a research hotspot in the field of storage. Recently, researchers have designed many multilevel cache replacement algorithms by analyzing the historical trajectories of data blocks between different cache levels but suffer from high overhead and poor adaptability. In addition, these methods still try to determine the most suitable victim to be replaced, given a new block to be loaded into the cache but do not consider how to cooperate between different cache levels. To overcome these challenges, we propose a multi-factor adaptive multi-level cooperative cache replacement policy in block storage systems that leverages the hierarchical characteristics of multilevel cache and multi-factor orthogonality by the probability distribution. This adaptation is mainly reflected in two aspects: each cache level automatically adapts different cache policies based on I/O access patterns, and different cache levels cooperate by adjusting the dynamic frequency threshold. In this work, we choose as our target for optimization the replacement policy of the cloud block cache, because rich storage stacks in cloud environments can be well used as multi-level cache scenarios. We have evaluated our proposed framework by using block-based I/O traces collected from Alibaba Cloud, one of the largest cloud providers in the world, and several open-source traces. Experimental results show that our method improves the hit ratios by 28.9% and reduces the response time by 9.6% to the state-of-the-art cache replacement policies for multi-level cache across different workloads and cache sizes.

Disk failure has always been a major problem for data centers, leading to data loss. Current disk failure prediction approaches are mostly offline and assume that the disk labels required for training learning models are available and accurate. However, these offline methods are no longer suitable for disk failure prediction tasks in large-scale data centers. Behind this explosive amount of data, most methods do not consider whether it is not easy to get the label values during the training or the obtained label values are not completely accurate. These problems further restrict the development of supervised learning and offline modeling in disk failure prediction. In this article, Active Semi-supervised Learning Disk-failure Prediction ( ASLDP ), a novel disk failure prediction method is proposed, which uses active learning and semi-supervised learning. According to the characteristics of data in the disk lifecycle, ASLDP carries out active learning for those clear labeled samples, which selects valuable samples with the most significant probability uncertainty and eliminates redundancy. For those samples that are unclearly labeled or unlabeled, ASLDP uses semi-supervised learning for pre-labeled by calculating the conditional values of the samples and enhances the generalization ability by active learning. Compared with several state-of-the-art offline and online learning approaches, the results on four realistic datasets from Backblaze and Baidu demonstrate that ASLDP achieves stable failure detection rates of 80–85% with low false alarm rates. In addition, we use a dataset from Alibaba to evaluate the generality of ASLDP . Furthermore, ASLDP can overcome the problem of missing sample labels and data redundancy in large data centers, which are not considered and implemented in all offline learning methods for disk failure prediction to the best of our knowledge. Finally, ASLDP can predict the disk failure 4.9 days in advance with lower overhead and latency.

Disk failure has always been a major problem for data centers, leading to data loss. Current research works used supervised learning to offline training through a large number of labeled samples. However, these offline methods are no longer suitable for disk failure prediction tasks in the current big data environment. Behind this explosive amount of data, most methods do not take into account the label values used in the model training phase may not be easy to obtain, or the obtained label values are not completely accurate. These problems further restrict the development of supervised learning and offline modeling in disk failure prediction. In this paper, ASLDP, a novel disk failure prediction method is proposed, which uses active learning and semi-supervised learning. According to the characteristics of data in the disk life cycle, ASLDP carries out active learning for those clear labeled samples, which selects valuable samples and eliminates redundancy. For those samples that are unclearly labeled or unlabeled, ASLDP combines with semi-supervised learning for pre-labeled, and enhances the generalization ability by active learning. The results on three realistic datasets demonstrate that ASLDP achieves stable failure detection rates of 80-85% with low false alarm rates, compared to current online learning methods. Furthermore, ASLDP can overcome the problems of the sample label missing and data redundancy in the massive data environment, compared to current offline learning methods.