LSM-tree-based key-value stores (KV stores) convert random-write requests to sequence-write ones to achieve high I/O performance. Meanwhile, compaction operations in KV stores update SSTables in forms of reorganizing low-level data components to high-level ones, thereby guaranteeing an orderly data layout in each component. Repeated writes caused by compaction (a.k.a. write amplification) impacts I/O bandwidth and overall system performance. Near-data processing (NDP) is one of the effective approaches to addressing this write-amplification issue. Most NDP-based techniques adopt synchronous parallel schemes to perform a compaction task on both the host and its NDP-enabled device. In synchronous parallel compaction schemes, the execution time of compaction is determined by a subsystem that has lower compaction performance coupled by under-utilized computing resources in a NDP framework. To solve this problem, we propose an asynchronous parallel scheme named PStore to improve the compaction performance in KV stores. In PStore, we designed a multi-tasks queue and three priority-based scheduling methods. PStore elects proper compaction tasks to be offloaded in host- and device-side compaction modules. Our proposed cross-leveled compaction mechanism mitigates write amplification induced by asynchronous compaction. PStore featured with the asynchronous compaction mechanism fully utilizes computing resources in both host- and device-side subsystems. Compared with the two popular synchronous compaction modes based on KV stores (TStore and LevelDB), our PStore immensely improves the throughput by up to a factor of 14 and 10.52 with an average of a factor of 2.09 and 1.73, respectively.

Inside a solid-state disk (SSD), cache stores frequently accessed data to shorten the user-I/O response time and reduce the number of read/write operations in flash memory, thereby improving SSD performance and lifetime. Most existing cache schemes anchor in the spatiotemporal locality of I/O requests in workloads. In the face of a long-time workload, high performance and hit rate often get lost in these caching schemes. Flash memory-aware caching schemes trade hit ratio to prolong SSD lifetime. In this article, we advocate for a machine-learning-based caching scheme named NCache to optimize both hit ratio and SSD performance. In NCache, we construct a machine learning (i.e., ML) model to predict whether data are reaccessed before being evicted from the cache. The cache replacement scheme preferentially evicts data that would not be accessed in the cache. The cache space is conserved for valid data that are likely to be repeatedly accessed. A pipelined scheme is implemented to accelerate the ML model, alleviating the time-cost of NCache. A double-linked list boosts the data addressing and cache replacement process. NCache is orthogonal to the existing caching schemes within the flash translation layer. The results validate NCache under a handful of real-world enterprise traces. Taking prn_0 as an example, NCache reduces the response time of LRU, clean first LRU (CFLRU), GCaR_LRU, GCaR_CFLRU, and LCR by up to 15% with an average of 6.4%. The erase count is slashed by 16% at the maximum. Importantly, NCache is adroit at optimizing write amplification by up to 15.9%.

Erasure-coded archival converts the redundancy mechanism of low access-frequency data from replication to erasure coding for balancing access performance and storage efficiency. A variety of pipelined schemes are designed to speed up the archival operation, however they neglect such three factors as heterogeneous network, under-utilization of replica resources and tight coupling with underlying platforms which restrict or even negate the performance gains. In this paper, we propose Archpipe, a fast and flexible pipelined erasure-coded archival scheme. It exhibits three distinct features: 1) heterogeneous network awareness, for a single-pipelined construction, sufficient-bandwidth links are given high scheduling priority to avoid network congestion, while considering locality to reducing network transmissions; 2) parallel encoding, the unused replica resources are exploited to adaptively construct multiple pipelines for each stripe based on the single-pipelined algorithm, thereby enabling parity blocks to be encoded in parallel; 3) loose coupling, it does not rely on specific block placement policies and stripe construction algorithms. Experimental results indicate that, Archpipe can be seamlessly integrated with common distributed storage systems, and it improves the erasure-coded archival performance by 3.6 ∼ 4.7× and 1.3 ∼ 2.6× in on-disk and in-memory scenarios, respectively.

To satisfy high reliability accompanied by space efficiency requirements, erasure coding is elected to substitute replication as a redundancy mechanism of in-memory clusters. More often than not, the use of erasure coding is limited to read-intensive applications, in which data in erasure-coded clusters are rarely updated. The essential rationale is that update penalty incurred by parity-synchronizations makes long write latency compared with read latency counterpart.In this paper, we propose F-Write: a fast RDMA-supported write optimization scheme for erasure-coded in-memory clusters. It entails two distinct features: 1) an extended version of consistency protocol called Fast2PC is created. It directly modifies remote memory regions using one-sided WRITE verb provided by RDMA to implement the operations of transaction log, and records multiple transactions in the log to submit together, effectively curtailing network latency; 2) a speculative update approach is given to substitute immediate update. Aggregated undo transactions are handled speculatively to synchronize parity blocks in the background when parity blocks are needed. For multiple writes to an identical data block at different times-tamps, only the original and the latest data blocks are involved in calculating parity blocks, thus mitigating encoding latency. Experimental results indicate that F-Write has lower latency, higher throughput compared to the candidate write schemes. Moreover, the impact on recovery time is negligible. Specifically, under the update-intensive workloads, F-Write cuts down write latency by more than 61%, thereby boosting system throughput by a factor of at least 2.6x.

When it comes to NAND Flash-based solid-state disks (SSDs), cache can narrow the performance gap between user-level I/Os and flash memory. Cache management schemes impose relentless impacts on the endurance and performance of flash memory. A vast majority of existing cache management techniques adopt a passive data-update style (e.g., GCaR, LCR), thereby undermining response times in burst I/O requests-based applications11.Burst I/O requests must be served in a real-time manner. This type of I/O access pattern is prevalent in data-intensive workloads.. To address this issue, we propose a collaborative active write-back cache management scheme, called Co-Active, customized for I/O access patterns and the usage status of a flash chip. We design a hot/cold separation module to determine whether data is cold or hot in workload. When a flash chip is idle, cold and dirty data in the cache is flushed into the idle flash chip to produce clean data. To curtail cache replacement cost, clean data are preferentially evicted amid the procedure of cache replacement. A maximum write-back threshold is configured according to the level of burst I/O requests in workload. This threshold is intended to avert redundant write I/Os flushing into flash memory, thereby boosting the endurance of flash memory. The experiments are conducted to validate the advantages of Co-Active in terms of average response time, write amplification, and erase count. The findings unveil that compared with the six popular cache management schemes (LRU, CFLRU, GCaR_CFLRU, LCR, and MQSim), Co-Active (1) slashes the average response time by up to 83.89 percent with an average of 32.7 percent; (2) drives up the performance cliff degree by up to 76.4 percent with an average of 42.3 percent; and (3) improves write amplification rate by up to 60.5 percent with an average of 5.4 percent.

Redundancy schemes are introduced to in-memory stores to provide fault tolerance. To achieve good trade-off between access performance and memory efficiency, it is appropriate to adopt replication and erasure coding to keep popular and unpopular data, respectively. Within such a hybrid-redundancy in-memory store, an issue of redundancy transition from replication to erasure coding (a.k.a., erasure-coded archival) should be addressed for unpopular in-memory datasets, since caching workloads exhibit long-tail distributions and most in-memory data are unpopular. If data replicas are distributed across nodes in randomly-selected racks, then subsequent data-block-replica retrieval for erasure-coded archival will create cross-rack traffic, and final parity-block relocation will cause extra cross-rack communications. In this article, we propose an encoding-oriented replica placement policy - ERP - by incorporating an interleaved declustering mechanism. We design two traffic-aware erasure-coded archival schemes -TEA-TL and TEA-SL - for ERP-powered in-memory stores by taking into account temporal locality and spatial locality, respectively. With ERP in place, both TEA-TL and TEA-SL schemes embrace the following three salient features: (i) they alleviate cross-rack traffic raised by retrieving required data-block replicas; (ii) they improve rack-level load balancing by distributing replicas via load-aware primary-rack-selection approach; and (iii) they mitigate block-relocation operations launched to sustain rack-level and node-level fault-tolerance. We conduct quantitative performance evaluations using the YCSB benchmark. The empirical results show that both TEA-TL and TEA-SL schemes not only bring forth lower cross-rack traffic than the four candidate encoding schemes, but also exhibit superb archival-throughput and rack-level-balancing performance. In particular, within a group of comparative tests using the baseline configurations, TEA-TL and TEA-SL accelerate archival throughput by at least 36.3 and 70.8 percent, respectively; both TEA-TL and TEA-SL schemes improve rack-level load-balancing by a factor of more than 1.45x relative to the four candidate encoding schemes.

With the growing volume of storage systems, the traditional relational databases cannot reach the high performance required by big-data applications. As high-throughput alternatives to relational databases, LSM-tree-based key-value stores (KV stores in short) are confronted with degraded write performance during compaction under update-intensive workloads. To address this issue, we design and implement a time-aware compaction optimization framework for KV stores called TStore. TStore explores the near-data processing (i.e., NDP) model. It dynamically partitions compaction tasks into both host and NDP-enabled device to minimize the total time of compaction. The partitioned compaction tasks are conducted by the host and the device in parallel. The NDP-based devices exhibit low-latency, high-performance and high-bandwidth capability, thus facilitating key-value stores. TStore can not only accomplish compaction for KV stores, but also improve overall performance by removing bottleneck in compaction. Results show that the TStore with an NDP framework can achieve 3.8x and 1.9x performance improvement over LevelDB and Co-KV under the db_bench workload. In addition, the TStore-enabled KV store outperforms LevelDB and Co-KV by a factor of 3.6x and 1.9x in throughput and 72.0% and 48.9% in latency, respectively, under realistic workloads generated by YCSB.

Existing RAID-6 codes are developed to optimize either reads or writes for storage systems. To improve both read and write operations, this paper proposes a novel RAID-6 MDS array code called N-Code. N-Code exhibits three aspects of salient features: (i) read performance. N-Code assigns both horizontal parity chains and horizontal parities across disks, without generating a dedicated parity disk. Such a parity layout not only makes all the disks service normal reads, but also allows continuous data elements to share the same horizontal chain to optimize degraded reads; (ii) write performance. Diagonal parities are distributed across disks in a decentralized manner to optimize partial stripe writes, and horizontal parity chains enable N-Code to reduce I/O costs of partial stripe writes by merging I/O operations; and (iii) balancing performance. Decentralized horizontal/diagonal parities potentially support the I/O balancing optimization for single writes.

To achieve good trade-off between access performance and memory efficiency, it is appropriate to adopt replication and erasure coding to keep popular and unpopular in-memory datasets, respectively. An issue of redundancy transition from replication to erasure coding (a.k.a., erasure-coded archival) should be addressed for unpopular in-memory datasets, since caching workloads exhibit long-tail distributions and most in-memory data are unpopular.

Data updates have become an important issue in erasure-coded in-memory stores owing to the two-fold reasons: (i) a handful of data-intensive in-memory stores adopt erasure coding for ‘hot’ data and (ii) small writes in update-intensive in-memory workloads cause expensive updating overheads. After delving into prior updating schemes in erasure-coded storage clusters, we investigate the applicability of these schemes to erasure-coded in-memory stores. We propose a grouped-updating mechanism—GU—to handle small writes in in-memory stores. With GU in place, requests in an updating window are categorized into several updating groups, where multiple small updates in an updating group can be concurrently executed. Two GU updating procedures—GU-stripe and GU-node—are developed to schedule updates according to a stripe and a node holding an updated data block, respectively. Furthermore, we develop two hybrid-updating schemes—Hybrid−U[GU-stripe] and Hybrid−U[GU-node]—to process common writes (i.e. small and large writes) initiated by the GU-stripe- and GU-node-based updating schemes, respectively. Replaying an update-heavy workload generated by YCSB benchmark, we extensively evaluate the four non-GU-based updating schemes, five GU-stripe-based updating schemes, and five GU-node-based updating schemes. Our experiments demonstrate that the GU mechanism boosts updating performance of small writes for RS-coded in-memory stores in terms of updating time and updating traffic. In particular, for a (8, 6) RS-coded in-memory store, the GU-stripe- and GU-node-based updating schemes shortens the updating time of the non-GU-based counterparts by a factor of at least 2.08 and 2.66, respectively. Compared to a single GU-based updating scheme, a GU-based hybrid updating scheme achieves an optimal updating-time and updating-traffic performance.

For the sake of cost-effectiveness, it is a conventional wisdom to employ (k + r,k) erasure codes to archive rarely accessed replicas, i.e. erasure-coded data archival. Existing researches on erasure-coded data archival optimizations are mainly aimed to reduce archival traffic within storage clusters. Apart from archival traffic, both non-sequential reads and imbalanced loads can deteriorate archival performance. Traditional distributed archival schemes (DArch for short) for randomly distributed replicas tend to suffer from two problems: (i) non-sequential reads because underlying file systems split a data block into multiple smaller data chunks and (ii) imbalanced loads since archival tasks are assigned according to data locality of replicas. To overcome such drawbacks, we incorporate both prefetching mechanism and balancing strategy into erasure-coded archival for replica-based storage clusters, and propose three new archival schemes: a prefetching-enabled archival scheme (i.e. P-DArch), a balancing-enabled archival scheme (i.e. B-DArch) and a prefetching-and-balancing-enabled archival scheme (i.e. PB-DArch). We implement a proof-of-concept prototype, where all the four archival schemes are deployed and quantitatively evaluated. The experimental results show that both the prefetching mechanism and balancing strategy can effectively optimize archival performance of a replica-based storage cluster exhibiting a random data layout. In a (12,9) RS-coded archival scenario, P-DArch, B-DArch and PB-DArch outperform DArch by a factor of 2.95, 1.72 and 3.85, respectively.

In-memory store has become a key component for an increasing number of data-intensive applications like OLTP and OLAP. To be resilient to data loss incurred by transient failures, redundancy strategies are incorporated into in-memory stores. In-memory datasets are characterized by skewed popularity, because they exhibit varied access frequencies (a.k.a., number of accesses). Therefore, it is prudent to apply customized redundancy schemes with dynamic memory efficiency and access parallelisms to different in-memory datasets. In this work, we propose an adaptive redundancy scheme-PaRS-for in-memory datasets. PaRS relies on a re-stripe or replication mechanism to transform involved redundancy groups according to their workload popularity growth. With PaRS in place, a memory-efficient redundancy layout is deployed for data blocks with low access frequencies; a redundancy layout exhibiting high access parallelism is adopted for highly-accessed data blocks. Compared with existing redundancy schemes that employ simple replication or erasure coding, PaRS facilitates a configurable tradeoff between memory efficiency and access parallelism for in-memory data blocks. Quantitative evaluations using YCSB show that PaRS enables in-memory stores to exhibit higher access performance and memory efficiency than the replication scheme. Furthermore, PaRS achieves better load balancing than the erasure coding, while sustaining superb access performance and memory efficiency. In particular, under a double-fault-tolerant in-memory store of limited memory, PaRS improves access latency by 15.1 to 31.5 percent compared to 3-way replication, and PaRS enhances load balancing by more than 3.9× relative to Reed-Solomon coding.

Due to limited numbers of program/erase cycles (i.e., P/Es) of NAND Flash, excessive out-of-place update and erase-before-write operations wear out these P/Es during garbage collections, which adversely shorten solid state disk (i.e., SSD) lifetime. The log space in NAND Flash space of an SSD performs as an updated page ′s buffer, which lowers garbage-collection frequency while reducing consumption of P/Es to extend SSD lifetime. In this article, we propose DLSpace, a novel distributed log space allocation strategy named d istributed l og space , which divides log space into block-level log space and page-level log space to significantly optimize SSD lifetime. DLSpace′s log page space is dedicated to data pages in a data block. Such log page space only buffers page-update operations in this data block; thereby the use of log blocks for postponing garbage collection delays. DLSpace is conducive to fully utilizing pages in data and log blocks to avoid erasures of blocks with free pages. Consequently, DLSpace decreases write amplification by reducing excessive valid page-rewrite and block-erase operations under random-write-intensive workloads. We carried out quantitative research on the extension of SSD lifetime by virtue of three metrics (i.e., write amplification, the number of block-erase operations, and the delay time before the first garbage collection occurring). Experimental results reveal that compared with the existing t raditional allocation strategy for l og space (i.e., TLSpace), DLSpace reduces write amplification and the number of erase operations by up to 55.2% and 64.1% to the most extent, respectively. DLSpace also extends TLSpace′s delay time of garbage collections by 73.3% to optimize SSD lifetime.

Data deduplication has been widely introduced to effectively reduce storage requirement of virtual machine (VM) images running on VM servers in the virtualized cloud platforms. Nevertheless, the existing state-of-the-art deduplication for VM images approaches can not sufficiently exploit the potential of underlying hardware with consideration of the interference of deduplication on the foreground VM services, which could affect the quality of VM services. In this paper, we present HPDV, a highly parallel deduplication cluster for VM images, which well utilizes the parallelism to achieve high throughput with minimum interference on the foreground VM services. The main idea behind HPDV is to exploit idle CPU resource of VM servers to parallelize the compute-intensive chunking and fingerprinting, and to parallelize the I/O-intensive fingerprint indexing in the deduplication servers by dividing the globally shared fingerprint index into multiple independent sub-indexes according to the operating systems of VM images. To ensure the quality of VM services, a resource-aware scheduler is proposed to dynamically adjust the number of parallel chunking and fingerprinting threads according to the CPU utilization of VM servers. Our evaluation results demonstrate that compared to a state-of-the-art deduplication system for VM images called Light, HPDV achieves up to 67% deduplication throughput improvement.

It is not uncommon for reconstruction I/Os to encounter workload fluctuation in heterogeneous RAID-coded storage systems. This paper proposes a heterogeneity-aware single-failure recovery scheme—SmartRec—to tolerate double and multiple disk failures in RAIDs. We start this study by formulating the data recovery problem of single-disk failures in form of an optimization function in the context of online and heterogeneous disk arrays. To take both static heterogeneity associated with disk configurations and dynamic heterogeneity affected by I/O loads into account, SmartRec periodically selects an appropriate reconstruction solution according to up-to-date disk utilization. The appropriate reconstruction solution indicates the amount of data being retrieved across surviving disks and is expected to achieve minimal recovery time, which is induced by both candidate reconstruction sequences and reconstruction I/O capability of surviving disks. We build a response-time model in SmartRec to measure the reconstruction I/O capability of surviving disks during a recovery process. To quantitatively compare the SmartRec scheme against three alternatives (i.e. ConRec, MinRec and BalRec), we build four analytical models and validate the correctness of the four models using empirical evaluations. We implement the four reconstruction schemes in a heterogeneous RAID, and carry out comparative online reconstruction tests by replaying real-world workloads under various configurations. The experimental results illustrate that our SmartRec scheme outperforms the three existing reconstruction schemes in terms of reconstruction time by up to 35.3% with an average of 25.8%.

Deep learning has exhibited high accuracy and applicability in machine learning field recently, by consuming tremendous computational resources processing massive data. To improve the performance of deep learning, GPUs have been introduced to accelerate the training phase. The complex data processing infrastructure demands high-efficient collaboration among underlying hardware components, such as CPU, GPU, memory, and storage devices. Unfortunately, few work has presented a systematic analysis about the impact of hardware configurations on the overall performance of deep learning. In this paper, we aim to make an experimental study on a standalone system to evaluate how various hardware configurations affect the overall performance of deep learning. We conducted a series of experiments using varied configurations on storage devices, main memory, CPU, and GPU to observe the overall performance quantitatively. Based on analyzing these results, we found that the performance greatly relies on the hardware configurations. Specifically, the computation is still the primary bottleneck as double GPUs and triple GPUs shorten the execution time by 44% and 59% respectively. Besides, both CPU frequency and storage subsystem can significantly affect running time while the memory size has no obvious effect on the running time for training neural network models. We believe our experimental results can help shed light on further optimizing the performance of deep learning in computer systems.

Data-intensive applications are increasingly depending on in-memory stores to meet high-I/O-performance requirements. To be resilient to server failures and in turn achieve high availability, both replication and erasure codes are introduced to in-memory stores. Since erasure codes have an advantage of memory efficiency over replication, we focus our work on erasure-coded in-memory stores and investigate placement schemes to address the issue of workload fluctuation. To mitigate the I/O imbalanced incurred by workload skew and maximize the utilization of all nodes, we proposed a Workload-aware Placement Scheme called WPS for Reed-Solomon-coded in-memory stores. WPS accomplishes balanced I/Os as follows: it divides in-memory data blocks into multiple groups based on access characteristics (e.g., popularity), and classifies all nodes into several groups according to nodes' access performance (e.g., indicated by available bandwidth), and places or migrates high-access-popularity in-memory data blocks to high-performance nodes without violating the essential principle of fault tolerance. The comparative experiments indicate that WPS can significantly improve load balancing for RS-coded in-memory stores exhibiting workload popularity skew; meanwhile, WPS achieves comparable mean, median, and tail latencies relative to two candidate placement schemes.

Erasure coding has been gradually adopted by existing data-intensive in-memory stores for 'hot' data; small writes lead to expensive updating overheads in such in-memory stores characterized by update-heavy workloads. There is a pressing demand to address the issue of data updates for erasure-coded in-memory stores. We revisit existing updating schemes in erasure-coded storage clusters by investigating the applicability of these updating schemes to erasure-coded in- memory stores. After an intensive analysis, we propose a grouping-update mechanism - GU - to handle small writes in in-memory stores. With GU in place, requests in an updating window are categorized into several updating groups, where multiple small updates in the same stripe can be executed concurrently. Furthermore, we bring forward a hybrid-updating scheme - Hybrid-U - to minimize total updating I/Os over network under common writes (e.g., small and large writes). We evaluate four dedicated updating schemes, four GU- based updating schemes and Hybrid-U. Our experiments illustrate that GU-based updating schemes and Hybrid-U outperform the four dedicated updating schemes in terms of updating time.

In this paper, we propose an erasure-coded data archival system called aHDFS for Hadoop clusters, where RS(k + r; k) codes are employed to archive data replicas in the Hadoop distributed file system or HDFS. We develop two archival strategies (i.e., aHDFS-Grouping and aHDFS-Pipeline) in aHDFSto speed up the data archival process. aHDFS-Groupinga MapReduce-based data archiving scheme - keeps each mapper's intermediate output Key-Value pairs in a local key-value store. With the local store in place, aHDFS-Grouping merges all the intermediate key-value pairs with the same key into one single key-value pair, followed by shuffling the single Key-Value pair to reducers to generate final parity blocks. aHDFS-Pipeline forms a data archival pipeline using multiple data node in a Hadoop cluster. aHDFS-Pipeline delivers the merged single key-value pair to a subsequent node's local key-value store. Last node in the pipeline is responsible for outputting parity blocks. We implement aHDFS in a real-world Hadoop cluster. The experimental results show that aHDFS-Grouping and aHDFS-Pipeline speed up Baseline's shuffle and reduce phases by a factor of 10 and 5, respectively. When block size is larger than 32 MB, aHDFS improves the performance of HDFS-RAID and HDFS-EC by approximately 31.8 and 15.7 percent, respectively.

Extracting high I/O performance from parallel file systems is no longer the only goal in modern data centres. As issues of the Energy Wall and the Reliability Wall become unavoidable, it is a demanding and challenging task to reduce energy consumption in large-scale storage systems in modern data centres while retaining acceptable systems reliability. Most energy conservation techniques inevitably have adverse impacts on the parallel disk systems. To address the reliability issues of energy-efficient parallel storage systems, we propose a reliable energy-efficient storage system called RESS, which aims at improving both energy efficiency and reliability of parallel storage systems by seamlessly integrating HDDs and SSDs. At the heart of the RESS is a transformative middleware layer, which reorganizes the I/O workload for the underlying parallel file systems. With the help of the middleware layer, RESS can distribute popular data to SSDs and put HDDs into the low-power mode under light workload conditions without modification of the parallel systems.

It is indispensable to speed up a reconstruction process in erasure-coded storage clusters, because a fast data recovery helps to shorten the vulnerability window while improving storage system reliability. To address double- and multiple-node failures, this paper proposes a cooperative reconstruction pattern - CoRec - to minimize reconstruction traffic. CoRec not only enables all rebuilding nodes to collaboratively reconstruct failed blocks but also limits each surviving block to be transferred over network only once. To clarify two CoRec based reconstruction schemes (i.e., CoRec-rn and CoRec-sn), we investigate two alternative reconstruction schemes (i.e., CRec and DRec). We develop reconstruction-time models, which are validated using empirical data, to estimate reconstruction performance of large-scale storage clusters and to pinpoint performance bottlenecks in the reconstruction process. We implement a proof-of-concept prototype where the four reconstruction schemes are quantitatively evaluated. Experimental results show that CoRec-rn and CoRec-sn significantly reduce the reconstruction time of CRec and DRec. In a real-world 9-node storage cluster, CoRec-rn speeds up the double-node reconstruction of CRec and DRec by a factor of at least 1.72, CoRec-sn accelerates the double-node reconstruction of CRec and DRec by a factor of at least 4.76.

Recent studies indicate that the energy cost and carbon footprint of data centers have become exorbitant. It is a demanding and challenging task to reduce energy consumption in large-scale storage systems in modern data centers. Most energy conservation techniques inevitably have adverse impacts on parallel disk systems. To address the reliability issues of energy-efficient parallel disks, we propose a reliable energy-efficient RAID system called REED, which aims at improving both energy efficiency and reliability of RAID systems by seamlessly integrating HDDs and SSDs. At the heart of REED is a high-performance cache mechanism powered by SSDs, which are serving popular data. Under light workload conditions, REED spins down HDDs into the low-power mode, thereby offering energy conservation. Importantly, during an I/O access turbulence (i.e., I/O load is dynamically and frequently changing), REED is conducive to reducing the number of disk power-state transitions by keeping HDDs in the low-power mode while serving requests with SSDs. We build a model to quantitatively show that REED is capable of improving the reliability of energy-efficient RAIDs. We implement the REED prototype in a real-world RAID-0 system. Our experimental results demonstrate that REED improves the energy-efficiency of conventional RAID-0 by up to 73% while maintaining good reliability.

RAID-6 is widely applied to tolerate double concurrent disk failures in both disk arrays and storage clusters. Among numerous erasure codes developed to implement RAID-6, Maximum Distance Separable (MDS) Codes are highly popular. Owing to the limitation of parity generating schemes used in MDS codes, RAID-6-based storage systems suffer from unbalance I/Os and low reconstruction performance. Out of consideration for high performance and reliability, we propose a new class of XOR-based RAID-6 code (i.e. |$V^{2}$|-Code), which improves both load balancing and reconstruction performance of the MDS RAID-6 codes. |$V^{2}$|-Code, a very simple yet flexible Non-MDS vertical code, can be easily implemented and deployed in storage systems. |$V^{2}$|-Code's unique features include lowest density code, steady parity chain length and well-balanced computation. We perform theoretical analysis and empirical evaluation of the coding scheme by running a wide range of workload under various configurations. Experimental results show that |$V^{2}$|-Code outperforms four popular codes (i.e. EVENODD, RDP, X-Code and Code-M) in terms of load balancing and reconstruction time. In the single-disk-failure and double-disk-failure cases, |$V^{2}$|-Code can speed up the reconstruction time of X-Code by a factor of up to 3.31 and 1.79, respectively.

Among redundant arrays of independent disks (RAID)-6 codes, maximum distance separable (MDS) based RAID-6 codes are popular because they have the optimal storage efficiency. Although vertical MDS codes exhibit better load balancing compared to horizontal MDS codes in partial stripes, an I/O unbalancing problem still exists in some vertical codes. To address this issue, we propose a novel efficient data layout, uniform P-code (UPC), to support highly balanced I/Os among P-coded disk arrays (i.e., PC). In UPC, the nonuniformly distributed information symbols in each parity chain of P-code are moved along their columns to other rows, thus enabling the parity chain to keep original parity relationships and tolerate double disk failures. The UPC scheme not only achieves optimal storage efficiency, computational complexity, and update complexity, but also supports better I/O balancing in the context of large-scale storage systems. We also conduct a performance study on reconstruction algorithms using an analytical model. Besides extensive theoretical analysis, comparative performance experiments are conducted by replaying real-world workloads under various configurations. Experimental results illustrate that our UPC scheme significantly outperforms the PC scheme in terms of average user response time. In particular, in the case of a 12-disk array, the UPC scheme can improve the access performance of the RAID-6 storage system by 29.9% compared to the PC scheme.

This paper addresses an I/O interference problem encountered in on-line reconstruction of erasure-coded storage clusters, where user I/Os compete with reconstruction I/Os for both disk and network bandwidth. We propose a redirection scheme called `RAM-RS' to minimize the I/O interference among user and reconstruction requests. RAM-RS redirects user read/writes targeted at failed nodes to an RS-coded RAM region, which is formed by pre-allocated main memory in surviving nodes in the RS-coding manner. The RS-coded RAM region quickly serves all user read/write misses; therefore, a rebuilding node can devote its disk and network bandwidths to the node reconstruction. The RAM region substantially reduces the amount of data rebuilt by the rebuilding node, because (1) missed writes are buffered in the RAM region and (2) missed reads are satisfied by using surviving nodes to co-rebuild failed blocks. We build two Markov models to estimate the reliability of the RAM-RS system. Modeling results demonstrate that the MTTDL of RS-coded RAM region in a storage cluster is larger than that of the same cluster comprised of surviving nodes. We implement both RAM-RS and the traditional Redirection schemes in an erasure-coded storage cluster, on which real-world I/O traces are replayed. Experimental results show that compared with the Redirection scheme running on a 9-node storage cluster, RAM-RS improves system performance in terms of both user response time and reconstruction time by a factor of 1.78 and 1.20, respectively.

A large write amplification ratio degrades the program/erase cycles (P/Es) of NAND Flashes and reduces the endurance and performance of solid state disks (SSDs). The lack of a practical way to measure write amplification for SSDs motivates us to propose a novel measuring method called RB-Explorer at the SSD level rather than the NAND Flash level. The goal of RB-Explorer is two-fold: (1) to accurately measure the write amplification of SSDs to quantify SSD endurance and (2) to study the impacts of I/O techniques on write amplification of SSDs. RB-Explorer incorporates a Ready/Busy (R/B) signal of one of the NAND Flashes in an SSD in a proposed write amplification model for SSDs with four full-parallelism levels (i.e., the channel, chip, die, and plane levels). RB-Explorer takes two steps toward measuring write amplification. First, RB-Explorer quantifies the number of page programs using the low R/B signal level, the duration of which varies with the different operation (i.e., read, program, and erase) in NAND Flash. Second, RB-Explorer measures data volume written to NAND Flashes by considering parallelisms at four levels. Data volume written to a die in a NAND Flash is obtained as a product of the number${\rm N_{p}}$of programs and page size${\rm P_{a}}$. Given the number${\rm N_{channel}}$of channels, the number${\rm N_{chip}}$of chips per channel, and the number${\rm N_{die}}$of dies per chip, one can obtain the data volume written to NAND Flashes as a product of${\rm N_{p}}, {\rm P_{a}}, {\rm N_{die}}, {\rm N_{chip}}$, and${\rm N_{channel}}$. RB-Explorer is applied to analyzing write amplification ratios of SSDs to track SSD endurance. Furthermore, we implement a real-world SSD (i.e., SSD-v) and employ a fine-tuned SSD simulator (i.e., SSDsim) to validate the accuracy of RB-Explorer. Our experimental results show that RB-Explorer improves on the accuracy of SSDsim—the state-of-the-art SSD simulator—in most tested cases. We conduct a series of measurements using micro-benchmarks and I/O traces to demonstrate how RB-Explorer may be applied to investigate SSDs.

A key design goal of erasure-coded storage clusters is to minimize reconstruction time, which in turn leads to high reliability by reducing vulnerability window size. PULL-Rep and PULL-Sur are two existing reconstruction schemes based on PULL-type transmission, where a rebuilding node initiates reconstruction by sending a set of read requests to surviving nodes to retrieve surviving blocks. To eliminate the transmission bottleneck of replacement nodes in PULL-Rep and mitigate the extra overhead caused by noncontiguous disk access in PULL-Sur, we incorporate PUSH-type transmissions to node reconstruction, where the reconstruction procedure is divided into multiple tasks accomplished by surviving nodes in a pipelining manner. We also propose two PUSH-based reconstruction schemes (i.e., PUSH-Rep and PUSH-Sur), which can not only exploit the I/O parallelism of PULL-Sur, but also maintain sequential I/O accesses inherited from PULL-Rep. We build four reconstruction-time models to study the reconstruction process and estimate the reconstruction time of the four schemes in large-scale storage clusters. We implement a proof-of-concept prototype where the four reconstruction schemes are deployed and quantitatively evaluated. Experimental results show that the PUSH-based reconstruction schemes outperform the PULL-based counterparts. In a real-world (9,6)RS-coded storage cluster, PUSH-Rep speeds up the reconstruction time by a factor of 5.76 compared with PULL-Rep; PUSH-Sur accelerates the reconstruction by a factor of 1.85 relative to PULL-Sur.

It is indispensable to scale erasure-coded storage clusters to meet requirements of increased storage capacity and I/O performance. In this study, we propose an efficient scaling scheme for Reed-Solomon-coded storage clusters called Scale-RS, which has three salient features. First, Scale-RS achieves uniform data distribution by equally placing data blocks among old and new chunks using a transposed data layout. Second, Scale-RS minimizes data movement incurred in the procedures of data redistribution and parity update. Scale-RS not only reaches the lower bound of data migration traffic by transferring necessary data blocks from old data chunks to new chunks, but it also reduces update traffic via generating parity difference blocks from data blocks stored in an individual data chunk. Third, Scale-RS improves the I/O performance of scaled storage clusters in terms of read parallelism and write throughput. We implement Scale-RS along with two alternative scaling schemes in a Reed-Solomon-coded storage cluster, on which real-world I/O traces are replayed. Experimental results demonstrate that Scale-RS achieves the highest read performance among the three scaling schemes after data redistribution. When it comes to scaling from six data chunks to nine, Scale-RS can outperform the other two scaling schemes in terms of aggregate write throughput by a factor of 2.85 and 3.05 under online filling and offline filling, respectively. We also show that user response time is slightly enlarged during data redistribution due to bandwidth competition between migration and user I/Os.

This paper addresses an issue of erasure-coded data archival, where (k + r; k) erasure codes are employed to archive rarely accessed replicas. The traditional synchronous encodingprocess neither leverages the existence of replicas, nor handles encoding operations in a decentralized manner. To overcome these drawbacks, we exploit pipelined encoding processes to boost the data archival performance on storage clusters. First, we propose two data layouts called [D + P]cdand [3X]cdby applying a chained-declustering mechanism to both Mirrored RAID-5 and triplication redundancy groups. Second, in light of the [D + P]cdand [3X]cdlayouts, we design two archiving schemes named DP and 3X, which exhibit the following three salient features: (i) exploiting data locality-two or three local blocks are read by each involved node for encoding; (ii) decentralized computation load-encoding operations are distributed among k nodes; and (iii) parallel archival processing-two or three encoding pipelines are simultaneously deployed to generate parity blocks. We implement both the DPand 3X schemes and three existing solutions (i.e., SynE, DE, and RapidRAID) in a real-world storage cluster. Experimental results show that our archival schemes outperform the other three solutions in terms of archiving time by a factor of at least 3.41 in a nine-node storage cluster. The experiments strongly indicate that the performance bottleneck of SynE lies in its block-receiving stage; it is disk I/O rather than network traffic that dominates archiving time for both the DE and RapidRAID schemes.

To reduce the probability of data unavailability, it is extremely important to quickly recover failed data in a (k+r, k) erasure-coded storage cluster. In practice, storage nodes in a large-scale storage system have various network bandwidths and I/O capabilities, therefore, the heterogeneity of storage systems increases along with the growing scale. Both traditional recovery scheme and Fastest recovery scheme simply retrieve k surviving blocks from k surviving nodes, thereby resulting in low recovery performance in a heterogeneous storage cluster. In this paper, we propose a Load-aware Recovery Scheme (Lars) for heterogeneous RS-coded storage clusters. Lars not only takes into account both the heterogeneity and load of nodes, but also enables all surviving nodes to service reconstruction reads. The amount of surviving blocks retrieved by a surviving node depends on its load weight which is determined by both network bandwidth and I/O capacity. More blocks are fetched from faster nodes, and vice versa. The three recovery schemes are implemented on a 9-node heterogeneous RS-coded storage cluster, where a set of comparative experiments are conducted. The experimental results show that our Lars scheme outperforms the other two schemes by a factor of up to 1.58.

There exist numerous erasure codes for RAID-6, of which MDS codes are popular due to the optimal storage efficiency. Although vertical MDS codes have better load balancing compared to horizontal MDS codes, unbalancing problem still exists in some vertical codes, e.g., P-Code. To address this issue, we propose a novel efficient RAID-6 code to support highly balanced I/Os among disk arrays - Balanced P-Code. In Balanced P-Code, We move the unevenly distributed information symbols in each parity chain of P-Code along their columns to other rows, thus enabling the parity chain to keep original parity calculation relationships and tolerate double disk failures. The Balanced P-Code can not only achieve optimal storage efficiency, computational complexity and update complexity, but also support better I/O balancing in the context of large scale storage systems. Apart from extensive theoretical analysis, empirical evaluation are conducted by running a wide range of workloads under various configurations. Experimental results show that Balanced P-Code has better load balancing ratio. Especially, in both random mixed single read/write and random mixed continuous read/write cases, Balanced P-Code outperforms P-Code in terms of load balancing ratio by a factor of up to 2.64 and 2.3, respectively.

In this paper, we propose a multi-level fault-tolerant storage cluster called MFTS, which provides flexible reliability for a wide variety of applications. MFTS makes use of a reliability upper-bound (i.e., Parameter r) to guide the process of adjusting fault-tolerance levels, i.e., i-erasure(s) and i {1, 2, .. ., r}. In particular, MFTS can map an appropriate coding scheme to an application with individual reliability requirements. MFTS is capable of partitioning multi-level reliable storage using a virtual storage space, thereby adapting to any changing reliability demands of applications. We present the implementation of the MFTS system, which adopts an intersecting zigzag sets code (IZS code) rather than replication or general-purpose erasure codes. Our MFTS has three salient features: partial updates, fast reconstructions, and minimal overhead of fault-tolerance level transitions. To quantify performance optimization in our storage cluster, we compare IZS-enabled MFTS with two storage clusters equipped with the Vandermondeand Cauchy-Reed-Solomon codes. The experimental results show that: 1) three schemes have comparable user-response-time performance in both the operational and degraded modes; 2) MFTS outperforms the other two alternatives by up to 26.1 percent in the offline reconstruction case; 3) MFTS speeds up the online reconstruction by up to 23.7 percent over the other two schemes with marginal increase in user response time.

RAID-6 is widely used to tolerate concurrent failures of any two disks in both disk arrays and storage clusters. Numerous erasure codes have been developed to implement RAID-6, of which MDS Codes are popular. Due to the limitation of parity generating schemes used in MDS codes, RAID-6-based storage systems suffer from low reconstruction performance. To address this issue, we propose a new class of XOR-based RAID-6 code (i.e., V2-Code), which delivers better reconstruction performance than the MDS RAID-6 code at low storage efficiency cost. V2-Code, a very simple yet flexible Non-MDS vertical code, can be easily implemented in storage systems. V2-Code's unique features include (1) lowest density, (2) steady length of parity chain, and (3) well balanced computation. We perform theoretical analysis and evaluation of the coding scheme under various configurations. The results show that V2-Code is a well-established RAID-6 code that outperforms both X-Code and Code-M in terms of reconstruction time. V2-Code can speed up the reconstruction time of X-Code by a factor of up to 3.31 and 1.79 under single disk failure and double disk failures, respectively.

We present a power-efficient scheme for erasure-coded storage clusters---ECS2---which aims to offer high energy efficiency with marginal reliability degradation. ECS2utilizes data redundancies and deferred writes to conserve energy. In ECS2parity blocks are buffered exclusively in active data nodes whereas parity nodes are placed into low-power mode. (k+r, k) RS-coded ECS2can achieve ⌈(r+ 1)/2⌉-fault tolerance forkactive data nodes andr-fault tolerance for allk+rnodes. ECS2employs the following three optimizing approaches to improve the energy efficiency of storage clusters. (1) An adaptive threshold policy takes system configurations and I/O workloads into account to maximize standby time periods; (2) a selective activation policy minimizes the number of power-transitions in storage nodes; and (3) a region-based buffer policy speeds up the synchronization process by migrating parity blocks in a batch method. After implementing an ECS2-based prototype in a Linux cluster, we evaluated its energy efficiency and performance using four different types of I/O workloads. The experimental results indicate that compared to energy-oblivious erasure-coded storage, ECS2can save the energy used by storage clusters up to 29.8% and 28.0% in read-intensive and write-dominated workloads when k = 6 and r = 3, respectively. The results also show that ECS2accomplishes high power efficiency in both normal and failed cases without noticeably affecting the I/O performance of storage clusters.

The buffer cache plays an essential role in smoothing the gap between the upper level computational components and the lower level storage devices. A good buffer cache management scheme should be beneficial to not only the computational components, but also the storage components by reducing disk I/Os. Existing cache replacement algorithms are well optimized for disks in normal mode, but inefficient under faulty scenarios, such as a parity-based disk array with faulty disk(s). To address this issue, we propose a novel penalty-aware buffer cache replacement strategy, named Victim Disk(s) First (VDF) cache, to improve the reliability and performance of a storage system consisting of a buffer cache and disk arrays. VDF cache gives higher priority to cache the blocks on the faulty disks when the disk array fails, thus reducing the I/Os addressed directly to the faulty disks. To verify the effectiveness of the VDF cache, we have integrated VDF into the popular cache algorithms least frequently used (LFU) and least recently used (LRU), named VDF-LFU and VDF-LRU, respectively. We have conducted intensive simulations as well as a prototype implementation for disk arrays to tolerate one disk failure (RAID-5) and two disk failures (RAID-6). The simulation results have shown that VDF-LFU can reduce disk I/Os to surviving disks by up to 42.3 percent in RAID-5 and 50.7 percent in RAID-6, and VDF-LRU can reduce those by up to 36.2 percent in RAID-5 and 48.9 percent in RAID-6. Our measurement results also show that VDF-LFU can speed up the online recovery by up to 46.3 percent in RAID-5 and 47.2 percent in RAID-6 under spare-rebuilding mode, or improve the maximum system service rate by up to 47.7 percent in RAID-5 under degraded mode without a reconstruction workload. Similarly, VDF-LRU can speed up the online recovery by up to 34.6 percent in RAID-5 and 38.2 percent in RAID-6, or improve the system service rate by up to 28.4 percent in RAID-5.

Nowadays, erasure codes have been widely used in data storage to achieve high fault-tolerance. However, compared with replica-based storage, erasure-coded system may suffer significant performance overhead in encoding, decoding and updating. Traditional updating schemes(e.g. DUM and PUM) use an individual manager node to accomplish the updating. In this paper, we propose two partial-updating schemes (i.e. PUM-P and PDN-P) to improve the small update in erasure coded storage clusters, where both schemes only read a portion of data, including the data blocks to be updated and the parity blocks, and utilize the calculation capacity of the storage nodes. We implement four updating algorithms (DUM, PUM, PUM-P and PDN-P) upon an erasure-coded storage cluster platform, and conduct a set of comparative tests under two real-world workloads with different fault-tolerance parameters. The experimental results shows that PUM-P and PDN-P can speed up the small updating by a factor of up to 1.62 and 2.72 compared with DUM under small update, respectively; and by a factor of up to 1.42 and 2.23 relative to PUM, respectively. We also validate that DUM scheme can achieve better updating performance than the other schemes within the large update scenario.

Sequential prefetching schemes are widely employed in storage servers to mask disk latency and improve system throughput. However, existing schemes cannot benefit parallel disk systems as expected due to the fact that they ignore the distinct internal characteristics of the parallel disk system, in particular, data striping. Moreover, their aggressive prefetching pattern suffers from premature evictions and prolonged request latencies. In this paper, we propose a strip-oriented asynchronous prefetching (SoAP) technique, which is dedicated to the parallel disk system. It settles the above-mentioned problems by providing multiple novel features, e.g., enhanced prediction accuracy, adaptive prefetching strength, physical data layout awareness, and timely prefetching. To validate SoAP, we implement a prototype by modifying the software redundant arrays of inexpensive disks (RAID) under Linux. Experimental results demonstrate that SoAP can consistently offer improved average response time and throughput to the parallel disk system under non-random workloads compared with STEP, SP, ASP, and Linux-like SEQPs.

As the technique developing, some important problems in storage systems have been solved appropriately. A good example is the development of RAID-6 code techinque, the appear of it has greatly improved the reliability, availability of modern storage systems. Some best known vertical RAID-6 code like P-code and X-code has acquire optimal or near optimal performance in encoding, decoding and update. But they do not detailedly analysis the status of reconstruction with single-disk failure. In the status, there are many paths to perform reconstructing. But the path you choice will greatly affect the performance of whole storage system. Based the phenomenon found above, we present a fast and effcient scheme, Path Directed Recovery Scheme (PDRS for short), to find a optimal path to reconstruct single-disk failure in P-code and X-code. Using PDRS, we will acquire some benefits: (1) it can decrease the disk I/O complexity caused by reconstruction and therefore accelerating the speed of reconstruction, (2) it can balance the load on each disk, consequently can avoid the hot problem in a degree. We perform theoretical analysis and evaluation of the PDRS when applied in P-code with (p-1)-disk and X-code with p-disk. Our theoretical analysis shows that PDRS applied in P-code with (p-1)-disk can acquire up to 25% performance improvement. To verify the effectiveness of PDRS, we have conducted intenvice simulation. The simulation results shows that PDRS applied in P-code with (p-1)-disk can speedup the recovery duration by up to 23.6% under spare-rebuilding mode. Overall, PDRS is a efficient and useful recovery scheme that can applied to all of the vertical RAID-6 code.

The improvement for energy efficiency has been increasingly becoming a major consideration in server and data center design, especially for the power-hungry ones. Numerous studies have provided various new methods or proposals for the building of "green" server and data center, but this paper concentrates on how different configuration schemes in a server effect practical performance and power consumption for specific applications. It is completely necessary to obtain thin provisioning for the particular applications to meet performance requirements with minimal energy consumption. This paper evaluates the different hardware configurations' impact on energy consumption and performance for typical applications, hoping for offering evidences or clues to subsequent researches. The File Bench is used to generate four sever workloads and ZH-101 is employed to collect relevant real-time power consumptions. Our result shows that different workloads need different hardware configurations at the demands of both energy-efficiency and performance. And running multiple workloads on a reduced hardware configuration is a wise choice.

Offering better performance for random access compared to conventional hard disks and providing larger capacity and lower cost than DRAM, NAND flash based SSDsare integrated in server storage hierarchy as a second tier of disk cache between DRAM and disks for caching more data from disks to meet the increasingly intensive I/O demands. Unfortunately, available hybrid storage architectures cannot fully exploit SSDs' potentials due to absorbing too much workload of disk tier, which results in excessive wear and performance degradation associated with internel garbage collection. In this paper, we propose RAF (Random Access First), an hybrid storage architecture that combines both of an SSD based disk cache and a disk drive subsystem. RAF focuses on extending the lifetime of SSD while improving system performance through providing priority to caching random-access data. In detail, RAF splits flash cache into read and write cache to service read/write requests respectively. Read cache only holds random-access data that are evicted from file cache to reduce flash wear and write hits. Write cache performs as a circular write-through log so as to improve system response time and simplify garbage collection. Similar to read cache, write cache only caches random-access data and flushes them to hard disks immediately. Note that, sequential access are serviced by hard disks directly to even the full workload between SSD and disk storage. RAF is implemented in Linux kernel 2.6.30.10. The results of experiments show that RAF can significantly reduce flash wear and improve performance compared with the state-of-art FlashCache architecture.