This paper addresses an I/O interference problem encountered in on-line reconstruction of erasure-coded storage clusters, where user I/Os compete with reconstruction I/Os for both disk and network bandwidth. We propose a redirection scheme called `RAM-RS' to minimize the I/O interference among user and reconstruction requests. RAM-RS redirects user read/writes targeted at failed nodes to an RS-coded RAM region, which is formed by pre-allocated main memory in surviving nodes in the RS-coding manner. The RS-coded RAM region quickly serves all user read/write misses; therefore, a rebuilding node can devote its disk and network bandwidths to the node reconstruction. The RAM region substantially reduces the amount of data rebuilt by the rebuilding node, because (1) missed writes are buffered in the RAM region and (2) missed reads are satisfied by using surviving nodes to co-rebuild failed blocks. We build two Markov models to estimate the reliability of the RAM-RS system. Modeling results demonstrate that the MTTDL of RS-coded RAM region in a storage cluster is larger than that of the same cluster comprised of surviving nodes. We implement both RAM-RS and the traditional Redirection schemes in an erasure-coded storage cluster, on which real-world I/O traces are replayed. Experimental results show that compared with the Redirection scheme running on a 9-node storage cluster, RAM-RS improves system performance in terms of both user response time and reconstruction time by a factor of 1.78 and 1.20, respectively.

A key design goal of erasure-coded storage clusters is to minimize reconstruction time, which in turn leads to high reliability by reducing vulnerability window size. PULL-Rep and PULL-Sur are two existing reconstruction schemes based on PULL-type transmission, where a rebuilding node initiates reconstruction by sending a set of read requests to surviving nodes to retrieve surviving blocks. To eliminate the transmission bottleneck of replacement nodes in PULL-Rep and mitigate the extra overhead caused by noncontiguous disk access in PULL-Sur, we incorporate PUSH-type transmissions to node reconstruction, where the reconstruction procedure is divided into multiple tasks accomplished by surviving nodes in a pipelining manner. We also propose two PUSH-based reconstruction schemes (i.e., PUSH-Rep and PUSH-Sur), which can not only exploit the I/O parallelism of PULL-Sur, but also maintain sequential I/O accesses inherited from PULL-Rep. We build four reconstruction-time models to study the reconstruction process and estimate the reconstruction time of the four schemes in large-scale storage clusters. We implement a proof-of-concept prototype where the four reconstruction schemes are deployed and quantitatively evaluated. Experimental results show that the PUSH-based reconstruction schemes outperform the PULL-based counterparts. In a real-world (9,6)RS-coded storage cluster, PUSH-Rep speeds up the reconstruction time by a factor of 5.76 compared with PULL-Rep; PUSH-Sur accelerates the reconstruction by a factor of 1.85 relative to PULL-Sur.

It is indispensable to scale erasure-coded storage clusters to meet requirements of increased storage capacity and I/O performance. In this study, we propose an efficient scaling scheme for Reed-Solomon-coded storage clusters called Scale-RS, which has three salient features. First, Scale-RS achieves uniform data distribution by equally placing data blocks among old and new chunks using a transposed data layout. Second, Scale-RS minimizes data movement incurred in the procedures of data redistribution and parity update. Scale-RS not only reaches the lower bound of data migration traffic by transferring necessary data blocks from old data chunks to new chunks, but it also reduces update traffic via generating parity difference blocks from data blocks stored in an individual data chunk. Third, Scale-RS improves the I/O performance of scaled storage clusters in terms of read parallelism and write throughput. We implement Scale-RS along with two alternative scaling schemes in a Reed-Solomon-coded storage cluster, on which real-world I/O traces are replayed. Experimental results demonstrate that Scale-RS achieves the highest read performance among the three scaling schemes after data redistribution. When it comes to scaling from six data chunks to nine, Scale-RS can outperform the other two scaling schemes in terms of aggregate write throughput by a factor of 2.85 and 3.05 under online filling and offline filling, respectively. We also show that user response time is slightly enlarged during data redistribution due to bandwidth competition between migration and user I/Os.

This paper addresses an issue of erasure-coded data archival, where (k + r; k) erasure codes are employed to archive rarely accessed replicas. The traditional synchronous encodingprocess neither leverages the existence of replicas, nor handles encoding operations in a decentralized manner. To overcome these drawbacks, we exploit pipelined encoding processes to boost the data archival performance on storage clusters. First, we propose two data layouts called [D + P]cdand [3X]cdby applying a chained-declustering mechanism to both Mirrored RAID-5 and triplication redundancy groups. Second, in light of the [D + P]cdand [3X]cdlayouts, we design two archiving schemes named DP and 3X, which exhibit the following three salient features: (i) exploiting data locality-two or three local blocks are read by each involved node for encoding; (ii) decentralized computation load-encoding operations are distributed among k nodes; and (iii) parallel archival processing-two or three encoding pipelines are simultaneously deployed to generate parity blocks. We implement both the DPand 3X schemes and three existing solutions (i.e., SynE, DE, and RapidRAID) in a real-world storage cluster. Experimental results show that our archival schemes outperform the other three solutions in terms of archiving time by a factor of at least 3.41 in a nine-node storage cluster. The experiments strongly indicate that the performance bottleneck of SynE lies in its block-receiving stage; it is disk I/O rather than network traffic that dominates archiving time for both the DE and RapidRAID schemes.