Erasure-coded storage systems can achieve highly reliable data storage with low storage overhead. However, updating data blocks necessitates updating parity blocks, and updating multiple data blocks incurs heavy I/O and computational overhead, leading to prolonged update times in heterogeneous storage environments. This paper proposes an erasure-coded multi-data block update method for heterogeneous storage environments, named MBUS, to expedite data updates. MBUS blends re-encoding and delta-based updates and considers factors such as intermediate blocks, computational overhead, and I/O overhead across different storage media. It iteratively computes the minimal I/O time for updating each parity block dynamically. Once the computation equation for each parity block is determined, MBUS prioritizes computing common XOR expressions and reuses the results to reduce the number of XOR operations. MBUS caches a certain number of scheduling plans to minimize redundant computations in subsequent updates, thereby reducing computation time. Experimental results replaying real-world traces demonstrate that MBUS reduces update time by 64.43% compared to state-of-the-art techniques.

Distributed storage systems typically use erasure codes for fault tolerance to reduce storage overhead. However, the data repair process in erasure-coded systems can generate heavy I/O overhead. Existing methods typically increase redundancy to improve repair speed, but this approach results in substantial storage overhead. To enhance repair speed while reducing storage costs, this paper proposes a Machine Learning-based Adaptive Recovery (MLAR) method. Given an application’s access patterns for a file, MLAR uses an adaptive encoding model to calculate the optimal code for each file. When applying fault tolerance with the optimal code, lower-redundancy codes can achieve faster repair speeds than higher-redundancy codes. MLAR employs machine learning to predict file access patterns. Experimental results replaying real-world I/O workloads show that, MLAR reduces storage overhead by 12.8% and recovery time by 23.7%, compared to state-of-the-art methods.

In the wide-area high-performance computing environment, heterogeneous storage resources are geographically distributed in different supercomputing centers, which leads to the barriers between applications and data. This paper proposes a global virtual data space, named GVDS, to meet the needs of unified data access across supercomputing centers. GVDS integrates the parallel/distributed file systems of supercomputing centers to present a virtual space with tremendous storage capability for users. GVDS organizes users into groups for easy management, which allows users to share, collaborate, and perform computations on the stored data. For failure tolerance, global metadata is replicated and distributed on multiple supercomputing centers, redundant I/O service components are deployed in each supercomputing center. GVDS uses adaptive prefetching, caching, and request merging to improve access performance. Experimental results running on real-world supercomputing centers show that, GVDS can deliver excellent I/O performance running micro-benchmark, real-world traces and applications.

Erasure code is widely used in storage systems since it can offer higher reliability at lower redundancy than data replication. However, erasure coding based storage systems have to perform multi-block updates for partial writes of an erasure coding group, which leads to a large number of XOR operations. This paper presents an efficient approach, named ECMU, for erasure-coded multi-block update under a stringent latency by scheduling update sequences. ECMU takes a hybrid of reconstructed-write and read-modify-write for parity blocks of an erasure coding group, it dynamically selects the write scheme with the fewer XORs for each parity block to be updated, in order to reduce the number of XORs. ECMU iteratively retrieves the unmodified parity blocks to calculate the minimum XORs for each write scheme. For all parity blocks to be updated, after the write schemes are determined, ECMU performs the common XORs first, then it reuses the computational results to further reduce the number of XORs. ECMU caches a certain number of scheduling schemes to reduce the construction count of the scheduling schemes. Experimental results on real-world trace replaying show that the number of XORs and update time can be reduced significantly, compared with the state-of-the-art.