The management of mapping tables within solid-state drives (SSDs) is critical to maintaining high performance and stability. Traditional fixed-granularity mapping approaches (page-level, block-level) face significant challenges in ultra-large-scale 3D NAND flash memory, including high DRAM cache overhead resulting from page mapping and high write amplification caused by block mapping. Existing hybrid mapping solutions fail to adapt to the dynamic-varying workloads and large-capacity SSDs, resulting in unstable performance and poor scalability. In this paper, we propose a novel light-weight learning-based combination mapping technique called CoMap, which can adaptively choose the optimal mapping scheme so as to effectively prevent excessive page-mapping table sizes and poor block mapping performance in large-capacity SSDs. Comprehensive evaluations with variety of real-world workloads and enterprise-class SSD simulator show that CoMap, with its strong adaptability, can reduce 60% of latency and 30% of write amplification on average compared with several state-of-the-art methods.

Abstract Social network alignment (SNA) aims to match corresponding users across different platforms, playing a critical role in cross-platform behavior analysis, personalized recommendations, security, and privacy protection. Traditional methods based on attribute and structural features face significant challenges due to the sparsity, heterogeneity, and dynamic nature of social networks, resulting in limited accuracy and efficiency. Recent advances in graph representation learning (GRL) provide promising solutions to these issues by leveraging deep learning to extract network features, effectively addressing sparsity, integrating heterogeneous data, and adapting to network dynamics. This paper presents a comprehensive survey of SNA methods based on GRL. We first introduce key definitions and outline a framework for SNA using GRL. Next, we systematically review state-of-the-art advancements in both static and dynamic networks, considering homogeneous and heterogeneous settings, including emerging approaches integrating large language models (LLMs). We further conduct an in-depth comparative analysis, highlighting the effectiveness of different GRL-based methods, with a particular emphasis on LLM-enhanced techniques. Finally, we discuss open challenges and outline potential future research directions in this rapidly evolving field.

To ensure reliability, modern deduplication-based storage systems exploit erasure coding to redundantly distribute objects that store post-deduplicated data across multiple storage nodes. In the case of node failures, the failure recovery of impacted objects performs degraded read operations by reading from these storage nodes. However, this degraded read scheme introduces the additional available objects into the restore cache, which poses challenges to state-of-the-art caching policies applied in erasure-coded deduplicated storage systems. On one hand, chunk-based caching method fails to cache the remaining available objects after the unavailable objects are rebuilt, resulting in multiple reads of accessed objects and severely degrading restore performance. On the other hand, object-based caching methods do not identify lost objects to which duplicate chunks belong, causing unused objects to unnecessarily occupy cache space and destroy cache locality. In this paper, we propose TC-Cache, a restore cache scheme for node failures, which combines object-based caching and chunk-based caching methods to improve restore performance. The main idea of TC-Cache is to identify objects to which redundant data belongs, and selectively handle degraded reads of two types of objects in caches of different granularities during failure recovery to enhance cache utilization and maintain cache locality. Extensive experiments based on three real-world datasets demonstrate that TC-Cache outperforms state-of-the-art cache algorithms in terms of restore performance by 21%-65% with low computing overhead.

Modern deduplication-based storage systems employ erasure coding to post-deduplicate data to ensure reliability. In the erasure-coded deduplicated storage systems, existing coding schemes mainly perform erasure coding over a single object (intra-object coding) to address the problems of degraded read performance and storage inefficiency. However, due to intra-object coding, traditional storage scaling schemes need to reorganize and relocate all stored data between the client and storage nodes to support storage scaling, which inevitably incurs substantial data transfer overhead, thereby degrading system scalability. In this paper, we propose a fast scaling scheme called QuickScale for erasure-coded deduplicated storage systems. QuickScale first implements the intra-object coding in a realistic distributed storage environment and verifies that this coding scheme can significantly alleviate the degraded read performance by 35%-67% and save storage space by 24% 30%. Motivated by these results, QuickScale uses a node-to-node scaling approach that avoids repeatedly transferring all stored data between the client and storage nodes during scaling, thereby improving the scalability. Experimental evaluations using six real-world datasets demonstrate that QuickScale achieves better scaling performance (in terms of scaling time during scaling) over the state-of-the-art scaling schemes by up to 63.5%-76.4%.

The growing volume of performance-critical parameters in distributed storage systems, coupled with diverse and dynamic workload patterns, has significantly increased the complexity of system configuration. These trends have expanded the parameter space while tightening the time window for tuning convergence, making it challenging to maintain high system performance. Existing tuning strategies often struggle to balance thorough parameter exploration with real-time responsiveness, limiting their effectiveness under fast-evolving workloads and heterogeneous deployment environments. To address these challenges, we propose KGQW, the first framework that formulates automated parameter tuning as a knowledge graph query workflow. KGQW models workload features and system parameters as graph vertices, with performance metrics represented as edges, and constructs an initial knowledge graph through lightweight performance tests. Guided by performance prediction and Bayesian-driven exploration, KGQW progressively expands the graph, prunes insensitive parameters, and refines performance relationships to build an informative and reusable knowledge graph that supports rapid configuration retrieval via graph querying. Moreover, KGQW enables efficient knowledge transfer across clusters, substantially reducing the construction cost for new clusters. Experiments on real-world applications and storage clusters demonstrate that KGQW achieves second-level tuning latency, while maintaining or surpassing the performance of state-of-the-art methods. These results highlight the promise of knowledge-driven tuning in meeting the scalability and adaptability demands of modern distributed storage systems.

For a long time, fully exploiting the multi-level parallelism of SSDs has been an important technique for optimizing SSD performance. By distributing data to different flash parallel units through address mapping policies, SSDs can achieve multi-task parallel processing, thereby significantly improving data read and write performance. However, our experiments show that when SSD blocking causes task queues to stack up, excessive pursuit of flash parallelism can cause performance interference between multiple users, thereby reducing SSD performance. Additionally, in DRAM-less SSDs, differences in data access patterns directly impact the address mapping module, thereby exacerbating the degree of performance degradation. Based on the above analysis, this paper conducts a systematic study of the impact mechanism of different data flows on address mapping policies and proposes ahybrid addressmapping allocationpolicy with dynamic switching (HMpp) for DRAM-less SSDs. This policy identifies and classifies different data access patterns to dynamically select the appropriate address mapping policy, thereby reducing multi-user interference and improving SSD performance. Experimental results based on real cloud block storage workloads show that this policy can achieve 87.2% performance improvement and 77.3% fairness optimization, while reducing the write amplification factor by 32.9% to 40.2%.

Graph embedding, which maps graph nodes to lowdimensional vectors, is a widely used technique for graph representation learning. However, most existing graph embedding models suffer from high memory consumption, limiting their scalability to large graphs. Heterogeneous memory systems that combine DRAM and Persistent Memory (PM) offer new opportunities for scaling up memory capacity. Despite this advantage, the performance gap (on the order of 5x) between DRAM and PM is magnified (by 3.3-4.2x) under non-uniform memory access (NUMA) architecture. Additionally, the inherent sparsity of graphs induces numerous random accesses in the fundamental Sparse Matrix and Dense Matrix Multiplication (SpMM) operations of graph embedding, hindering high-performance heterogeneous memory processing. To address these challenges, this paper presents OMeGa that focuses on Optimizing heterogeneous Memory processing for large-scale Graph embedding. OMeGa leverages an entropy-aware thread allocation, simultaneously achieving workload balancing and tail latency reduction across threads. It also incorporates a workload feature-aware prefetcher to alleviate random accesses during streaming heterogeneous processing. In addition, OMeGa devises a NUMA-aware data placement, aiming to minimize the adverse impact of NUMA on heterogeneous memory. The experiments conducted on billion-scale graphs demonstrate that OMeGa exhibits an average acceleration of 32.03x with strong scalability. This pioneering capability enables the efficient generation of large-scale graph embeddings, free from the memory size constraints and performance disparities typically encountered in heterogeneous memory systems.

The learned index technique has been widely explored as a strong competitor to traditional indexes. It adopts static learning-based models to fit the distribution of sorted data and locate keys through predictions, which shows outstanding query speed. However, frequent retraining is required when it comes to concurrent insertion scenarios. Despite existing studies introducing sparse slots and delta buffers to mitigate this effect, the read-write performance of the learned index still falls short of expectations, especially in concurrent conditions. In this paper, we first propose a novel hybrid index scheme that combines a read-efficient learned index with an insert-efficient Adaptive Radix Tree (ART) to realize high performance for read-write scenarios. However, it is not trivial due to expensive model prediction errors, complicated model hierarchy, and redundant node traversals. Therefore, we then introduce ALT-index, an efficient hybrid learned index with high concurrency for memory database systems. ALT-index highlights a delicate two-tier architecture where linear data are stored in the learned index without prediction errors and conflict data are hosted in the lower layer as an optimized ART. Besides, we develop a Greedy Pessimistic Linear (GPL) algorithm to support flattened data structures for concurrency. In the optimized ART layer, we introduce a fast and compact pointer buffer to further improve the overall performance. Experimental results conducted on various real-world datasets with 32 threads illustrate that ALT-index improves performance by up to 1.9x, 2.1x, and 2.3x compared with ALEX+, FINEdex, and XIndex in read-write-balanced scenarios, respectively.

Deep neural network (DNN) training is both data-intensive and compute-intensive. As datasets grow, storing them entirely in memory becomes infeasible, making I/O a major bottleneck—often accounting for 30%-90% of total training time due to the widening gap between data loading and computation speed. While caching can mitigate I/O latency, traditional strategies fail under random sampling. Recent work shows that importance sampling can induce data locality, enabling more effective caching; however, loss-based importance ignores semantic attributes, limiting effectiveness in I/O-bound tasks.

SSD-based graph processing systems have emerged as a cost-effective solution for handling the ever-growing, large-scale graphs that exceed the memory capacity of a single machine. However, the mismatch between the large SSD access granularity (e.g., 4KB) and the small size of the graph vertex data leads to significant read amplification and low I/O efficiency. Despite existing works proposing techniques like dynamic active data gathering or reordering-based graph preprocessing to tackle this challenge, they inevitably cause problems such as expensive on-line computation overheads, inefficient graph traversal, and I/O imbalance, thus degrading the performance of graph processing.

Key-Value Stores (KVS) based on log-structured merge-trees (LSM-trees) are widely used in storage systems but face significant challenges, such as high write amplification caused by compaction. KV-separated LSM-trees address write amplification but introduce significant space amplification, a critical concern in cost-sensitive scenarios. Garbage collection (GC) can reduce space amplification, but existing strategies are often inefficient and fail to account for workload characteristics. Moreover, current key-value (KV) separated LSM-trees overlook the space amplification caused by the index LSM-tree. In this paper, we systematically analyze the sources of space amplification in KV-separated LSM-trees and propose Scavenger+, which achieves a better performance-space tradeoff. Scavenger+ introduces (1) an I/O-efficient garbage collection scheme to reduce I/O overhead, (2) a space-aware compaction strategy based on compensated size to mitigate index-induced space amplification, and (3) a dynamic GC scheduler that adapts to system load to make better use of CPU and storage resources. Extensive experiments demonstrate that Scavenger+ significantly improves write performance and reduces space amplification compared to state-of-the-art KV-separated LSM-trees, including BlobDB, Titan, and TerarkDB.

Graph embedding maps graph nodes to low-dimensional vectors and is widely used in machine learning tasks. The increasing availability of billion-edge graphs underscores the importance of learning efficient and effective embeddings on large graphs, such as link prediction on Twitter with over one billion edges. Most existing graph embedding methods fall short of reaching high data scalability. In this paper, we present a general-purpose, distributed, information-centric random walk-based, and pipeline-optimized graph embedding framework,$\sf{DistGER-Pipe}$DistGER−Pipe, which scales to embed billion-edge graphs.$\sf{DistGER-Pipe}$DistGER−Pipeincrementally computes information-centric random walks to reduce redundant computations for more effective and efficient graph embedding. It further leverages a multi-proximity-aware, streaming, parallel graph partitioning strategy, simultaneously achieving high local partition quality and excellent workload balancing across machines.$\sf{DistGER-Pipe}$DistGER−Pipealso improves the distributed$\sf{Skip-Gram}$Skip−Gramlearning model to generate node embeddings by optimizing access locality, CPU throughput, and synchronization efficiency. Finally,$\sf{DistGER-Pipe}$DistGER−Pipedesigns pipelined execution that decouples the operators in sampling and training procedures with an inter-round serial and intra-round parallel processing, attaining optimal utilization of computing resources. Experiments on real-world graphs demonstrate that compared to state-of-the-art distributed graph embedding frameworks, including$\sf{KnightKing}$KnightKing,$\sf{DistDGL}$DistDGL,$\sf{Pytorch-BigGraph}$Pytorch−BigGraph, and$\sf{DistGER}$DistGER,$\sf{DistGER-Pipe}$DistGER−Pipeexhibits 3.15×–1053× acceleration, 45% reduction in cross-machines communication, >10% effectiveness improvement in downstream tasks, and 38% enhancement in CPU utilization.

Serverless computing relies on keeping functions alive or pre-warming them before invocation to mitigate the cold start problem, stemming from the overhead of initializing function startup environments. However, under constrained cloud resources, accurately predicting the invocation patterns of sparse functions remains challenging. This limits the formulation of effective pre-warm and keep-alive strategies, leading to frequent cold starts and degraded user satisfaction. To address these challenges, we proposeSPFaaS, a hybrid framework based on sparse function prediction. To enhance the learnability of sparse function invocation data,SPFaaStakes into account the characteristics of cloud service workloads along with the features of pre-warm and keep-alive strategies, transforming function invocation records into probabilistic data. It captures the underlying periodicity and temporal dependencies in the data through multiple rounds of sampling and the combined use of Gated Recurrent Units and Temporal Convolutional Networks for accurate prediction. Based on the final prediction outcome and real-time system states,SPFaaSdetermines adaptive pre-warm and keep-alive strategies for each function. Experiments conducted on two real-world serverless clusters demonstrate thatSPFaaSoutperforms state-of-the-art methods in reducing cold starts and improving user satisfaction.

In modern datacenters, memory disaggregation un-packs monolithic servers to build independent network-connected compute and memory pools, greatly improving resource uti-lization. Existing memory-disaggregated key-value stores adopt ownership sharing or ownership partitioning for request scheduling and they all show poor performance and scalability. The former generates multiple cache misses and multiple network round trips due to poor cache locality, while the latter fails to fully use resources on all Compute Nodes (CNs) due to load imbalance. In this paper, we propose Seastar, a fast and scalable key-value store that achieves high cache locality as well as load balancing. To enable high concurrency and ensure linearizability, Seastar exploits a self-verification version chain to provide out-of-place atomic updates and ensure consistent reads. To improve cache locality, Seastar leverages a pair-based ownership assigning scheme to assign the processing permission of requests corre-sponding to each key to a CN pair through two independent hash functions. To ensure load balancing among CNs, Seastar proposes a decentralized power-of-two-choices routing scheme to route the requests to the less-loaded CNs in CN pairs for handling. Experimental results demonstrate that Seastar improves the throughput by up to 6.7x and significantly reduces the latency compared with state-of-the-art memory-disaggregated key-value stores.

Modern SSDs achieve low latency and high throughput by utilizing multiple levels of SSD parallelism. Fairness is also a critical design consideration in cloud environments and has inspired great interest in recent years to study it. However, we find that excessive striving for SSD parallelism can inadvertently harm the SSD fairness due to the striped data layout. We observe from our experimental analysis that some common access patterns in cloud loads (e.g., high-intensity I/O) are not addressed by SSD parallelism, which can affect both SSD parallelism and fairness. In this paper, we analyze two address mapping policies with orthogonal characteristics from SSD parallelism and fairness perspectives. Based on extensive experiments and observations, we schedule a hybrid address mapping policy to achieve fairness of SSDs in cloud environments by distinguishing different I/O access modes and cold/hot data, while ensuring SSD performance. Experimental results on the latest block-level I/O traces show that our approach significantly improves performance and fairness by up to 77.3% and 62.8%, respectively, over the previous schemes with negligible cost. Meanwhile, it can also reduce the write amplification by 23.5% to 33.5%.

Hash indexes are widely used in key-value storage systems due to their ability to perform rapid single-point queries. The persistent memory (PM) technology has received significant attention in both academia and industry due to its high performance, non-volatility, and large capacity characteristics. Currently, hash indexes tailored for persistent memories have been extensively researched. However, through an in-depth experimental study, we have discovered that existing persistent hash indexes suffer from low query performance. This is primarily due to persistent memory's higher read latency than DRAM's, which reduces the performance of both positive and negative queries in persistent hash indexes. Additionally, the former's higher read lock overhead further diminishes query performance. To address the above problems, we propose in this paper a Read-Optimized Persistent Hash Index, referred to as ROPHI, based on fingerprint filtering and lock-free prefetching. By employing a fingerprint filtering method, ROPHI introduces a DRAM-based Cuckoo filter to store fingerprints of keys on top of the PM-based hash table, effectively mitigating the time-consuming access overhead of persistent memory hash tables by accessing only the DRAM-based filter. Additionally, ROPHI employs lock-free prefetching for positive query acceleration, utilizing lock-free optimistic concurrent read techniques to avoid read lock overhead and high-speed cache prefetching techniques to reduce access overhead to persistent memory. Experimental results on the Intel Optane DC Persistent Memory Module (DCPMM) platform demonstrate that ROPHI significantly improves query performance over existing persistent hash index schemes. Specifically, ROPHI achieves an improvement of 2.67×-13.59× in negative query performance and 1.72x-7.86x in positive query performance. ROPHI outperforms the state-of-the-art SmartHT in positive query throughput by 34.5%, and in insertion and deletion throughput by 9.20% and 19.87% respectively, while sacrificing only 1.93% of negative query throughput. Additionally, it achieves a 5.07x improvement in recovery efficiency.

Key- Value Stores (KVS) implemented with log- structured merge-tree (LSM-tree) have gained widespread ac-ceptance in storage systems. Nonetheless, a significant challenge arises in the form of high write amplification due to the compaction process. While KV-separated LSM-trees successfully tackle this issue, they also bring about substantial space am-plification problems, a concern that cannot be overlooked in cost-sensitive scenarios. Garbage collection (GC) holds significant promise for space amplification reduction, yet existing GC strategies often fall short in optimization performance, lacking thorough consideration of workload characteristics. Additionally, current KV-separated LSM-trees also ignore the adverse effect of the space amplification in the index LSM-tree. In this paper, we systematically analyze the sources of space amplification of KV- separated LSM-trees and introduce Scavenger, which achieves a better trade-off between performance and space amplification. Scavenger initially proposes an I/O-efficient garbage collection scheme to reduce I/O overhead and incorporates a space-aware compaction strategy based on compensated size to minimize the space amplification of index LSM-trees. Extensive experiments show that Scavenger significantly improves write performance and achieves lower space amplification than other KV-separated LSM-trees (including BlobDB, Titan, and TerarkDB).

As a complementary technique to data deduplication, delta compression calculates and stores differences between highly similar but non-duplicate chunks to reduce storage costs in storage systems. A key aspect of efficient post-deduplication delta compression is resemblance detection, which identifies similar reference chunks to achieve significant data reduction. However, our evaluation of existing methods indicates that the throughput of resemblance detection is the system bottleneck due to the computationally intensive nature of feature extraction, which is carried out sequentially. Additionally, these methods suffer from low similarity between detected chunks and miss some potential similar chunks, reducing the redundancy elimination effect. This paper introduces ERD, a technique aimed at optimizing resemblance detection, featuring two key advancements. First, it employs AVX-512 instructions to generate rolling hashes in parallel based on a novel overlapping sub-chunk partition strategy, and optimizes the linear transformation process, thereby reducing computational overhead while eliminating computational errors introduced by parallelization. Second, it introduces a locality-aware feature matching technique to further detect potentially similar chunks and enhance the detection capabilities. Experimental results show that ERD accelerates feature extraction for resemblance detection by$2.1\times\sim 2.18\times$(the state-of-the-art method Finesse) and$5.66\times\sim 6.21\times$(N-trans) while increasing the delta compression ratio by up to 32.5% (18.2% on average) (Finesse).

Existing graph systems focus mainly on the execution efficiency of the graph analysis tasks, often ignoring the importance and efficiency of time-evolving graph storage. However, to effectively mine the potential application values, an efficient storage system is important for time-evolving graphs whose storage requirement scales with the increasing number of snapshots. Storage cost and snapshot access speed are the two most important performance indicators for a time-evolving graph storage system, which are challenging for designers of such systems because they are conflicting goals. In this article, we address these challenges by proposing an efficient storage scheme for the large time-evolving graphs. We first design aSnapshot-level Data Deduplication (SLDD)strategy to eliminate the large number of repeated vertices and edges among the snapshots, and then aStructure-Changing Graph Representation (SCGR)to significantly improve the snapshot access speed. We implement an efficient time-evolving graph storage system, TgStore, based on this scheme to effectively store large-scale time-evolving graphs, aiming to efficiently support the time-evolving graph analysis tasks. Experimental results show that TgStore can obtain a high compression ratio of 43.03:1 when storing 100 snapshots of Twitter, while with an average snapshot access speedup of 16×. Efficient storage scheme enables TgStore to efficiently support time-evolving graph algorithms. For example, when executing the Pagerank algorithm on the time-evolving graph of Twitter, TgStore outperforms Graphone, a state-of-the-art time-evolving graph storage system, by 15.9× in algorithm execution speed and 1.45× in memory usage.

With the popularity of cloud services, cloud block storage (CBS) systems have been widely deployed by cloud providers. Cloud cache plays a vital role in maintaining high and stable performance in cloud block storage systems. In the past few decades, much research has been conducted on the design of cache replacement policies. Prior work frequently relies on manually-engineered heuristics to capture the most common cache access patterns, or predict the reuse distance and try to identify the blocks that are either cache-friendly or cache-averse. Researchers are now applying recent advances in machine learning to guide cache replacement policy, augmenting or replacing traditional heuristics and data structures. However, most existing approaches depend on a certain environment which restricted their application, e.g., some methods only consider the on-chip cache consisting of program counters (PCs). Moreover, those approaches with attractive hit rates are usually unable to deal with modern irregular workloads, due to the limited feature used. In contrast, we propose a cloud cache replacement framework to automatically learn the relationship between the probability distribution of different replacement policies and workload distribution by using deep reinforcement learning. We train an end-to-end cache replacement policy based on the requested address with two efficient and stable cache replacement policies. Furthermore, by using prioritized experience replay and setting parameter constraints, our framework can accelerate the offline training process without affecting the cloud application. We have evaluated our proposed framework by using block-based I/O traces collected from Alibaba Cloud and Tencent Cloud, two of the largest cloud providers in the world, and several open-source traces. Experimental results show that our method not only outperforms several state-of-the-art cache methods in hit rate, but also reduces request latency and data traffic to the backend storage.

Since cloud service providers adopt the sharing mode to improve the utilization of solid state drives (SSDs) and reduce management costs, fairness is a critical design consideration and has drawn great interest in recent years. There are many methods to achieve fairness at the SSD device level, including cache-based resource allocation and queue rescheduling of transaction scheduling unit (TSU). However, poor locality data in cloud environments makes these methods of achieving fairness through the cache level invalid. In addition, existing fairness approaches at the TSU level treat flash memory as a black box and do not consider some characteristics of flash memory, leading to room for performance improvement. In this work, we propose a novel collaborative SSD fairness scheme, named a coordinated SSD cache and TSU fair scheme (CoFS), that achieves end-to-end full path fairness at the device level. A reinforcement learning-assisted fairness management scheme is designed to coordinate the SSD cache and TSU considering both the cache space and bandwidth resources which are significant for fairness control. The key idea is to enable the front-end SSD cache to achieve workload-level fairness by recognizing workload patterns, while the back-end TSU achieves flash-level fairness by sensing SSD internal status. Finally, CoFS collaborates with them to achieve SSD device-level fairness. In addition, we have designed a flexible reward function mechanism in the cache to balance different optimization objectives and augment TSU queue scheduling to adapt to different types of SSDs. Experimental results show that CoFS improves the overall fairness and performance by 30.8% to 56.7% and 42.1% to 98.7% in different scenarios.

Disaggregated memory (DM) architecture separates CPU and DRAM into computing/memory resource pools and interconnects them with high-speed networks. Storage systems on DM locate data by distributed index. However, existing distributed indexes either suffer from prohibitive synchronization overhead of write operation or sacrifice the performance of read operation, resulting in low throughput, high tail latency, and challenging trade-off. In this paper, we present Marlin+, a scalable and optimized B+-tree on DM. Marlin+ provides atomic granularity synchronization between write operations via three strategies: 1) a concurrent algorithm that is friendly to IDU operations (Insert, Delete, and Update), enabling different clients to concurrently operate on the same leaf node, 2) shared-exclusive leaf node lock, effectively preventing conflicts between index structure modification operation (SMO) and IDU operations, and 3) critical path compression of write to reduce latency of write operation. Moreover, Marlin+ proposes an adaptive remote address cache to accelerate the access of hot data. Compared to the state-of-the-art schemes based on DM, Marlin achieves 2.21× higher throughput and 83.4% lower P99 latency under YCSB hybrid workloads. Compared to Marlin, Marlin+ improves the throughput by up to 1.58× and reduces the P50 latency by up to 50.5% under YCSB read-intensive workloads.

Consistent Hashing (CH) algorithms are widely adopted in networks and distributed systems for their ability to achieve load balancing and minimize disruptions. However, the rise of the Internet of Things (IoT) has introduced new challenges for existing CH algorithms, characterized by high memory usage and update overhead. This article presents DxHash, a novel CH algorithm based on repeated pseudo-random number generation. DxHash offers significant benefits, including a remarkably low memory footprint, high lookup throughput, and minimal update overhead. Additionally, we introduce a weighted variant of DxHash, enabling adaptable weight adjustments to handle heterogeneous load distribution. Through extensive evaluation, we demonstrate that DxHash outperforms AnchorHash, a state-of-the-art CH algorithm, in terms of the reduction of up to 98.4% in memory footprint and comparable performance in lookup and update.

The address allocation policy in SSD aims to translate the logical address of I/O requests into a physical address, and the static address allocation is widely used in modern SSD. Through extensive experiments, we find that there are significant differences in the utilization of SSD parallelism among different static address allocation policies. We also observe that the fixed address allocation design prevents SSDs from continuing to meet the challenges posed by cloud workloads and misses the possibility of further optimization. These situations stem from our excessive reliance on SSD parallelism over time. In this paper, we proposeHsaP, ahybridstatic addressallocationpolicy, that adaptively chooses the best static allocation policy to meet the SSD performance at runtime.HsaPis adynamicscheduling scheme based onstaticaddress allocation policy. The static policy ensures thatHsaPhas stable performance and light-weight overhead, while dynamic scheduling can effectively combine different allocation policies, selecting the best-performing static mapping mode for a given SSD state. Meanwhile,HsaPcan further improve the read and write performance of SSDs simultaneously through plane reallocation and data rewrite. Experimental results show thatHsaPachieves significant read and write performance gain of a wide range of the latest cloud block storage traces compared to several state-of-the-art address allocation approaches.

Clustered storage systems often deploy erasure coding that encodes data into coded chunks and distributes them across nodes to tolerate node failures. It is a storage-efficient redundancy scheme but incurs high repair penalty; thus some state-of-the-arts aim to pipeline the above repair process to improve the repair performance. However, we observe that all existing repair pipelining methods only use a single pipeline, making network bandwidth resources of storage nodes underutilized.In this paper, we propose FullRepair, a new repair pipelining mechanism based on multiple pipelines with the aim of fully exploiting all available bandwidth resources during repair. We construct four constraints to model the repair pipelining problem such that we can obtain the optimal pipelined repair throughput under full bandwidth utilization. We design a multi-pipeline scheduling scheme for FullRepair so as to achieve the above optimality. Experiments on the Amazon EC2 show that compared with the state-of-the-art repair pipelining methods RP and PivotRepair, FullRepair reduces the repair time of single chunk by up to 45.40% and 33.19%, respectively.

Since multiple flows (users) compete for a single, shared solid state drives (SSDs) concurrently in cloud environments, fairness has drawn great interest in recent years. Although many fair schemes are proposed on different modules of SSDs, such as resource allocation for the cache and chip queue reordering for the transaction scheduling unit (TSU), they fail to continue to provide fairness for the worse locality of I/O requests due to a deeper cloud storage stack that makes cache performance poor. Moreover, existing methods only treat back-end flash as a black box and do not exploit the differences in flash features across requests. This paper proposes CoFS, a novel coordinated SSD cache and TSU light-weight learning-based fair scheme to achieve complete fairness considering cache and bandwidth resources which are significant for fairness control. The key idea is to make the front-end SSD cache achieves fairness at the workload-level by recognizing dynamic I/O changes, while the back-end TSU achieves fairness at the flash-level by awareness of flash page-types, and finally CoFS collaborates with them to achieve fairness at the SSD device-level. Experimental results show that CoFS achieves significant fairness and performance gain of a wide range of the latest cloud block storage workloads, with an average improvement of 46.3% and 94.5% compared to state-of-the-art fairness policies, respectively.

Smartphones are everyday necessities with limited power supply. Charging a smartphone twice a day or more affects user experience. Flash friendly file system (F2FS) is a widely-used log-structured file system for smartphones. Free space fragmentation of F2FS consists of invalid blocks mainly causing performance degradation. F2FS reclaims invalid blocks by garbage collection (GC). We explore the energy consumption of GC and the effect of GC on reducing free space fragments. We observe the energy consumption of one background GC is large but its effect on reducing free space fragments is limited. These motivate us to improve the energy efficiency of GC. We reassess how much free space is a free space fragment based on data analysis, use the free space fragmentation factor to measure the degree of free space fragmentation quickly. We suggest the free space fragmentation aware GC scheme (FAGC) that optimizes the selection for victim segments and the migration for valid blocks. Experiments on real platform show that FAGC reduces GC count by 82.68% and 74.51% respectively than traditional F2FS and the latest GC optimization of F2FS, ATGC. FAGC reduces the energy consumption by 164.37 J and 100.64 J compared to traditional F2FS and ATGC respectively for a synthetic benchmark.

Distributed file systems (DFS) are the cornerstone of modern mass data processing systems. In DFS, the metadata service, as the core component, often becomes a performance bottleneck. Existing metadata service solutions have implemented flattened metadata management and full-path indexing to achieve high scalability in terms of capacity and throughput. However, these approaches have limitations, such as conflicts with POSIX-style permission verification and flawed support for super directories, leading to high and unstable latency that cannot provide reliable service for latency-sensitive applications. To overcome these limitations, we propose Duplex, a scalable DFS metadata service based on full-path indexing, which aims for low and stable latency. Duplex incorporates three key designs: a fast access path featuring a centralized permission server for efficient permission verification, a permission merging algorithm to reduce the PMS's space footprint, and flattened metadata management based on double consistent hashing that enables low-latency access to super directories. Our evaluations demonstrate that, compared to state-of-the-art metadata solutions, Duplex significantly reduces the average lookup latency by up to 84% and the 99th percentile tail latency by up to 88.2% for metadataintensive benchmarks. Additionally, Duplex improves the lookup IOPS by up to 7.6 × /2.3× compared to CephFS and BeeGFS.

As the storage and I/O demand of contemporary computer systems grow, SSDs with deduplication are widely deployed as shared storage devices to provide high performance in cloud platforms where diverse tenant workloads are collocated. However, existing global and fixed fingerprint management schemes are inefficient in the multi-tenant environment. Contention for in-memory fingerprint cache, embedded in DRAM to speed up fingerprint lookups, results in decreased cache utilization, and abundant fingerprint lookups into the backend flash memory degrade the system performance. Besides, a fixed cache replacement policy for all the tenants fails to capture the diverse characteristics, further increasing the lookup overhead.This paper introduces CostFM, a novel fingerprint management mechanism highlighted by two notable features. First, it provides a benefit-aware cache allocation scheme, which estimates the cache hit gains of each tenant and uses dynamic programming to find the optimal cache allocation scheme, avoiding the cache contention for the fingerprint resource and improving cache utilization. Second, it applies a user-based policy model that captures access characteristics to select the suitable cache management policy for each tenant, boosting overall cache efficiency. Extensive experimental results show that CostFM decreases the average request latency by up to 30%, and it also reduces the fingerprint writes into the flash memory by up to 4.7x.

Applications powered by Deep Neural Networks (DNNs) are shifting from cloud to edge computing due to the promising performance efficiency of Resistive Random Access Memory (RRAM) devices. However, modern RRAM devices have various possible faults, including hard and soft faults. Existing RRAM-based deep learning systems lack a comprehensive fault tolerance approach, leading to safety risks in deploying edge-end DNN inference. In this paper, we propose an effective fault recovery framework called TOSA to safeguard edge-based DNN accelerators from Stuck-At-Faults (SAFs). Two novel ideas are presented to enhance reliability in edge-based RRAM inference accelerators: (1) SAFs-aware priority-based bit-memristor mapping algorithm to match integrated SAF variations and DNN’s defect tolerance, (2) a new retraining method called joint retraining to recover model accuracy from multiple faulty edge devices. We thoroughly evaluate TOSA on multiple DNN models, demonstrating its effectiveness in reducing costs and improving efficiencies. Experimental results show that TOSA achieves a 45.28× speedup when compared to the per-device retraining methods while preserving inference accuracy better than the latest competitor, making it a reliable architecture for accelerating DNN inference on edge devices.

Memory disaggregation architecture can achieve higher resource utilization, independent scaling of CPUs and memory. Disaggregated memory systems manage memory resources and locate data by distributed index. However, existing distributed indexes suffer from high synchronization overhead of write, due to naive node lock, thus resulting in high tail latency, low throughput, and poor scalability.

Object cloud storage systems are deployed with diverse applications that have varying latency service level objectives (SLOs), posting challenges for supporting quality of service with limited storage resources. Existing methods provide prediction-based recommendations for dispatching requests from applications to storage devices, but the prediction accuracy can be affected by complex system topology. To address this issue, Graph3PO is designed to combine storage device queue information with system topological information for forming a temporal graph, which can accurately predict device queue states. Additionally, Graph3PO contains the urgency degree model and cost model for measuring SLO violation risks and penalties of scheduling requests on storage device queues. When the urgency degree of a request exceeds a threshold, Graph3PO determines whether to schedule it in the queue or initiate a hedge request to another storage device. Experimental results show that Graph3PO outperforms its competitors, with SLO violation rates 2.8 to 201.1 times lower.

Abstract Accurate resource allocation and colocating jobs are effective ways to increase resource utilization and reduce costs in modern datacenters. The main challenges are fluctuations in resource consumption and interference among colocated jobs. Therefore, we propose a resource management schema based on the probability distributions of resource consumption and completion time for multitenant cloud clusters. First, we found that the characteristics of the task can be well described by the probability distribution of resource consumption and completion time, and the probability distribution function can be obtained by Gaussian fitting. Second, we propose a probability distribution based resource allocation (PDRA) strategy for batch jobs. Third, we design a tail latency aware allocation (TLAA) strategy to use transient resources efficiently while ensuring tail latency requirements. Finally, we design a cost‐effective resource revocation (CERR) strategy to revoke transient resources with minimal eviction costs. Experimental results demonstrate the efficiency of our resource management. Our resource allocation strategies (PDRA and TLAA) can effectively improve resource utilization and reduce job completion time. CERR can reduce the impact of resource revocation on batch jobs and perform better than existing resource revocation strategies.

Graph embedding maps graph nodes to low-dimensional vectors, and is widely adopted in machine learning tasks. The increasing availability of billion-edge graphs underscores the importance of learning efficient and effective embeddings on large graphs, such as link prediction on Twitter with over one billion edges. Most existing graph embedding methods fall short of reaching high data scalability. In this paper, we present a general-purpose, distributed, information-centric random walk-based graph embedding framework, DistGER, which can scale to embed billion-edge graphs. DistGER incrementally computes information-centric random walks. It further leverages a multi-proximity-aware, streaming, parallel graph partitioning strategy, simultaneously achieving high local partition quality and excellent workload balancing across machines. DistGER also improves the distributed Skip-Gram learning model to generate node embeddings by optimizing the access locality, CPU throughput, and synchronization efficiency. Experiments on real-world graphs demonstrate that compared to state-of-the-art distributed graph embedding frameworks, including KnightKing, DistDGL, and Pytorch-BigGraph, DistGER exhibits 2.33×--129× acceleration, 45% reduction in cross-machines communication, and >10% effectiveness improvement in downstream tasks.

Disaggregated memory architecture has risen in popularity for large datacenters with the advantage of improved resource utilization, failure isolation, and elasticity. Replicated state machines (RSMs) have been extensively used for reliability and consistency. In traditional RSM protocols, each replica stores replicated data and has the computing power to participate in some part of the protocols. However, traditional RSM protocols fail to work in the disaggregated memory architecture due to asymmetric resources on CPU nodes and memory nodes. This article proposes ECHO, a fast one-sided RDMA-based RSM protocol with lightweight log replication and remote applying, efficient linearizability guarantee, and fast coordinator failure recovery. ECHO enables all operations in the protocol to be efficiently executed using only one-sided RDMA, without the participation of any computing resource in the memory pool. To provide lightweight log replication and remote applying, ECHO couples the replicated log and the state machine to avoid dual-copy and performs remote applying by updating pointers. To enable efficient remote log state management, ECHO leverages a hitchhiked log state updating scheme to eliminate extra network round trips. To provide efficient linearizability guarantee, ECHO performs immediate remote applying after log replication and leverages the local locks at the coordinator to ensure linear consistency. Moreover, ECHO adopts a commit-aware log cache to make data visible immediately after being committed. To achieve fast failure recovery, ECHO leverages a commit point identification scheme to reduce the overhead of log consistency recovery. Experimental results demonstrate that ECHO outperforms the state-of-the-art RSM protocol (namely Sift) in multiple scenarios. For example, ECHO achieves 27%–52% higher throughput on typical write-intensive workloads. Moreover, ECHO reduces the consistency recovery time by three orders of magnitude for coordinator failure.

Graph embedding is becoming widely adopted as an efficient way to learn graph representations required to solve graph analytics problems. However, most existing graph embedding methods, owing to computation-efficiency challenges for large-scale graphs, generally employ a one-size-fits-all strategy to extract information, resulting in a large amount of redundant or inaccurate representations. In this work, we propose HuGE+, an efficient and scalable graph embedding method enabled by an entropy-driven mechanism. Specifically, HuGE+ leverages hybrid-property heuristic random walk to capture node features, which considers both information content of nodes and the number of common neighbors in each walking step. More importantly, to guarantee information effectiveness of sampling, HuGE+ adopts two heuristic methods to decide the random walk length and the number of walks per node, respectively. Extensive experiments on real-world graphs demonstrate that HuGE+ achieves both efficiency and performance advantages over recent popular graph embedding approaches. For three downstream graph tasks, our approach not only offers >10% average gains, but also exhibits 23×–127× speedup over existing sampling-based methods. In addition, HuGE+ significantly reduces memory footprint by an average of 68.9%, facilitating training for billion-node-scale graph embeddings.

The LSM-tree based key-value store has been widely deployed in modern storage systems because of its optimized write performance. However, its query performance is frequently criticized owing to the multi-layer structure. Caching is one of the most common approaches for boosting read performance, but existing caching in LSM-tree may be invalidated due to compaction. Simultaneously, obsolete data brought by duplicate writes may cause significant read and write amplification during compaction. Caching invalidation and compaction with obsolete data both impair performance.By developing a fine-grained Entry Cache in memory and inventing a hotspot-aware compaction strategy, we propose a Hotspot-aware LSM-tree based storage engine (simplified as HaLSM). The Entry Cache stores hot data in memory efficiently, and all data in it are valid indefinitely and are unaffected by compaction. Using the hotspot-aware compaction strategy, HaLSM can reduce the impact on tree structures brought by hotspot data and keep hot data in upper levels. We have implemented HaLSM based on LevelDB, and the evaluation results with the YCSB benchmark show that the performance of HaLSM is higher than that of LevelDB in various workloads, and the number of disk I/Os can be reduced significantly.

As a vital method for computer system design, multilevel cache technology is still a research hotspot in the field of storage. Recently, researchers have designed many multilevel cache replacement algorithms by analyzing the historical trajectories of data blocks between different cache levels but suffer from high overhead and poor adaptability. In addition, these methods still try to determine the most suitable victim to be replaced, given a new block to be loaded into the cache but do not consider how to cooperate between different cache levels. To overcome these challenges, we propose a multi-factor adaptive multi-level cooperative cache replacement policy in block storage systems that leverages the hierarchical characteristics of multilevel cache and multi-factor orthogonality by the probability distribution. This adaptation is mainly reflected in two aspects: each cache level automatically adapts different cache policies based on I/O access patterns, and different cache levels cooperate by adjusting the dynamic frequency threshold. In this work, we choose as our target for optimization the replacement policy of the cloud block cache, because rich storage stacks in cloud environments can be well used as multi-level cache scenarios. We have evaluated our proposed framework by using block-based I/O traces collected from Alibaba Cloud, one of the largest cloud providers in the world, and several open-source traces. Experimental results show that our method improves the hit ratios by 28.9% and reduces the response time by 9.6% to the state-of-the-art cache replacement policies for multi-level cache across different workloads and cache sizes.

Data security is a key issue that non-volatile memory (NVM) system designers must consider. However, this is challenging because implementing security mechanisms such as bonsai merkle tree (BMT) in NVM needs to ensure crash recovery due to the non-volatile property of NVM. Existing schemes fail to efficiently guarantee the atomic BMT root update and instant system recovery required for BMT crash recovery, resulting in large write traffic and performance overhead. In this paper, we propose DR-TREE, a dynamic and recoverable BMT scheme for secure NVM, which reduces the update overhead of BMT root and achieves fast crash recovery with low write traffic. DR-TREE dynamically builds BMT and adjusts the updated BMT levels according to memory write requests, thus reducing unnecessary update overhead of BMT root. Next, based on the locality of memory write requests, DR-TREE merges repeated BMT updates, further decreasing the update overhead of BMT root. Moreover, DR-TREE achieves fast crash recovery with extremely low write traffic by delaying the partial recovery process. Experiments show that compared to the state-of-the-art design, DR-TREE improves the performance by 44.6%, decreases write traffic by 78.2% and achieves the system recovery in 5ms.

In recent years, system researchers have proposed many out-of-core graph processing systems to efficiently handle graphs that exceed the memory capacity of a single machine. Through disk-friendly graph data organizations and well-designed execution engines, existing out-of-core graph processing systems can maintain sequential locality on disk access and greatly reduce disk I/Os during processing. However, they have not fully explored the characteristics of graph data and algorithm execution to further reduce disk I/Os, leaving significant room for performance improvement. In this paper, we present a novel out-of-core graph processing system called GraphSD, which optimizes the I/O traffic by simultaneously capturing the state and dependency of graph data during computation. At the heart of GraphSD is a state- and dependency-aware update strategy that includes two adaptive update models, selective cross-iteration update (SCIU) and full cross-iteration update (FCIU). These two update models are dynamically triggered at runtime to enable active-vertex aware processing and cross-iteration vertex value computation, which avoid loading inactive edges and reduce disk I/Os in the future iterations. Moreover, an efficient sub-block based buffering scheme is proposed to further minimize I/O overheads. Our evaluation results show that GraphSD outperforms two state-of-the-art out-of-core graph processing systems HUS-Graph and Lumos by up to 2.7 × and 3.9 × respectively.

Persistent memory (PM) exhibits a huge potential to provide B+-tree indexes with high performance, efficient persistence, and instant recovery. A large number of PM-optimized B+-tree indexes have been proposed, but most of them fail to provide high performance for both read and write operations because: (1) their designs of search optimization and insert improvement are often traded off against each other, and (2) they overlook the read/write interference problem of PM which incurs unpredictable performance degradation. In this paper, we propose ROWE-tree, a read-optimized and write-efficient B+-tree for PM. The designs of our ROWE-tree consist of three key points. First, we propose two techniques to make a good trade-off between write and read performance: self-verifying insertion, which reduces consistency overhead by using the key itself as a persist mark instead of additional metadata, and semi-sorted leaf nodes, which use append-only insertion to avoid the shifting overhead of sorting nodes but keep intra-cache-line items sorted to accelerate the lookup. Second, based on the observation that data accesses are highly skewed in real-world workloads, we build an in-DRAM cache of hot items to outsource accesses to hot items to DRAM. By doing so, we can alleviate the read/write interference of PM and significantly improve overall performance. Third, to cope with the dynamic changes of hot items, we exploit a lightweight mechanism to track such changes at run-time. Using Intel Optane DCPMM, our evaluations show that ROWE-tree obtains up to 3.86 × higher performance than the state-of-the-art PM B+-tree indexes under YCSB workloads.

Abstract Data deduplication is a widely used technique to remove duplicate data to reduce the storage overhead. However, deduplication typically cannot eliminate the redundancy among nonidentical but similar data chunks. To reduce the storage overhead further, delta compression is often applied to compress the post‐deduplication data. While the two techniques are effective in saving storage space, they introduce complex references among data chunks, which inevitably undermines the system reliability and introduces fragmentation that may degrade the restore performance. In this paper, we observe that the delta compressed chunks (DCCs) are much smaller than regular chunks (non‐DCCs). Also, most fragmentation caused by the base chunk of DCCs remain fragmented in consecutive backups. Based on these observations, we introduce a framework called , which combines replication and erasure coding and uses History‐aware Delta Selection to ensure high reliability and restore performance. Specifically, uses a delta‐utilization‐aware filter and a cooperative cache scheme (CCS) to maintain cache locality and avoid unnecessary container reads, respectively. Moreover, the system selectively performs delta compression by historical information to avoid cyclic fragmentation in consecutive backups. Experimental results based on four real‐world datasets demonstrate that significantly improves the restore performance by 58.3%–76.7% with a low storage overhead.

Abstract The evolution of persistent memory (PM) has significantly affected the design of today's indexing structures. Hashing‐based structures are widely used in storage systems to achieve fast query responses. Recently, several concurrent and failure‐atomic hashing schemes for PM have been proposed to improve the scalability. However, these works still suffer from limited scalability, especially under write‐intensive workloads or at a high number of threads. Our empirical study concludes three issues harm the scalability of PM hashing schemes: the NUMA effects, the resizing operations, and the inter‐thread interference overhead in PM. Based on the above scalability issues, we present SPHT, a scalable and persistent hashing scheme for hybrid DRAM‐PM memory. To eliminate the NUMA effects, SPHT maintains a hash subtable for every NUMA node and stores the key‐value items in the designated nodes. By doing so, all operations can be executed in the local memory without cross‐node communication. The hash subtable consists of two components: the search layer in DRAM for fast accesses and the data layer in PM for efficient persistence. The data layer is organized in a log‐structured way and its log chunks own separate PM space, thus avoiding the resizing operations in PM. Meanwhile, the compacted log structure also supports batching multiple small items, which effectively reduces the persistence overhead. Furthermore, to maximize concurrency, we assign threads to different partitions in the data layer to reduce the inter‐thread interference overhead. On Intel Optane DCPMM, our evaluations show that SPHT scales well and achieves up to 2.7 higher performance than state‐of‐the‐art PM hashing schemes under YCSB workloads.

Data security is an indispensable part of non-volatile memory (NVM) systems. However, implementing data security efficiently on NVM is challenging, since we have to guarantee the consistency of user data and the related security metadata. Existing consistency schemes ignore the recoverability of the SGX style integrity tree (SIT) and the access correlation between metadata blocks, thereby generating unnecessary NVM write traffic. In this article, we propose SecNVM, an efficient and write-friendly metadata crash consistency scheme for secure NVM. SecNVM utilizes the observation that for a lazily updated SIT, the lost tree nodes after a crash can be recovered by the corresponding child nodes in NVM. It reduces the SIT persistency overhead through a restrained write-back metadata cache and exploits the SIT inter-layer dependency for recovery. Next, leveraging the strong access correlation between the counter and DMAC, SecNVM improves the efficiency of security metadata access through a novel collaborative counter-DMAC scheme. In addition, it adopts a lightweight address tracker to reduce the cost of address tracking for fast recovery. Experiments show that compared to the state-of-the-art schemes, SecNVM improves the performance and decreases write traffic a lot, and achieves an acceptable recovery time.

Data deduplication is widely deployed in solid-state drives (SSDs) to improve the storage space utilization and alleviate the endurance issue. However, deduplication increases the degree of fragmentation and the access contention, significantly hurting the system performance. First, the fragmentation at the storage medium level in SSD is closely related to the internal parallelism, and the degree of fragmentation increases as the degree of the parallelism decreases. Deduplication removes the duplicate parts of the write sequences, decreasing the read parallelism. Thus, the degree of fragmentation is increased, eventually degrading the read performance. Second, the uneven distribution of the highly referenced data increases the access contention. This increased access contention prolongs the queueing time, further degrading the system performance. Motivated by our observations, we propose an elastic data cache (EDC) to improve the I/O performance in the deduplicated SSD. EDC redesigns the built-in DRAM-based data cache, tracks the popular and highly-referenced data, and maintains them in the cache. To fulfill the novel data cache, EDC changes the request process. The read requests that access the fragments in the flash memory are mostly performed in the fast-speed data cache, alleviating the negative impact of fragmentation on the read performance. The reduced read accesses to the flash memory also ease the access contention, which improves the system performance significantly. Extensive experimental results validate the efficiency of EDC, showing that it effectively improves the read performance and the write performance by up to 79% and 85% on average, respectively, in the deduplicated SSD.

As we all know, file and free space fragmentation negatively affect file system performance. F2FS is a file system designed for flash memory. However, it suffers from severe fragmentation due to out-of-place updates and the highly synchronous, multithreaded writing behaviors of applications. The adaptive reserved space (ARS) scheme chooses some files instead of all files to update in the reserved space which collects file features associated with fragmentation to construct datasets and uses decision trees to pick reserved files. However, ARS processes file features according to writefds. On the one hand,fdsand files are mapped many-to-many, so there is an issue in mapping reserved files according to reservedfdsthat a normal file is selected as a reserved file. On the other hand, the mapping of files tofdsfor big data traces is confusing. We propose ARST that traces the file of writefdto optimize ARS. Moreover, by selecting time-independent file features, ARST predicts whether files with little historical information are reserved to maximize the performance improvement brought by reserving space. Besides, adjustable reserved space and dynamic reservation strategies are adopted. We implement ARST on a HiKey960 development platform and a commercial smartphone with slight space and file creation time overheads. Experimental results show that ARST reduces file and free space fragmentation dramatically and improves file system performance. ARST reduces the running time of realistic workloads by up to 94.28% than F2FS with in-place updates and outperforms ARS by up to 49.06% for Wechat.

Disk failure has always been a major problem for data centers, leading to data loss. Current disk failure prediction approaches are mostly offline and assume that the disk labels required for training learning models are available and accurate. However, these offline methods are no longer suitable for disk failure prediction tasks in large-scale data centers. Behind this explosive amount of data, most methods do not consider whether it is not easy to get the label values during the training or the obtained label values are not completely accurate. These problems further restrict the development of supervised learning and offline modeling in disk failure prediction. In this article, Active Semi-supervised Learning Disk-failure Prediction ( ASLDP ), a novel disk failure prediction method is proposed, which uses active learning and semi-supervised learning. According to the characteristics of data in the disk lifecycle, ASLDP carries out active learning for those clear labeled samples, which selects valuable samples with the most significant probability uncertainty and eliminates redundancy. For those samples that are unclearly labeled or unlabeled, ASLDP uses semi-supervised learning for pre-labeled by calculating the conditional values of the samples and enhances the generalization ability by active learning. Compared with several state-of-the-art offline and online learning approaches, the results on four realistic datasets from Backblaze and Baidu demonstrate that ASLDP achieves stable failure detection rates of 80–85% with low false alarm rates. In addition, we use a dataset from Alibaba to evaluate the generality of ASLDP . Furthermore, ASLDP can overcome the problem of missing sample labels and data redundancy in large data centers, which are not considered and implemented in all offline learning methods for disk failure prediction to the best of our knowledge. Finally, ASLDP can predict the disk failure 4.9 days in advance with lower overhead and latency.

With the widespread use of flash memory from mobile devices to large data centers, flash friendly file system (F2FS) designed for flash memory features has become popular. However, F2FS suffers from severe cleaning overhead due to its logging scheme writes. Mixed storage of data with different hotness in the file system aggravates segment cleaning. We propose multi-log delayed writing and modified segment cleaning based on dynamically identified hotness (M2H). M2H defines the hotness by the file block update distance and uses the K-means clustering to identify hotness accurately for dynamic access patterns. Based on fine-grained hotness, we design multi-log delayed writing and modify the selection and release of the victim segment. The hotness metadata cache is used to reduce overheads induced by hotness metadata management and clustering calculations. Compared with the existing strategy of F2FS, migrated blocks of segment cleaning in M2H reduce by 36.05% to 36.51% and the file system bandwidth increases by 69.52% to 70.43% cumulatively.

Graph embedding is becoming widely adopted as an efficient way to learn graph representations required to solve graph analytics problems. However, most existing graph embedding methods, owing to computation-efficiency challenges for large-scale graphs, generally employ a one-size-fits-all strategy to extract information, resulting in a large amount of redundant or inaccurate representations. In this work, we propose HuGE, an efficient and scalable graph embedding method enabled by an entropy-driven mechanism. Specifically, HuGE leverages hybrid-property heuristic random walk to capture node features, which considers both node degree and the number of common neighbors in each walking step. More importantly, to guarantee information effectiveness of sampling, HuGE adopts two heuristic methods to decide the random walk length and the number of walks per node, respectively. Extensive experiments on real-world graphs demonstrate that HuGE achieves both efficiency and performance advantages over recent popular graph embedding approaches. For link prediction and multi-label classification, our approach not only offers >10% average gains, but also exhibits 22×-126× speedup compared with existing sampling-based methods.

For performance benefits, recent trends of modern data centers tend to use NVM (Non-volatile Memory) as storage and utilize one-sided primitives of RDMA (Remote Direct Memory Access) to directly access NVM for I/O requests. However, one-sided RDMA doesn’t have durability semantics currently, and data may partially exist in the NVM when crashes happen, thus causing unexpected data inconsistency.

Data security is a very important issue in non-volatile memory (NVM) systems. Due to high security and low decryption latency, counter mode encryption (CME) is often used, which unfortunately suffers from extra efforts to guarantee the crash consistency of counters. Existing schemes fail to efficiently guarantee the crash consistency of counters and data, thus resulting in inefficient recovery and high costs. In this paper, we propose a Crash-Consistency-Aware Encryption scheme (CCAE) for NVM systems. CCAE is inspired by two key observations. First, we observe that all used log entries in a transaction have the same number of encryption times before being recycled due to the write appending feature. This motivates our shared counter optimization for log encryption. Second, we observe that the counters related to uncommitted data blocks only need to be restored to the latest or newer value before the crash to avoid OTP reuse. This motivates our delayed counter persistency scheme for data encryption. Evaluation results show that compared with the state-of-the-art design, CCAE reduces NVM write traffic caused by counters by 67%, improves system performance by 14%, and decreases NVM energy consumption by 35%.

Disk failure has always been a major problem for data centers, leading to data loss. Current research works used supervised learning to offline training through a large number of labeled samples. However, these offline methods are no longer suitable for disk failure prediction tasks in the current big data environment. Behind this explosive amount of data, most methods do not take into account the label values used in the model training phase may not be easy to obtain, or the obtained label values are not completely accurate. These problems further restrict the development of supervised learning and offline modeling in disk failure prediction. In this paper, ASLDP, a novel disk failure prediction method is proposed, which uses active learning and semi-supervised learning. According to the characteristics of data in the disk life cycle, ASLDP carries out active learning for those clear labeled samples, which selects valuable samples and eliminates redundancy. For those samples that are unclearly labeled or unlabeled, ASLDP combines with semi-supervised learning for pre-labeled, and enhances the generalization ability by active learning. The results on three realistic datasets demonstrate that ASLDP achieves stable failure detection rates of 80-85% with low false alarm rates, compared to current online learning methods. Furthermore, ASLDP can overcome the problems of the sample label missing and data redundancy in the massive data environment, compared to current offline learning methods.

Big data applications increasingly rely on the analysis of large graphs. In order to analyze and process the large graphs with high cost efficiency, researchers have developed a number of out-of-core graph processing systems in recent years based on just one commodity computer. On the other hand, with the rapidly growing need of analyzing graphs in the real-world, graph processing systems have to efficiently handle massive concurrent graph processing (CGP) jobs. Unfortunately, due to the inherent design for single graph processing job, existing out-of-core graph processing systems usually incur redundant data accesses and storage and severe competition of I/O bandwidth when handling the CGP jobs, thus leading to very long waiting time experienced by users for the computing results. In this paper, we propose an I/O-efficient out-of-core graph processing system, GraphCP, to support the processing of CGP jobs. GraphCP proposes a benefit-aware sharing execution model that shares the I/O access and processing of graph data among the CGP jobs and adaptively schedules the loading of graph data, which efficiently overcomes above challenges faced by existing out-of-core graph processing systems. In addition, GraphCP organizes the graph data with a Source-Sorted Sub-Block graph representation for better processing capacity and I/O access locality. Extensive evaluation results show that GraphCP is 10.3x and 4.6x faster than two state-of-the-art out-of-core graph processing systems GridGraph and GraphZ respectively, and 2.1x faster than a CGP-oriented graph processing system Seraph.

The integrity tree is a crucial part of the secure non-volatile memory (NVM) system design. For NVM with large capacity, the SGX-style integrity tree (SIT) is practical due to its parallel updates and variable arity. However, employing SIT in secure NVM is not easy. This is because the secure metadata SIT must be strictly persisted or restored after a sudden power¬loss, which unfortunately incurs unacceptable run-time overhead or recovery time. In this paper, we propose PSIT, a metadata persistency solution for SIT-protected secure NVM with high performance and fast restoration. PSIT utilizes the observation that for a lazily updated SIT, the lost tree nodes after a crash can be recovered by the corresponding child nodes in the NVM. It reduces the persistency overhead of the SIT nodes through a restrained write-back meta-cache and leverages the SIT inter¬layer dependency for recovery. Experiments show that compared to ASIT, a state-of-the-art secure NVM using SIT, PSIT decreases write traffic by 47% and improves the performance by 18% on average while maintaining an acceptable recovery time.

As we all know, file and free space fragmentation negatively affect file system performance. F2FS is a file system designed for flash memory. However, it suffers from severe fragmentation due to its out-of-place updates and the highly synchronous, multi-threaded writing behaviors of mobile applications. We observe that the running time of fragmented files is 2.36× longer than that of continuous files and that F2FS's in-place update scheme is incapable of reducing fragmentation. A fragmented file system leads to a poor user experience.(p)(/p)Reserving space to prevent fragmentation is an intuitive approach. However, reserving space for all files wastes space since there are a large number of files. To deal with this dilemma, we propose an adaptive reserved space (ARS) scheme to choose some specific files to update in the reserved space. How to effectively select reserved files is critical to performance. We collect file characteristics associated with fragmentation to construct data sets and use decision trees to accurately pick reserved files. Besides, adjustable reserved space and dynamic reservation strategy are adopted. We implement ARS on a HiKey960 development platform and a commercial smartphone with slight space and file creation time overheads. Experimental results show that ARS reduces file and free space fragmentation dramatically, improves file I/O performance and reduces garbage collection overhead compared to traditional F2FS and F2FS with in-place updates. Furthermore, ARS delivers up to 1.26× transactions per second under SQLite than traditional F2FS and reduces running time of realistic workloads by up to 41.72% than F2FS with in-place updates.

Emerging Non-Volatile Memory (NVM) techniques suffer from many attacks on endurance and data confidentiality. Oblivious RAM (ORAM) can prevent both types of attacks by address obfuscation and encryption. However, ORAM also introduces a lot of redundant memory operations, worsening the NVM endurance and system performance. Furthermore, the slower NVM increases the response latency of target blocks compared to DRAM. As the read requests are on the critical path of program execution, the increased latency further reduces system performance. To alleviate endurance and performance problems, we propose H2ORAM, a low response latency optimized ORAM scheme for NVM. Hybrid DRAM/NVM system is introduced to eliminate 57% read/write traffic to NVM, which improves the lifetime of NVM devices. To reduce response latency of read requests, we securely manipulate data distributions in hybrid memory to speed up hot data access. The manipulation leverages a frequency-based eviction scheme and a data duplication scheme to keep more hot data accessible from DRAM earlier and faster. Moreover, we utilize the empty slots in DRAM region to further improve performance. The experimental results show that 44% response latency reduction and 18% performance improvement are achieved over the basic Ring ORAM with hybrid memory architecture integrated.

Datacenter network load balancing schemes handle network traffic generated by massive different applications. Some packet-based or flowlet-based schemes capture traffic bursts for load balancing. But frequent rerouting within a flow can mix ACKs belonging to different paths in congestion control protocols, which adversely affects flow rate control. Besides, performance optimization effect of flowlet-based schemes may be less noticeable under smoother workloads. And several packet-based mechanisms implemented at end hosts can proactively reroute congested flows based on flow status even under a smooth workload, but fail to improve performance with the bursty nature of traffic. Therefore, existing schemes cannot adapt to different burst levels of dynamic traffic in datacenter networks and still have significant performance flaws in some ways. This paper proposes IntFlow, a novel load balancing scheme that integrates end-host based per-packet monitoring of flow status with flowlet switching in programable switches. IntFlow proactively reroutes flows experiencing network congestion or failures and avoids doing flowlet switching for small flows with high sending rate. IntFlow can provide excellent performance under both high burst and smooth workloads. Finally experimental results show IntFlow achieves up to 32% and 28% better performance than CONGA and Hermes under asymmetries, respectively.

In recent years, a number of out-of-core graph processing systems have been proposed to process graphs with billions of edges on just one commodity computer, due to their high cost efficiency. To obtain a better performance, these systems adopt a full I/O model that scans all edges during the computation to avoid the inefficiency of random I/Os. Although this model ensures good I/O access locality, it leads to a large number of useless edges to be loaded when running graph algorithms that only access a small portion of edges in each iteration. An intuitive method to solve this I/O inefficiency problem is the on-demand I/O model that only accesses the active edges. However, this method only works well for the graph algorithms with very few active edges, since the I/O cost will grow rapidly as the number of active edges increases due to the increasing amount of random I/Os. In this article, we present HUS-Graph, an efficient out-of-core graph processing system to address the above I/O issues and achieve a good balance between I/O traffic and I/O access locality. HUS-Graph adopts a hybrid update strategy including two update models, Row-oriented Push (ROP) and Column-oriented Pull (COP). It supports switching between ROP and COP adaptively, for the graph algorithms that have different computation and I/O features. For traversal-based algorithms, HUS-Graph also provides an immediate propagation-based vertex update scheme to accelerate the vertex state propagation and convergence speed. Furthermore, HUS-Graph adopts a locality-optimized dual-block representation to organize graph data and an I/O-based performance prediction method to enable the system to dynamically select the optimal update model between ROP and COP. To save the disk space and further reduce I/O traffic, HUS-Graph implements a space-efficient storage format by combining several graph compression methods. Extensive experimental results show that HUS-Graph outperforms two existing out-of-core systems GraphChi and GridGraph by 1.2x-52.8x.

Data deduplication, though being efficient for redundancy elimination in storage systems, introduces chunk fragmentation which severely decreases restore performance. Rewriting algorithms are proposed to reduce the chunk fragmentation. Typically, the backup software aggregates files into larger “tar” type files for storage. We observe that, in tar type datasets, a large number of Persistent Fragmented Chunks (PFCs) are repeatedly rewritten by state-of-the-art rewriting algorithms in every backup, which severely impacts restore performance. We found that the existence of PFCs is due to the traditional strategy of storing PFCs along with other chunks in the containers to preserve the stream locality, rendering them always stored in the containers with low utilization. We propose DePFC to reduce PFCs. DePFC identifies and removes PFCs from the containers preserving the stream locality, and groups them together, to increase the utilization of containers holding them for the subsequent backup, thus preventing them from being rewritten again. We further propose an FC Buffer to avoid mistaken rewrites of PFCs and grouping PFCs that cause restore cache thrashing together. Experimental results demonstrate that DePFC improves restore performance of state-of-the-art rewriting algorithms by 44.24-89.42 percent, while attaining comparable deduplication efficiency, and FC Buffer further improves restore performance.

Data deduplication is a widely deployed technique to remove duplicate content to save storage space, which is however incapable of eliminating the redundancy between nonidentical but similar data blocks. To achieve further space savings in deduplicated storage systems, delta compression is employed to compress post-deduplication data. Both deduplication and delta compression introduce content references among blocks, which inevitably undermines the reliability of deduplicated and delta compressed storage systems. To ensure better reliability, existing approaches utilize either replication or erasure codes to redundantly distribute data across multiple nodes. In deduplicated and delta compressed storage systems, we observe that delta compressed chunks (DCCs) are far smaller than regular chunks called non-DCCs. Motivated by this observation, we suggest a straightforward approach in which replication is used to protect DCCs and erasure code is deployed to protect non-DCCs. However, we need to address two critical challenges to ensure this solution effective. First, the random placement of DCCs replicas destroys cache locality. Second, the separate and individual recovery and restore cache could cause storage containers to be accessed repeatedly. To address these two challenges, in this paper, we propose RepEC-Duet which employs both replication and erasure codes to ensure high reliability and performance for deduplicated and delta-compressed storage systems. RepEC-Duet introduces a delta-utilization-aware filter to select and replicate containers based on the percentage of DCCs in the containers to maintain cache locality. Moreover, to avoid unnecessary container reads, we design a cooperative cache scheme that is aware of both failure recovery and regular restore cache. Our experimental results based on three real-world datasets demonstrate that RepEC-Duet significantly improves the restore performance by 26%-59%, and reduces the storage overhead by 54%-98% than the existing approaches.

Reservoir sampling is widely employed to characterize connectivity of large graph streams by producing edge samples. However, existing reservoir-based sampling methods mainly characterize large graph streams by a measure of counting triangles but perform poorly in accuracy when used to analyze the topological characteristics reflected by node degrees because they produce disconnected edge samples, making them ineffective in many applications that require both types of connectivity estimation simultaneously in real time. This paper proposes a new method, called triangle-induced reservoir sampling, or T-Sample, to produce connected edge samples. While every edge in a graph stream is still processed only once by T-Sample, a dual sampling mechanism performing both uniform sampling and non-uniform sampling is carefully designed with a base reservoir and an incremental reservoir. Specifically, the uniform sampling can be used to count triangles by employing the existing algorithms while the non-uniform sampling ensures that the edge samples are connected. Experimental results driven by real datasets show that T-Sample can obtain much more accurate estimations on the distributions of node degrees than the existing reservoir-based sampling methods.

NVMe SSDs have been wildly adopted to provide storage services in cloud platforms where diverse workloads (including latency-sensitive, throughput-oriented and capacity-oriented workloads) are colocated. To achieve performance isolation, existing solutions partition the shared SSD into multiple isolated regions and assign each workload a separate region. However, these isolation solutions could result in inefficient resource utilization and imbalanced wear. More importantly, they cannot reduce the interference caused by embedded cache contention. In this paper, we present CostPI to improve isolation and resource utilization by providing latency-sensitive workloads with dedicated resources (including data cache, mapping table cache and NAND flash), and providing throughput-oriented and capacity-oriented workloads with shared resources. Specifically, at the NVMe queue level, we present an SLO-aware arbitration mechanism which fetches requests from NVMe queues at different granularities according to workload SLOs. At the embedded cache level, we use an asymmetric allocation scheme to partition the cache (including data cache and mapping table cache). For different data cache partitions, we adopt different cache polices to meet diverse workload requirements while reducing the imbalanced wear. At the NAND flash level, we partition the hardware resources at the channel granularity to enable the strongest isolation. Our experiments show that CostPI can reduce the average response time by up to 44.2%, the 99% response time by up to 89.5%, and the 99.9% by up to 88.5% for latency-sensitive workloads. Meanwhile, CostPI can increase resource utilization and reduce wear-imbalance for the shared NVMe SSD.

Deduplication, as a space-saving technology, is widely deployed in the flash-based storage systems to address the capacity and endurance limitations of flash devices. In this paper, we find that deduplication changes the physical data layout, which raises the chances of the uneven read distribution. This uneven read distribution not only increases the access contention but also deteriorates the read parallelism, thus leading to the read performance degradation. To solve this issue, we propose an efficient read-leveling data distribution scheme (RLDDS), which scatters the highly-duplicated data into different parallel units, to improve the read performance for SSDs with deduplication for access-intensive workloads. RLDDS writes data into a parallel unit with lower potential read-hotness to balance the read distribution among all the parallel units. Extensive experimental results show that RLDDS effectively improves the read performance by up to 21.61% compared to deduplication with the conventional dynamic data allocation scheme. Additional benefits of RLDDS include the promoted write performance (up to 23.69%) in access-intensive workloads and the overall system performance improvement (up to 18.22%) with the same write traffic reduction.

Block access patterns refer to the regularities of accessed blocks, and can be used to effectively enhance the intelligence of block storage systems. However, existing algorithms fail to uncover block access patterns in efficient ways. They either suffer high time and space overhead or only focus on the simplest patterns like sequential ones.

Big data applications increasingly rely on the analysis of large graphs. In recent years, a number of out-of-core graph processing systems have been proposed to process graphs with billions of edges on just one commodity computer, by efficiently using the secondary storage (e.g., hard disk, SSD). On the other hand, the vertex-centric computing model is extensively used in graph processing thanks to its good applicability and expressiveness. Unfortunately, when implementing vertex-centric model for out-of-core graph processing, the large number of random memory accesses required to construct subgraphs lead to a serious performance bottleneck that substantially weakens cache access locality and thus leads to very long waiting time experienced by users for the computing results. In this paper, we propose an efficient out-of-core graph processing system, LOSC, to substantially reduce the overhead of subgraph construction without sacrificing the underlying vertex-centric computing model. LOSC proposes a locality-optimized subgraph construction scheme that significantly improves the in-memory data access locality of the subgraph construction phase. Furthermore, LOSC adopts a compact edge storage format and a lightweight replication of vertices to reduce I/O traffic and improve computation efficiency. Extensive evaluation results show that LOSC is respectively 6.9x and 3.5x faster than GraphChi and GridGraph, two state-of-the-art out-of-core systems.

Erasure coding is widely used in the distributed storage systems due to its significant storage efficiency compared with replication at the same fault tolerance level. However, erasure coding introduces high cross-rack traffic since (1) repairing a single failed data block needs to read other available blocks from multiple nodes and (2) updating a data block triggers parity updates for all parity blocks. In order to alleviate the impact of these traffic on the performance of erasure coding, many works concentrate on designing new transmission schemes to increase bandwidth utilization among multiple storage nodes but they don't actually reduce network traffic. With the emergence of programmable network devices, the concept of in-network computation has been proposed. The key idea is to offload compute operations onto intermediate network devices. Inspired by this idea, we propose XORInc, a framework that utilizes programmable network devices to XOR data flows from multiple storage nodes so that XORInc can effectively reduce network traffic (especially the cross-rack traffic) and eliminate network bottleneck. Under XORInc, we design two new transmission schemes, NetRepair and NetUpdate, to optimize the repair and update operations, respectively. We implement XORInc based on HDFS-RAID and SDN to simulate an in-network computation framework. Experiments on a local testbed show that NetRepair reduces the repair time to almost the same as the normal read time and reduces the network traffic by up to 41%, meanwhile, NetUpdate reduces the update time and traffic by up to 74% and 30%, respectively.

Recent research works on disk failure prediction achieve a high detection rate and a low false alarm rate with complex models at the cost of explainability. The lack of explainability is likely to hide bias or overfitting in the models, resulting in bad performance in real-world applications. To address the problem, we propose a new explanation method DFPE designed for disk failure prediction to explain failure predictions made by a model and infer prediction rules learned by a model. DFPE explains failure predictions by performing a series of replacement tests to find out the failure causes while it explains models by aggregating explanations for the failure predictions. A presented use case on a real-world dataset shows that compared to current explanation methods, DFPE can explain more about failure predictions and models with more accuracy. Thus it helps to target and handle the hidden bias and overfitting, measures feature importances from a new perspective and enables intelligent failure handling.

Most high-performance computing (HPC) clusters use a global parallel file system to enable high data throughput. The parallel file system is typically centralized and its storage media are physically separated from the compute cluster. Compute nodes as clients of the parallel file system are often additionally equipped with SSDs. The node internal storage media are rarely well-integrated into the I/O and compute workflows. How to make full and flexible use of these storage media is therefore a valuable research question.

Summary Performance isolation is highly desirable in cloud platforms where the virtual disks of virtual machines are simply large files on the shared and networked storage servers. However, existing isolation techniques cannot deal with the implications of the file system used by the networked storage servers, such that underlying resource usage is unpredictable (eg, the delayed write‐back mechanism could postpone writes, and the journaling mechanism could amplify writes). The lack of visibility on underlying resource usage leads to the predicament of being unable to meet isolation goals. In this paper, we present a software‐defined file system (SDFS) that exploits the underlying file system to allocate resources at per‐image‐file granularity and provide tenants with guaranteed throughput. The SDFS comprises two components: control plane and data plane. At the control plane, we provide a set of system calls to document tenant performance requirements into the metadata of image files. At the data plane, we construct a file‐based scheduler to manage memory and disk resources according to the tenant performance requirements. The SDFS design does not require a modification to guest operating systems, hypervisors, or file server protocols. Through a prototype implementation, we demonstrate that the SDFS can meet isolation goals and increase resource utilization with negligible overhead.

Nowadays, high-bandwidth networks are more easily accessible than ever before. However, existing distributed graph-processing frameworks, such as GPS, fail to efficiently utilize the additional bandwidth capacity in these networks for higher performance, due to their inefficient computation and communication models, leading to very long waiting times experienced by users for the graph-computing results. The root cause lies in the fact that the computation and communication models of these frameworks generate, send and receive messages so slowly that only a small fraction of the available network bandwidth is utilized. In this paper, we propose a high-performance distributed graph-processing framework, called BlitzG, to address this problem. This framework fully exploits the available network bandwidth capacity for fast graph processing. Our approach aims at significant reduction in (i) the computation workload of each vertex for fast message generation by using a new slimmed-down vertex-centric computation model and (ii) the average message overhead for fast message delivery by designing a light-weight message-centric communication model. Evaluation on a 40Gbps Ethernet, driven by real-world graph datasets, shows that BlitzG outperforms GPS by up to 27x with an average of 20.7x.

This paper proposes a simple CNN model for creating general-purpose sentence embeddings that can transfer easily across domains and can also act as effective initialization for downstream tasks. Recently, averaging the embeddings of words in a sentence has proven to be a surprisingly successful and efficient way of obtaining sentence embeddings. However, these models represent a sentence, only in terms of features of words or uni-grams in it. In contrast, our model (CSE) utilizes both features of words and n-grams to encode sentences, which is actually a generalization of these bag-of-words models. The extensive experiments demonstrate that CSE performs better than average models in transfer learning setting and exceeds the state of the art in supervised learning setting by initializing the parameters with the pre-trained sentence embeddings.

Nowadays, there are lots of disks from various disk models in datacenter. It is a challenge to make failure prediction for all disk models with high precision and high coverage. One-for-one modeling, transfer learning modeling and one-for-all modeling are proposed to address the challenge. However, none of them works well for all disk models and the automation problem for method selection and parameter tuning still persists. In this paper, we propose OME, an optimized modeling engine for disk failure prediction in heterogeneous datacenter. It builds a basis predictive model with one-for-all modeling and searches for the optimized with one-for-one and transfer learning modeling for every disk model. To achieve automation, OME employs a simple but effective transfer learning method, does cross-validation for comparison, prunes the tuning space, and constructs a directed acyclic graph for parallelism. Evaluation on a dataset from a real-world datacenter shows that OME outperforms a one-for-all predictive model from previous work by 18.5% overall, and the improvement for 43.3% disk models reaches over 30%.

Data deduplication, a lossless data compression technique, has been widely deployed in backup systems to save storage space. However, data fragmentation introduced by data deduplication seriously degrades restore performance. To alleviate the fragmentation problem, rewriting algorithms such as CBR, CAP, and HAR have been proposed. In certain backup datasets containing packed files like UNIX tar files, we have observed that a large amount of rewritten fragmented chunks tend to remain fragmented in following backups and thus get rewritten repeatedly in consecutive backups. Such repeatedly rewritten chunks are referred to as persistent fragmented chunks (PFCs), and they severely impact the efficiency of rewriting algorithms and restore performance. In this paper, we propose PFCG, an efficient scheme to enhance the efficiency of rewriting algorithms and restore performance. The central idea of PFCG is to identify and differentiate PFCs from regular fragmented chunks, i.e., non-PFCs, and store these two types of fragmented chunks into separate containers during backups to prevent the proliferation of PFCs in following backups. Our experimental results based on six real-world datasets demonstrate that PFCG significantly improves the restore performance by 21% to 47% over the state-of-the-art rewriting algorithms, without sacrificing deduplication efficiency.

In recent years, a number of out-of-core graph processing systems have been proposed to process graphs with billions of edges on just one commodity computer, due to their high cost efficiency. To obtain the better performance, these systems adopt a full I/O model that accesses all edges during the computation to avoid the ineffectiveness of random I/Os. Although this model ensures good I/O access locality, it loads a large number of useless edges when running graph algorithms that only require a small portion of edges in each iteration. A natural method to solve this problem is the on-demand I/O model that only accesses the active edges. However, this method only works well for the graph algorithms with very few active edges, since the I/O cost will grow rapidly as the number of active edges increases due to larger amount of random I/Os.

In datacenter networks, multipath exists to facilitate parallel data transmission. Taking deployment challenges into account, some optimized alternatives (e.g. CLOVE, Hermes) to ECMP balance load at the virtual edge or hosts. However inaccuracies of congestion detection and reaction exist in these solutions. They either detect congestion through ECN and coarse-grained RTT measurements or are congestion-oblivious. These congestion feedbacks are not sufficient enough to indicate the accurate congestion status under asymmetry. And when rerouting events occur on multiple paths, ACKs with congestion feedback of other paths can improperly influence the current sending rate. Therefore, we explore how to balance load by solving above inaccuracy problems while ensuring good adaptation to commodity switches and existing network protocols. We propose ALB, an adaptive load-balancing mechanism based on accurate congestion feedback running at end hosts, which is resilient to asymmetry. ALB leverage a latency-based congestion detection to precisely route flowlets to lighter load paths, and an ACK correction method to avoid inaccurate flow rate adjustment. In large-scale simulations ALB achieves up to 7% and 40% better flow completion time (FCT) than CONGA and CLOVE-ECN under asymmetry.

Storage hierarchy is widely used to mitigate the vast performance gap between different storage components economically, and the cache plays an important role in increasing the efficiency of memory access. However, the in-memory data organization of traditional graph computing framework is not well-optimized for various caches, especially the CPU cache, since classical caches are effectiveless towards irregular access pattern of graph applications. This work presents a cache- friendly graph data layout strategy to improve the efficiency of graph processing. By both considering the parameters of cache line and the pattern of access to adjacent list, we sort the edges to generate a sequential layout, and use BFS (Breadth First Search) algorithm to reorder the vertices for improving locality, thus benefit the CPU cache, without altering the code of graph processing toolkits. The efficiency improvement of Sort layout ranges from 11% up to 78.76% on BGL and SNAP with CC (Connected Components [1]). The BFS layout can benefit 3 classical algorithms on GraphChi: CC, TC (Triangle Counting) and PageRank [2], with the ratios of 15%-20%.

NAND flash-based Solid State Drives (SSDs) employ the Flash Translation Layer (FTL) to perform logical-to-physical address translation. Modern page-level FTLs selectively cache the address mappings in the limited SRAM while storing the mapping table in flash pages (called translation pages). However, many extra accesses to the translation pages are required for address translation, which decreases the performance and lifetime of an SSD. In this paper, we propose a Log-aware Synergized scheme for page-level FTL to reduce the extra overheads, called LSFTL. The contribution of LSFTL consists of two key elements: (i) By exploiting the partial programmability of SLC flash, in-place logging decreases garbage collection overhead via reserving a small portion of each translation page as a logging area to hold multiple updates to the entries of that translation page. (ii) Log-aware flush back reduces the number of translation page updates by evicting multiple dirty cache lines that share the same translation page in a single transaction. Extensive experimental results of trace-driven simulations show that LSFTL decreases the system response time by 39.40% on average, and up to 58.35%, and reduces the block erase count by 37.55% on average, and up to 39.99%, compared to the well-known DFTL.

Research on SCM tends to follow the conventional path: using part of persistent memory as persistent heap to recover data sets after power failure; and using part of persistent memory as persistent storage to save files abstracted by files systems. However, file abstraction causes a big performance gap between persistent heap and persistent storage. persistent data objects in persistent heap is difficult for reuse and management because of these objects losing their address meaningfulness while process is over. We propose a unified access framework to manage persistent objects in persistent heap, which provides a unified access manner for persistent objects both in persistence (storage) and in temporary (memory), not requiring serializing and de-serializing operations for file abstraction. Persistent objects are organized according to application data structures in temporary, and independently exist out of process in persistence, even the process is over. Compared to PMFS, PXFS, NV-Heap, PMSE can get 1.8X, 1.1X, 44% performance gains respectively under the workloads with mixed data sets (50% of reads, 50% of writes).

Modern distributed storage systems often store redundant data in multiple replications or erasure coding according to their access frequencies. Multiple replications scheme is well-performance for hot data while erasure coding scheme is storage-efficient for warm and cold data. When hot data turn cold, an encoding procedure starts to do the conversion. However, due to sequential striping, current conversion methods do not perform well for different data layouts, and cause risky blocks and expensive network consumption.In this paper, we propose Sice, a new encoder which deploys non-sequential striping. It constructs non-sequential stripes according to the data layout, performs conversion quickly with low overheads and ends to no reduction of system reliability. The results of both simulation and evaluation show that Sice gains almost the same good performance for different data layouts and has a great scalability. Sice helps HDFS-RAID reduce network consumption by about 65% and reduce influence on concurrent I/O-intensive applications by about 60%.

Nowadays, high-bandwidth networks are easily accessible in data centers. However, existing distributed graph-processing frameworks fail to efficiently utilize the additional bandwidth capacity in these networks for higher performance, due to their inefficient computation and communication models, leading to very long waiting times experienced by users for the graph-computing results. The root cause lies in the fact that the computation and communication models of these frameworks generate, send and receive messages so slowly that only a small fraction of the available network bandwidth is utilized. In this paper, we propose a high-performance distributed graph-processing framework, called BlitzG, to address this problem. This framework fully exploits the available network bandwidth capacity for fast graph processing. Our approach aims at significant reduction in (i) the computation workload of each vertex for fast message generation by using a new slimmed-down vertex-centric computation model and (ii) the average message overhead for fast message delivery by designing a lightweight message-centric communication model. Evaluation on a 40Gbps Ethernet, driven by real-world graph datasets, shows that BlitzG outperforms the state-of-the-art distributed graph-processing frameworks by up to 27x with an average of 20.7x.

Cloud datacenters use limited network resources to host complex and diverse applications that requires us to treat upper applications as a black box and provide low latency for latency-sensitive short flows. Many good performance schemes need to preliminarily obtain flow information (flow size, deadline or traffic distribution) or modify hardware or software, which leads to difficult use and inefficient in practice.

The size of write unit in PCM, namely the number of bits allowed to be written concurrently at one time, is restricted due to high write energy consumption. It typically needs several serially executed write units to finish a cache line service when using PCM as the main memory, which results in long write latency and high energy consumption. To address the poor write performance problem, we propose a novel PCM write scheme called Min-WU (Minimize the number of Write Units). We observe data access locality that some frequent zero-extended values dominate the write data patterns in typical multi-threaded applications (more than 40 and 44.9 percent of all memory accesses in PARSEC workloads and SPEC 2006 benchmarks, respectively). By leveraging carefully designed chip-level data redistribution method, the data amount is balanced and the data pattern is the same among all PCM chips. The key idea behind Min-WU is to minimize the number of serially executed write units in a cache line service after data redistribution through sFPC (simplified Frequent Pattern Compression), eRW (efficient Reordering Write operations method) and fWP (fine-tuned Write Parallelism circuits). Using Min-WU, the zero parts of write units can be indicated with predefined prefixes and the residues can be reordered and written simultaneously under power constraints. Our design can improve the performance, energy consumption and endurance of PCM-based main memory with low space and time overhead. Experimental results of 12 multi-threaded PARSEC 2.0 workloads show that Min-WU reduces 44 percent read latency, 28 percent write latency, 32.5 percent running time and 48 percent energy while receiving 32 percent IPC improvement compared with the conventional write scheme with few memory cycles and less than 3 percent storage space overhead. Evaluation results of 8 SPEC 2006 benchmarks demonstrate that Min-WU earns 57.8/46.0 percent read/write latency reduction, 28.7 percent IPC improvement, 28 percent running time reduction and 62.1 percent energy reduction compared with the baseline under realistic memory hierarchy configurations.

With the rapid growth of application migration, the anonymity in data center networks becomes important in breaking attack chains and guaranteeing user privacy. However, existing anonymity systems are designed for the Internet environment, which suffer from high computational and network resource consumption and deliver low performance, thus failing to be directly deployed in data centers. In order to address this problem, this paper proposes an efficient and easily deployed anonymity scheme for software defined networking-based data centers, called mimic channel (MIC). The main idea behind MIC is to conceal the communication participants by modifying the source/destination addresses, such as media access control (MAC) and Internet protocol (IP) address at switch nodes, so as to achieve anonymity. Compared with the traditional overlay-based approaches, our in-network scheme has shorter transmission paths and less intermediate operations, thus achieving higher performance with less overhead. We also propose a collision avoidance mechanism to ensure the correctness of routing, and three mechanisms to enhance the traffic-analysis resistance. To enhance the practicality, we further propose solutions to enable MIC co-existing with some MIC-incompatible systems, such as packet analysis systems, intrusion detection systems, and firewall systems. Our security analysis demonstrates that MIC ensures unlinkability and improves traffic-analysis resistance. Our experiments show that MIC has extremely low overhead compared with the base-line transmission control protocol (TCP) (or secure sockets layer (SSL)), e.g., less than 1% overhead in terms of throughput. Experiments on MIC-based distributed file system show the applicability and efficiency of MIC.

Due to the high complexity in software hierarchy and the shared queue & lock mechanism for synchronized access, existing I/O stack for accessing the FCoE based SAN storage becomes a performance bottleneck, thus leading to a high I/O overhead and limited scalability in multi-core servers. In order to address this performance bottleneck, we propose a synergetic and efficient solution that consists of three optimization strategies for accessing the FCoE based SAN storage: (1) We use private per-CPU structures and disabling kernel preemption method to process I/Os, which significantly improves the performance of parallel I/O in multi-core servers; (2) We directly map the requests from the block-layer to the FCoE frames, which efficiently translates I/O requests into network messages; (3) We adopt a low latency I/O completion scheme, which substantially reduces the I/O completion latency. We have implemented a prototype (called FastFCoE, a protocol stack for accessing the FCoE based SAN storage). Experimental results demonstrate that FastFCoE achieves efficient and scalable I/O throughput, obtaining 1132.1K/836K IOPS (6.6/5.4 times as much as original Linux Open-FCoE stack) for read/write requests.

Existing distributed graph-processing frameworks, e.g.,GPS, Pregel and Giraph, handle large-scale graphs in the memory of clusters built of commodity compute nodes for better scalability and performance. While capable of scaling out according to the size of graphs up to thousands of compute nodes, for graphs beyond a certain size, these frameworks usually require the investments of machines that are either beyond the financial capability of or unprofitable for most small and medium-sized organizations. At the other end of the spectrum of graph-processing frameworks research, the single-node disk-based graph-processing frameworks, e.g., GraphChi, handle large-scale graphs on one commodity computer, leading to high efficiency in the use of hardware but at the cost of low user performance and limited scalability. Motivated by this dichotomy, in this paper we propose a distributed disk-based graph-processing framework, called DD-Graph, that can process super-large graphs on a small cluster while achieving the high performance of existing distributed in-memory graph-processing frameworks.

Persistent memory has the potential to become universal storage for memory and storage uses. Unfortunately, our current programming model is still only geared to a two-level storage model supporting different semantics to access volatile memory and non-volatile hard disk. This, however, leads a performance gap in accessing the same persistent memory stemming from the difference in the aforementioned semantics and the performance degradation introduced by not appropriately adapting to characteristics of SCM. We propose a general-purpose SCM-access methodology for persistent data on universal storage to hide such semantics difference. Specifically, we design the Persistent Memory Server Engine (PMSE), a special user-level process based on a unified optimization and transactional mechanism, that provides a group of general calls related to allocation, optimization, transactions of persistent memory and a fine-grained data abstraction for persisting data. Our prototype evaluation of PMSE shows that, compared to several state-of-the-art solutions at the file-system and persistent-heap levels, PMSE can provide 4× better performance than kernel-based persistent-memory file system, 1.7× and 1.5× better performance than user-level persistent-memory file system and persistent heap, under the write-intensive workloads. For traditional applications, applications with IOs achieve similar performance to those without IOs.

Parity-based RAID storage systems have been widely deployed in production environments. However, they suffer from poor random write performance due to the parity update overhead, i.e., small write problem. With the increasing density and decreasing price, SSD-based caching offers promising opportunities for improving RAID storage I/O performance. However, as a cache device, frequent writes to SSD leads to being quickly worn out, which causes high costs and reliability problems. In this paper, we propose an efficient cache management scheme by Keeping Data and Deltas (KDD) in SSD. KDD dynamically partitions the cache space into Data Zone (DAZ) and Delta Zone (DEZ). DAZ stores data that are first admitted into SSD. On write hits, KDD writes the data to RAID storage without updating the parity blocks. Meanwhile, the deltas between old version of data and the currently accessed data are compactly stored in DEZ. In addition, KDD organizes the metadata partition on SSD as a circular log to make the cache persistent with low overhead. We evaluate the performance of KDD via both simulations and prototype implementations. Results show that KDD effectively reduces the small write penalty while significantly improving the lifetime of the SSD-based cache.

The noises at the power lines limit the charge pump to provide large instantaneous current to PCM cells, which results in the number of bits can be written concurrently, i.e. the size of write unit, is restricted in PCM. When implementing PCM as the main memory, the inequality of cache line's size and write unit's size may result in many consecutive executed write units, which greatly decreases the system performance. Existing PCM write schemes, however, consider the worst power and time cases of written data, and ignore the actual current consumption. It is assumed that all data bits are changed and the electric current of each data unit is under fully utilized. The write performance is blocked due to pessimistic estimates, i.e. the current is often excessively supplied but is not used effectively, which leads to huge energy consumption. As a result, the write parallelism is limited and therefore restricts the overall system performance. To address this problem, this paper proposes a novel PCM write scheme named Tetris Write to explore more write parallelism and reduce the critical number of write units in PCM chip. The key idea behind Tetris Write is to monitor the number of '1' and '0' changed in each data unit, and schedule the order of data units' write-1 and write-0 execution considering not only the time and power asymmetries, but also the number asymmetry between RET and SET operations, to allow a larger number of concurrent bit-writes and make the best use of power supply. Tetris Write tries to schedule the dominating long term write-1s first and attempts to steal interspaces remained by write-1s to put the extraessential short write-0s. 4-core PARSEC benchmarks' results show that Tetris Write can get 65% read latency reduction, 40% write latency reduction, 46% running time reduction and 2X IPC improvement compared with the baseline on average. In addition, Tetris Write earns 26%, 15% and 10% more read latency reduction, 15%, 7% and 5% more write latency reduction, and outperforms 22%, 12% and 7% more running time reduction, compared with the state-of-the-art Flip-N-Write, 2-Stage-Write and Three-Stage-Write schemes, whose IPC improvements are 1.4X, 1.6X and 1.8X, respectively.

With the rapid growth of application migration, the anonymity in data center networks becomes important in breaking attack chains and guaranteeing user privacy. However, existing anonymity systems are designed for the Internet environment, which suffer from high computational and network resource consumption and deliver low performance, thus failing to be directly deployed in data centers. In order to address this problem, this paper proposes an efficient and easily deployed anonymity scheme for SDN-based data centers, called MIC. The main idea behind MIC is to conceal the communication participants by modifying the source/destination addresses (such as MAC, IP and port) at switch nodes, so as to achieve anonymity. Compared with the traditional overlay-based approaches, our in-network scheme has shorter transmission paths and less intermediate operations, thus achieving higher performance with less overhead. We also propose a collision avoidance mechanism to ensure the correctness of routing, and two mechanisms to enhance the traffic-analysis resistance. Our security analysis demonstrates that MIC ensures unlinkability and improves traffic-analysis resistance. Our experiments show that MIC has extremely low overhead compared with the base-line TCP (or SSL), e.g., less than 1% overhead in terms of throughput.

With the rapid growth of data, communication overhead has become an important concern in applications of data centers and cloud computing. However, existing distributed graph-processing frameworks routinely suffer from high communication costs, leading to very long waiting times experienced by users for the graph-computing results. In order to address this problem, we propose a new computation model with low communication costs, called LCC-BSP. We use this model to design and implement a high-performance distributed graph-processing framework called LCC-Graph. This framework eliminates the high communication costs in existing distributed graph-processing frameworks. Moreover, LCC-Graph also minimizes the computation workload of each vertex, significantly reducing the computation time for each superstep. Evaluation of LCC-Graph on a 32-node cluster, driven by real-world graph datasets, shows that it significantly outperforms existing distributed graph-processing frameworks in terms of runtime, particularly when the system is supported by a high-bandwidth network. For example, LCC-Graph achieves an order of magnitude performance improvement over GPS and GraphLab.

Continuously enriched distributed systems in data centers generate much network traffic in push-style one-to-many group mode, raising new requirements for multicast transport in terms of efficiency and robustness. Existing reliable multicast solutions, which suffer from low robustness and inefficiency in either host-side protocols or multicast routing, are not suitable for data centers. In order to address the problems of inefficiency and low robustness, we present a sender-initiated, efficient, congestion-aware and robust reliable multicast solution mainly for small groups in SDN-based data centers, called MCTCP. The main idea behind MCTCP is to manage the multicast groups in a centralized manner, and reactively schedule multicast flows to active and low-utilized links, by extending TCP as the host-side protocol and managing multicast groups in the SDN-controller. The multicast spanning trees are calculated and adjusted according to the network status to perform a better allocation of resources. Our experiments show that, MCTCP can dynamically bypass the congested and failing links, achieving high efficiency and robustness. As a result, MCTCP outperforms the state-of-the-art reliable multicast schemes. Moreover, MCTCP improves the performance of data replication in HDFS compared with the original and TCP-SMO based ones, e.g., achieves 101% and 50% improvements in terms of bandwidth, respectively.

NAND flash memory has been widely used as storage medium in diverse environments due to its light weight, low power consumption, and high I/O performance. Page replacement is an important operation in NAND flash- based storage systems. However, traditional replacement algorithms designed for magnetic disks fail to meet the needs of NAND flash memory due to inherent features such as asymmetric I/O latencies and erase-before- write. In order to address this problem, this paper proposes a new page replacement algorithm, called Ghost buffer Assisted and Self-tuning Algorithm (GASA). GASA reduces expensive flash write operations by evicting cold clean pages preferentially and maintains reasonable buffer hit ratios via a ghost buffer. In addition, GASA is self-tuning due to the use of a simple learning scheme, thus adaptively matching different workloads and buffer sizes. Experimental results based on real-world OLTP traces demonstrate that GASA offers a good trade-off between the hit ratio and the flash write count, thus achieving better I/O performance than the state-of-the-art page replacement algorithms.

Phase Change Memory (PCM) is one of the promising memory technologies but suffers from some critical problems such as poor write performance and high write energy consumption. Due to the high write energy consumption and limited power supply, the size of concurrent bit-write is restricted inside one PCM chip. Typically, the size of concurrent bit-write is much less than the cache line size and it is normal that many serially executed write units are consumed to write down the data block to PCM when using it as the main memory. Existing state-of-the-art PCM write schemes, such as FNW (Flip-N-Write) and two-stage-write, address the problem of poor performance by improving the write parallelism under the power constraints. The parallelism is obtained via reducing the data amount and leveraging power as well as time asymmetries, respectively. However, due to the extremely pessimistic assumptions of current utilization (FNW) and optimistic assumptions of asymmetries (two-stage-write), these schemes fail to maximize the power supply utilization and hence improve the write parallelism. In this article, we propose a novel PCM write scheme, called MaxPB (Maximize the Power Budget utilization) to maximize the power budget utilization with minimum changes about the circuits design. MaxPB is a “think before acting” method. The main idea of MaxPB is to monitor the actual power needs of all data units first and then effectively package them into the least number of write units under the power constraints. Experimental results show the efficiency and performance improvements on MaxPB. For example, four-core PARSEC and SPEC experimental results show that MaxPB gets 32.0% and 20.3% more read latency reduction, 26.5% and 16.1% more write latency reduction, 24.3% and 15.6% more running time decrease, 1.32× and 0.92× more speedup, as well as 30.6% and 18.4% more energy consumption reduction on average compared with the state-of-the-art FNW and two-stage-write write schemes, respectively.

We propose Rotated Logging (RoLo), a new logging architecture for parallel disk-based mirrored storage systems for enhanced energy efficiency, which is one of the key concerns in modern data centers. By spreading destaging I/O activities among short idle time slots and proactively reclaiming the stale logging space, RoLo rotates loggers among a logical logging space pool formed collectively from the free storage space available among mirrored disks. We develop three flavors of RoLo, that is, RoLo-P/R/E, to emphasize performance, reliability, and energy efficiency respectively. Without the extra dedicated log disks and the corresponding centralized destaging, RoLo eliminates the additional hardware and energy costs, potential single point of failure and performance bottleneck. Furthermore, RoLo-P/R/E, applied to specific scenes correctly, can prolong the lifecycle of the disks and improve the system's energy efficiency by reducing the disk spin up/down frequency. We propose RoLo-S to further alleviate the performance bottleneck and energy consumption caused by frequent disk head seeks in on-duty logger disks. We have implemented RoLo and RoLo-S on real disk systems. Extensive trace-driven evaluations demonstrate the advantages of the three RoLo schemes over both a RAID10 system with centralized logging architecture and a typical RAID10 system, and the advantages of RoLo-S over RoLo.

Modern solid state disk (SSD) has a buffer (SDRAM), which is used to store commonly used data and map in the near future. How to efficient management of this buffer is an important things of improving performance of SSD. Flash read and write speed have asymmetric characteristic. SSD buffer management algorithms must consider this characteristic of flash. Current page mapping SSD buffer management algorithms mainly use the Clean-First LRU (CFLRU) algorithm to first replace the clean buffer pages regardless of whether these pages will soon be used in the near future. At the same time, LRU buffer management algorithm of SSD does not consider file scanning. In order to solve these problems, we proposes a new SSD internal buffer management algorithm, called Two Queue Weight-Clock (2QW-Clock). This algorithm combines the advantages of 2Q and gives different weights to read page and write page to reflect the asymmetry of flash read and write speed. Therefore, it can get high write page hit ratios while maintaining high total page hit ratios. With the high write ratios, 2QW-Clock reduces the numbers of SSD write and erase operations. So it can greatly extend the life of the SSD. Conducting simulations with a variety of traces and a wide range of buffer sizes, we show that 2QW-Clock write hit ratios are significantly higher than CFLRU, LRU and 2Q in most cases while total hit ratios are almost as the 2Q. Simulation result shows that the numbers of 2QW-Clock write and erase counts reduced by up to 30% less than that of 2Q and CFLRU.

Due to the high complexity in software hierarchy and the shared queue & lock mechanism for synchronized access, existing I/O stack for remote target access in FCoE-based SAN storage becomes a performance bottleneck, thus leading to a high I/O overhead and limited I/O scalability in multi-core servers. For scalable performance, existing works focus on improving the efficiency of lock algorithm or reducing the number of synchronization points to decrease the synchronization overhead. However, the synchronization problem still exists and leads to a limited I/O scalability. In this paper, we propose Fast FCoE, a protocol stack framework for remote storage access in FCoE based SAN storage. Fast FCoE uses private per-CPU structures and disables the kernel preemption to process I/Os. This method avoids the synchronization overhead. For further I/O efficiency, Fast FCoE directly maps the requests from the block-layer to the FCoE frames. A salient feature of Fast FCoE is using the standard interfaces, thus supporting all upper softwares (such as existing file systems and applications) and offering flexible use in existing infrastructure (e.g., Adaptors, switches, storage devices). Our results demonstrate that Fast FCoE achieves efficient and scalable I/O throughput, obtaining 1107.3K/831.3K IOPS (5.43/4.88 times as much as Open-FCoE stack) for read/write requests.

Compared with either block or page-mapping Flash Translation Layer (FTL), hybrid-mapping FTL for flash Solid State Disks (SSDs), such as Fully Associative Section Translation (FAST), has relatively high space efficiency because of its smaller mapping table than the latter and higher flexibility than the former. As a result, hybrid-mapping FTL has become the most commonly used scheme in SSDs. But the hybrid-mapping FTL incurs a large number of costly full-merge operations. Thus, a critical challenge to hybrid-mapping FTL is how to reduce the cost of full-merge operations and improve partial merge operations and switch operations. In this article, we propose a novel FTL scheme, called Virtual Block-based Parallel FAST (VBP-FAST), that divides flash area into Virtual Blocks (VBlocks) and Physical Blocks (PBlocks) where VBlocks are used to fully exploit channel-level, die-level, and plane-level parallelism of flash. Leveraging these three levels of parallelism, the cost of full merge in VBP-FAST is significantly reduced from that of FAST. In the meantime, VBP-FAST uses PBlocks to retain the advantages of partial merge and switch operations. Our extensive trace-driven simulation results show that VBP-FAST speeds up FAST by a factor of 5.3--8.4 for random workloads and of 1.7 for sequential workloads with channel-level, die-level, and plane-level parallelism of 8, 2, and 2 (i.e., eight channels, two dies, and two planes).

Traditional analysis on the TCP performance attributes the overhead of TCP mainly on these aspects: many times of memory access and copy, much data transmission on bus and a large number of CPU cycles required by the protocol processing. Compared to TCP, the main advantage of RDMA lies in its single memory access which brings a higher data transmission bandwidth and lower CPU utilization. However, the hardware platforms that these analyses are based on are already out-of-date from today's view. The key point is that TCP and RDMA performance can change with the rapid development of the computer architecture and computer hardware components. In this paper, we re-analysis the performance of TCP and RDMA on the modern server platform (e.g., multi-core), and we find that the upper limit of TCP transmission bandwidth has been greatly upgraded compared to the old platform. Besides, the compatibility and complexity problems in programming brought by RDMA technology are still serious. Based on this conclusion, we constructed a network platform which uses multi-thread technology and TCP protocol to achieve a high speed transmission bandwidth in the InfiniBand network environment.

We propose RoLo (Rotated Logging), a new logging architecture for RAID10 systems for enhanced energy efficiency, performance and reliability. By spreading destaging I/O activities among short idle time slots and proactively reclaiming the stale logging space, RoLo rotates loggers among a logical logging space pool formed collectively from the free storage space available among mirrored disks. Therefore, without the extra dedicated log disks and the corresponding centralized logging, RoLo eliminates the additional hardware and energy costs, potential single point of failure and performance bottleneck. Furthermore, RoLo prolongs the lifecycle of the disks and improves the system's energy efficiency by reducing the disk spin up/down frequency. We develop three flavors of RoLo, that is, RoLo-E/R/P, to emphasize energy efficiency, reliability, and performance respectively. Extensive trace-driven evaluations demonstrate the advantages of the three RoLo schemes over both a RAID10 system with centralized logging architecture and a typical RAID10 system.

File correlation, which refers to a relationship among related files that can manifest in the form of their common access locality (temporal and/or spatial), has become an increasingly important consideration for performance enhancement in peta-scale storage systems. Previous studies on file correlations mainly concern with two aspects of files: file access sequence and semantic attribute. Based on mining with regard to these two aspects of file systems, various strategies have been proposed to optimize the overall system performance. Unfortunately, all of these studies consider either file access sequences or semantic attribute information separately and in isolation, thus unable to accurately and effectively mine file correlations, especially in large-scale distributed storage systems.

Geographic Information System Grid integrates geographic information systems and Grid technology for data gathering, accessing, transmitting and service, in different I/O patterns, built upon massive storage systems. Existing non-standardized multi-source and multi-scale data lack spatial information sharing either internally or externally between organizations or departments, especially in national or global applications. HUSt is a massive storage system that was built at Wuhan National Laboratory for Optoelectronics, in China. There are heterogeneous storage areas in the system, including Object-based Storage System for the main data storing especially for the data searched frequently, Virtual Interface based Storage System for the data required at high transfer speed, and InfiniBand based SAN for high performance. HUSt is primarily meant for research on the organization and key technologies of storage systems for the next generation Internet. The goal is to unify network storage and construct a peta-byte storage system, which supports GIS Grid and applications.

Traditional storage systems, such as NAS, SAN, are largely unaware of the users and applications actually using the storage, because block-based storage devices manage opaque data blocks. But, with OBSS (object-based storage system), the attributes and methods among the storage devices can be adopted in the storage system, the data can be distributed on some of the storage devices and organized better to anticipate users demand. In this paper, the scalability of object storage (including object attributes, object methods and OBSS) is studied. And a self-managing approach, denoted adaptive policy trigger mechanism (APTM), is presented. APTM borrows proven machine learning techniques and takes the perspective scalable object storage. The implementation reveals that APTM is the embodiment of the idea about smart storage device and facilitates to self-manage mass storage system.