Memristive in-situ computing offers energy-efficient DNN acceleration, but faces ADC-induced energy bottlenecks. We observe that bitline outputs exhibit significant non-uniformity and cycle-to-cycle variation, rendering conventional A/D conversion schemes suboptimal. We thus propose a data distribution-aware A/D conversion strategy that predicts key bits of digital outputs and skips unnecessary steps, with a switching mechanism adapting the optimal conversion method across cycles. Implemented via a reconfigurable SAR-ADC, our approach significantly reduces the energy consumption of in-situ accelerators.

Memristive accelerators (MAs) facilitate efficient matrix-vector multiplication (MVM) by performing in situ computation within memory crossbar arrays, thereby ensuring a fast and energy-efficient application acceleration. A significant challenge associated with the MA lies in the limited computing accuracy caused by the IR drop effect. However, existing IR drop mitigation works provide an approximate compensation, resulting in less accurate results. In this paper, we propose an IR drop-robust mapping method for reliable memristive accelerators. Firstly, the IR drop-robust mapping (IRM) method exploits the residuals between the equivalent matrix after the IR drop effect and the original matrix, and iteratively maps them to the crossbars for IR drop compensation. Based on the IRM method, a novel mechanism of the matrix-vector multiplication (MVM) operation is derived, ensuring that MVM is computed correctly. Secondly, the Calibrate-Shift-Reflect (CSR) strategy is developed to significantly reduce the number of arrays required by the IRM method to map the residuals. Thirdly, the hardware support for the IRM method is designed, and the overhead is reduced by sharing drivers/selectors between neighboring arrays. The experimental results indicate that the IRM-CSR method can effectively mitigate the IR drop effect, restoring inference accuracy by at most 80% (for neural network applications), and achieving a reduction in the relative root-mean-squared error by 103×~1010× (for scientific computing), compared with the state-of-the-art methods.

The management of mapping tables within solid-state drives (SSDs) is critical to maintaining high performance and stability. Traditional fixed-granularity mapping approaches (page-level, block-level) face significant challenges in ultra-large-scale 3D NAND flash memory, including high DRAM cache overhead resulting from page mapping and high write amplification caused by block mapping. Existing hybrid mapping solutions fail to adapt to the dynamic-varying workloads and large-capacity SSDs, resulting in unstable performance and poor scalability. In this paper, we propose a novel light-weight learning-based combination mapping technique called CoMap, which can adaptively choose the optimal mapping scheme so as to effectively prevent excessive page-mapping table sizes and poor block mapping performance in large-capacity SSDs. Comprehensive evaluations with variety of real-world workloads and enterprise-class SSD simulator show that CoMap, with its strong adaptability, can reduce 60% of latency and 30% of write amplification on average compared with several state-of-the-art methods.

The growing adoption of prediction serving systems (PSSes) has made cost-saving deployment on preemptible spot instances crucial, yet frequent preemptions severely harm availability. While coded computation (CC) can keep availability cost-effectively by encoding original jobs into parity ones, its direct application to spot instances incurs prohibitive decoding overhead and tail latency under frequent preemptions. We identify two findings for optimization: (i) decoding asymmetry (only original job failures require decoding); (ii) preemption unevenness (variation in preemption rates across cloud regions). Leveraging these findings, we propose SpotCC, a new CC framework that strategically dispatches parity jobs to high-preemption (volatile) regions and original jobs to low-preemption (stable) regions. SpotCC designs locality-based and fine-grained volatility identification to reduce decoding operations and mitigate job congestion, respectively, and adaptively tunes configurations for decoding minimization. Experiments show that SpotCC improves P99 latency by 83.9% over state-of-the-arts, while maintaining ultra-low monetary costs.

Identifying intrusions from massive logs has long been a great challenge. To address this issue, this paper proposes Pontus, a novel host-based intrusion detection method via accurate provenance clustering, efficient graph serialization and classification with minimum provenance loss. Pontus first utilizes a novel multi-round label propagation algorithm (MLPA) based on overlapping community discovery to cluster the behavior instances that constitute user behavior accurately. In this way, Pontus can analyze behavior instances to extract behavior features effectively while reducing the analysis workload. Then, Pontus enables efficient graph serialization via neighbor node aggregation to convert the behavior instance into vectors while maximizing the retention of provenance information. Finally, Pontus uses a hybrid method that combines the convolutional autoencoder with Bisecting Kmeans clustering to accurately extract the provenance features of behavior instances to identify host-based intrusions. The experimental results show that compared with the state-of-the-art methods, Pontus’s accuracy increases by an average of 0.243, and F1-score increases by an average of 0.201, with small runtime overheads.

Abstract Social network alignment (SNA) aims to match corresponding users across different platforms, playing a critical role in cross-platform behavior analysis, personalized recommendations, security, and privacy protection. Traditional methods based on attribute and structural features face significant challenges due to the sparsity, heterogeneity, and dynamic nature of social networks, resulting in limited accuracy and efficiency. Recent advances in graph representation learning (GRL) provide promising solutions to these issues by leveraging deep learning to extract network features, effectively addressing sparsity, integrating heterogeneous data, and adapting to network dynamics. This paper presents a comprehensive survey of SNA methods based on GRL. We first introduce key definitions and outline a framework for SNA using GRL. Next, we systematically review state-of-the-art advancements in both static and dynamic networks, considering homogeneous and heterogeneous settings, including emerging approaches integrating large language models (LLMs). We further conduct an in-depth comparative analysis, highlighting the effectiveness of different GRL-based methods, with a particular emphasis on LLM-enhanced techniques. Finally, we discuss open challenges and outline potential future research directions in this rapidly evolving field.

B+-Trees are widely used in storage systems and diverse applications. However, their performance is hindered by frequent memory accesses required for key comparisons and structural maintenance. The limited parallelism of CPUs, along with their sensitivity to data order and volume, further restricts the efficiency of B+-Trees. Processing-in-memory (PIM) offers a promising alternative with its in-situ parallel computing capability. In this paper, we propose pTree, a novel set of PIM techniques and architecture specifically designed to accelerate B+-Tree operations. pTree introduces in-situ parallel comparison mechanisms that significantly reduce costly memory accesses and inefficient CPU-side comparisons during tree traversal. This parallelism eliminates the need to maintain intra-node key order and, together with our in-situ parallel node bisecting techniques, greatly minimizes tree structure maintenance overhead during insertions and deletions. Additionally, pTree decouples both comparison and modification efficiency from node size, enabling the use of larger nodes to reduce tree height and traversal complexity. Evaluation shows that pTree reduces the average latency to 35%, 26%, 52%, and 32% compared to state-of-theart B+-Trees forInsert, Search, Update,andDeleteoperations.

Minimum-storage regenerating (MSR) codes are provably optimal erasure codes that minimize the repair bandwidth (i.e., the amount of traffic being transferred during a repair operation), while minimizing storage redundancy, in distributed storage systems. However, the practical repair performance of MSR codes still has significant room for improvements, as their mathematical structure makes repair operations difficult to parallelize. In this article, we present HyperParaRC, a parallel repair framework for MSR codes. HyperParaRC leverages the sub-packetization nature of MSR codes to parallelize the repair of sub-blocks and balance repair load (i.e., the amount of traffic sent or received by a node) across available nodes. We first demonstrate that there exists a trade-off between repair bandwidth and maximum repair load. We then propose an affinity-based heuristic for HyperParaRC, which approximately minimizes the maximum repair load by examining the bandwidth incurred during sub-block computations and significantly reduces the search time for large coding parameters compared with our earlier work, ParaRC. Based on our affinity-based heuristic, we further design a full-node recovery mechanism for HyperParaRC that combines both intra-stripe and inter-stripe parallel repair scheduling to repair multiple lost blocks in a failed node. We prototype HyperParaRC on Hadoop HDFS and evaluate it on Alibaba Cloud. Our evaluation results show that HyperParaRC reduces both single-block repair and full-node recovery times compared with state-of-the-art repair approaches.

The growing volume of performance-critical parameters in distributed storage systems, coupled with diverse and dynamic workload patterns, has significantly increased the complexity of system configuration. These trends have expanded the parameter space while tightening the time window for tuning convergence, making it challenging to maintain high system performance. Existing tuning strategies often struggle to balance thorough parameter exploration with real-time responsiveness, limiting their effectiveness under fast-evolving workloads and heterogeneous deployment environments. To address these challenges, we propose KGQW, the first framework that formulates automated parameter tuning as a knowledge graph query workflow. KGQW models workload features and system parameters as graph vertices, with performance metrics represented as edges, and constructs an initial knowledge graph through lightweight performance tests. Guided by performance prediction and Bayesian-driven exploration, KGQW progressively expands the graph, prunes insensitive parameters, and refines performance relationships to build an informative and reusable knowledge graph that supports rapid configuration retrieval via graph querying. Moreover, KGQW enables efficient knowledge transfer across clusters, substantially reducing the construction cost for new clusters. Experiments on real-world applications and storage clusters demonstrate that KGQW achieves second-level tuning latency, while maintaining or surpassing the performance of state-of-the-art methods. These results highlight the promise of knowledge-driven tuning in meeting the scalability and adaptability demands of modern distributed storage systems.

Data centers commonly use erasure codes to maintain high data reliability with lower storage overhead than replication. However, recycling invalid data blocks caused by deletion and update operations is challenging in erasure-coded data centers. Erasure codes organize data blocks into stripes, and we cannot directly delete invalid data blocks like replication to ensure the redundancy of the remaining valid blocks within a stripe. When considering the recycling issues in data centers, existing studies still need to address the following problems: ignoring heavy cross-rack traffic and the load imbalance problem during recycling, and incurring high disk seeks that affect writing performance after recycling. This paper presents the first systematic study on data re cycling in erasure-coded data centers and proposes a Cross rack Aware Recycle (CARecycle) technique. The key idea is migrating valid data blocks from certain stripes to rewrite invalid ones in others, thereby releasing the invalid blocks for certain stripes. Specifically, CARecycle first carefully examines the block distribution for each stripe and generates an efficient recycle solution for migrating and releasing, with the primary objective of reducing cross-rack traffic and disk seek load of nodes. Due to the rewriting of invalid data blocks, parity blocks in multiple stripes need to be updated concurrently. Thus, it further batch processes multiple stripes and selectively arranges appropriate stripes into a batch to achieve uniform cross-rack traffic load distribution. In addition, CARecycle can be extended to adapt to different erasure codes and boost recycling in heterogeneous network environments. Large-scale simulations and Amazon EC2 experiments show that CARecycle can reduce up to 33.8% cross rack traffic and 28.64%-59.64% recycle time while incurring low disk seek, compared to a state-of-the-art recycling technique.

For a long time, fully exploiting the multi-level parallelism of SSDs has been an important technique for optimizing SSD performance. By distributing data to different flash parallel units through address mapping policies, SSDs can achieve multi-task parallel processing, thereby significantly improving data read and write performance. However, our experiments show that when SSD blocking causes task queues to stack up, excessive pursuit of flash parallelism can cause performance interference between multiple users, thereby reducing SSD performance. Additionally, in DRAM-less SSDs, differences in data access patterns directly impact the address mapping module, thereby exacerbating the degree of performance degradation. Based on the above analysis, this paper conducts a systematic study of the impact mechanism of different data flows on address mapping policies and proposes ahybrid addressmapping allocationpolicy with dynamic switching (HMpp) for DRAM-less SSDs. This policy identifies and classifies different data access patterns to dynamically select the appropriate address mapping policy, thereby reducing multi-user interference and improving SSD performance. Experimental results based on real cloud block storage workloads show that this policy can achieve 87.2% performance improvement and 77.3% fairness optimization, while reducing the write amplification factor by 32.9% to 40.2%.

To improve storage efficiency in large-scale clustered storage systems, deduplication that removes duplicate chunks has been widely deployed in distributed ways. Many distributed deduplication-related studies focus on backup storage, and some recent studies focus on deploying deduplication in clustered primary storage systems which store active data. While fragmentation is one of the traditional challenges in backup deduplication, we observe that a new fragmentation problem arises when performing deduplication in the clustered primary storage system due to the system's concurrent file writes. However, we find that existing state-of-the-art methods that address traditional fragmentation in backup deduplication fail to work effectively for the new fragmentation problem, as they significantly incur additional redundancy or lower the deduplication ratio. In this paper, we revisit fragmentation-solving methods in memory management and our main idea is inspired by the classic garbage collection methods in memory management: relocating fragments consecutively. Based on the idea, we propose an effective deduplication mechanism for clustered primary storage systems, ReoDedup, which applies: i) a cosine-similarity based chunk relocating algorithm that aims to minimize the fragmentation; ii) an adjacency-table based relocating heuristic that reduces the relocating's time complexity by placing two chunks residing in the same file consecutively; and iii) an indexremapping update scheme that alleviates the extra fragmentation caused by updates. We implement ReoDedup atop Ceph and our cloud experiments show that the average read throughput of ReoDedup can be increased by up to$1.72 \times$over state-of-thearts, without any deduplication ratio loss.

NAND flash-based SSDs have emerged as a critical storage solution due to their exceptional performance and cost-effectiveness. However, the sequential write limitation of NAND flash blocks necessitates garbage collection (GC) to reclaim space occupied by stale data. Nevertheless, the extensive data migration involved in GC significantly impacts performance and Quality of Service (QoS) of SSDs. To mitigate this issue, copyback has been proposed as a means to accelerate GC by eliminating off-chip data movements. Specifically, copyback reads data into on-plane latches and immidiately re-writes it into another page on the same plane. However, in the case of modern high-performance SSDs, copyback is rarely utilized due to the following challenges: (1) Copyback operates at the page-level and thus fails to effectively reclaim invalid data within the context of subpage-level mapping; (2) Copyback eliminates off-chip data movements, preventing data pages from being read out for Redundant Array of Independent NAND (RAIN) parity computation, thereby compromising SSD reliability. In this study, we introduce SuperCopyback as a solution that efficiently addresses these issues for modern SSDs. Firstly, we propose a Multiple-Read-One-Write (MROW) copyback approach through lightweight latch circuit modifications to enable subpage-level copyback implementation; additionally, we propose an orchestrated GC method to effectively utilize MROW copyback. Furthermore, we present a novel copyback-based RAIN scheme that conceals data pages readout latency in write operations and relocates parities to support efficient copyback. The experimental results on both synthetic and real traces demonstrate that SuperCopyback achieves a performance comparable to an ideal scenario where data movement of $\mathbf{4 K B}$ takes only 1ns.

With integrated circuits shrinking in feature size, layout printability has become increasingly challenging, making lithographic hotspot detection ever-crucial in computer-aided design (CAD) flows. In recent years, numerous studies have explored deep learning to detect lithographic hotspots, offering promising results. However, neural networks can easily be biased and overfit when lacking sufficient training data, especially in the CAD domain. A generalizable DL-based hotspot detector should learn the genuine lithography principle and ensure consistent accuracy across layouts from various designs at the same technology node, regardless of their varying design styles. However, we find that existing convolutional neural network (CNN)-based hotspot detectors fail to generalize to different circuit layouts other than the design it has been trained for. To this end, we propose a few-shot learning-based framework for generalizable CNN-based hotspot detection. We develop a meta-learning scheme that asynchronously updates the CNN feature extraction and classification component to obtain a metainitialized model that can quickly adapt to new designs using as few as one training layout clip. We propose a layout topologybased sampling strategy for few-shot adaptation to enhance generalization stability. Experimental results on ICCAD 2012 and 2019 datasets show that our framework enables superior generalization capabilities than prior arts on unseen new designs.

11This work was supported by the Young Scientists Fund of the National Natural Science Foundation of China under Grant 62302182.Persistent memory (PM) leveraging memory-mapped I/O(MMIO) delivers superior I/O performance, leading to the development of user-space PM file systems based on MMIO. While effective in single-process scenarios, these systems encounter challenges in multi-process environments, such as performance degradation due to repeated page faults and cross-process synchronizations, as well as a large memory footprint from duplicated paging structures. To address these problems, we propose a Multi-process PM File System (MPFS). MPFS builds a shareable page table and shares it among processes, avoiding building duplicate paging structures for distinct processes, thereby significantly reducing the software overhead and memory footprint caused by repeated page faults. MPFS further proposes a PGD-aligned (512GB) mapping method to accelerate page table sharing. Furthermore, MPFS provides a cross-process memory protection mechanism based on the PGD-aligned mapping, ensuring multi-process data reliability with negligible overheads. The experimental results show that MPFS outperforms existing user-space PM file systems by 1560% in multi-process scenarios.

With integrated circuits at advanced technology nodes shrinking in feature size, lithographic hotspot detection has become increasingly important. Deep learning, especially convolutional neural networks (CNNs) and graph neural networks (GNNs) have recently succeeded in lithographic hotspot detection, where layout patterns, represented as images or graph features, are classified into hotspots and non-hotspots. However, with increasingly sophisticated CNN architectural designs, CNN-based hotspot detection requires excessive training and inference costs with expanding model sizes but only marginally improves detection accuracy. Existing GNN-based hotspot detector requires more intuitive and efficient layout graph feature representation. Driven by the understanding that lithographic hotspots result from complex interactions among metal polygons through the light system, we propose the absolute and relative locations of metal polygons are all we need to detect hotspots of a layout clip. We propose a novel layout graph feature representation for hotspot detection where the coordinates of each polygon and the distances between them are taken as node and edge features, respectively. We design an advanced GNN architecture using graph attention and different feature update functions for different edge types of polygons. Our experimental results demonstrate that our GNN hotspot detector achieves the highest hotspot accuracy and the lowest false alarm on different datasets. Notably, we employ one-third of the graph features of the previous GNN hotspot detector and achieve higher accuracy. We outperform all CNN hotspot detectors with higher accuracy, up to 32× speed up in inference time, and 64× reduction in model size.

A hashing index provides rapid search performance by swiftly locating key-value items. Non-volatile memory (NVM) technologies have driven research into hashing indexes for NVM, combining hard disk persistence with DRAM-level performance. Nevertheless, current NVM-based hashing indexes must tackle data inconsistency challenges caused by NVM write reordering or partial writes, and mitigate rapid local wear due to frequent updates, considering NVM's limited endurance. The temporary allocation of buckets in NVM-based chained hashing to resolve hash collisions prolongs the critical path for writing, thus hampering write performance. This paper presents WOPHI, a write-optimized persistent hash index scheme for NVM. By utilizing log-free failure-atomic writes, WOPHI minimizes data consistency overhead and addresses hash conflicts with bucket pre-allocation. Experimental results underscore WOPHI's significant performance enhancements, with insertion latency slashed by up to 88.2% and deletion latency boosted by up to 82.6% compared to existing state-of-the-art schemes. Moreover, WOPHI substantially mitigates data consistency overhead, reducing cache line flushes by 59.3%, while maintaining robust write throughput for insert and delete operations.

Distributed deep neural networks (DNN) training systems deployed across workers have been widely used in various domains, while the communication overhead among workers for synchronizing gradient tensors often becomes the performance bottleneck. To optimize communication efficiency, state-of-the-art studies often apply both two techniques: i) gradient sparsification compression, which truncates the gradient to its largest elements to reduce the communication traffic, and ii) tensor fusion, which merges multiple gradient tensors within a fusion buffer to transmit them together to reduce the communication startup overhead. However, we find that existing studies often apply gradient sparsification after tensor fusion (we call sparsification-behind tensor fusion), which leads to a fact that a lot of fused gradient tensors are missed after the sparsification, thus impairing the convergence performance.

Persistent memory (PM) offers a compelling combination of durability and near-DRAM performance, but it also introduces new challenges for hashing indexes. Existing persistent hashing designs prioritize resizing efficiency at the expense of increased query latency, losing the key advantage of hash tables. This paper introduces$\mathbf{R}^{2}$Hash, a persistent hashing index redesigned from the persistent cache-line hash table, to balance high read performance with efficient resizing.$R^{2}$Hash is guided by a migration rule, enabling it to meet both goals through two main contributions: (i) a cooperative and lowoverhead resizing strategy based on split-order hashing, and (ii) a shift-aware search combined with a two-layer bucket layout that enables lock-free reads with only one PM access on average. Furthermore,$\mathbf{R}^{2}$Hash provides log-free consistency and a nonblocking recovery mechanism. Experimental results demonstrate that$\mathbf{R}^{\mathbf{2}}$Hash achieves up to$8.1 \times$higher search throughput and$7.5 \times$higher insert throughput compared to other persistent hash indexes across a range of workloads.

Although Federated Learning (FL) enables distributed clients to cooperatively train deep learning models without sharing local data, the iterative FL training process imposes considerable computation and communication overheads on clients. Especially in cloud-edge collaboration situations, heterogeneous and resource-limited edge clients can become a bottleneck for FL. In this paper, we propose FedLTH (Federated Learning with the Lottery Ticket Hypothesis), an FL framework based on the Lottery Ticket Hypothesis and adaptive differential privacy, which aims to improve communication and computing efficiency and privacy security for resource-limited edge clients. First, the pruning rate of each client is set according to their respective resource constraints. The server divides the clients into groups with balanced data distribution and similar pruning rates to ensure the convergence of the global model. Then, a structured model pruning method based on the Lottery Ticket Hypothesis is introduced. Each client group participates in a pruning phase to reduce the computing overhead of clients. Last, an adaptive differential privacy algorithm is designed to preserve client data privacy and improve model accuracy. Through experiments on multiple datasets and non-IID scenarios, we show the effectiveness of FedLTH in privacy preservation and reducing computation and communication overheads.

Graph embedding, which maps graph nodes to lowdimensional vectors, is a widely used technique for graph representation learning. However, most existing graph embedding models suffer from high memory consumption, limiting their scalability to large graphs. Heterogeneous memory systems that combine DRAM and Persistent Memory (PM) offer new opportunities for scaling up memory capacity. Despite this advantage, the performance gap (on the order of 5x) between DRAM and PM is magnified (by 3.3-4.2x) under non-uniform memory access (NUMA) architecture. Additionally, the inherent sparsity of graphs induces numerous random accesses in the fundamental Sparse Matrix and Dense Matrix Multiplication (SpMM) operations of graph embedding, hindering high-performance heterogeneous memory processing. To address these challenges, this paper presents OMeGa that focuses on Optimizing heterogeneous Memory processing for large-scale Graph embedding. OMeGa leverages an entropy-aware thread allocation, simultaneously achieving workload balancing and tail latency reduction across threads. It also incorporates a workload feature-aware prefetcher to alleviate random accesses during streaming heterogeneous processing. In addition, OMeGa devises a NUMA-aware data placement, aiming to minimize the adverse impact of NUMA on heterogeneous memory. The experiments conducted on billion-scale graphs demonstrate that OMeGa exhibits an average acceleration of 32.03x with strong scalability. This pioneering capability enables the efficient generation of large-scale graph embeddings, free from the memory size constraints and performance disparities typically encountered in heterogeneous memory systems.

The learned index technique has been widely explored as a strong competitor to traditional indexes. It adopts static learning-based models to fit the distribution of sorted data and locate keys through predictions, which shows outstanding query speed. However, frequent retraining is required when it comes to concurrent insertion scenarios. Despite existing studies introducing sparse slots and delta buffers to mitigate this effect, the read-write performance of the learned index still falls short of expectations, especially in concurrent conditions. In this paper, we first propose a novel hybrid index scheme that combines a read-efficient learned index with an insert-efficient Adaptive Radix Tree (ART) to realize high performance for read-write scenarios. However, it is not trivial due to expensive model prediction errors, complicated model hierarchy, and redundant node traversals. Therefore, we then introduce ALT-index, an efficient hybrid learned index with high concurrency for memory database systems. ALT-index highlights a delicate two-tier architecture where linear data are stored in the learned index without prediction errors and conflict data are hosted in the lower layer as an optimized ART. Besides, we develop a Greedy Pessimistic Linear (GPL) algorithm to support flattened data structures for concurrency. In the optimized ART layer, we introduce a fast and compact pointer buffer to further improve the overall performance. Experimental results conducted on various real-world datasets with 32 threads illustrate that ALT-index improves performance by up to 1.9x, 2.1x, and 2.3x compared with ALEX+, FINEdex, and XIndex in read-write-balanced scenarios, respectively.

Deep neural network (DNN) training is both data-intensive and compute-intensive. As datasets grow, storing them entirely in memory becomes infeasible, making I/O a major bottleneck—often accounting for 30%-90% of total training time due to the widening gap between data loading and computation speed. While caching can mitigate I/O latency, traditional strategies fail under random sampling. Recent work shows that importance sampling can induce data locality, enabling more effective caching; however, loss-based importance ignores semantic attributes, limiting effectiveness in I/O-bound tasks.

Compared to DRAM, persistent memory (PM) offers higher density and persistence but encounters more severe reliability challenges. Erasure coding is widely adopted to enhance reliability with minimal space overhead. Unfortunately, applying erasure coding to PM introduces significant additional latency. Previous work to mitigate coding latency has primarily focused on optimizing computational efficiency. Instead, we reveal that the main performance bottleneck is high memory latency due to inefficient hardware prefetchers, rather than computation. We further observe that the prefetching inefficiency mainly results from: (i) too wide or narrow coding stripes, (ii) small block sizes, and (iii) high concurrency.

Erasure coding is widely adopted to maintain data reliability, yet it introduces a significant update penalty. We analyze real-world traces and observe several challenges that are not addressed by existing studies, which thereby restricts the performance gains. We propose FastUpdate, an efficient multi-stripe updates framework that assists existing update schemes for fast updates. FastUpdate comprises three key designs: (i) it perceives the update locality and carefully merges multiple update requests accessing the same stripe to reduce the incurred network traffic; (ii) it abstracts the existing update schemes into collector selection and tree construction, greedily generates the update solution for each stripe to balance the transmission load across nodes; (iii) it dynamically schedules appropriate stripes to update in heterogeneous and dynamic networks to fully saturate the bandwidth resources. Comprehensive evaluations verify the effectiveness of FastUpdate on Alibaba ECS. It can increase the update throughput by 16.15%-88.71% for various update schemes.

Abstract As modern attack methods become more concealed and complex, obtaining many labeled samples in big data streams is difficult. Active learning has long been used to achieve better intrusion detection performance by using only a small number of training samples. Intrusion behaviors can be described by provenance graphs that record the dependency relationships between intrusion processes and the infected files. It is a challenge to develop active learning strategies that consider defining and selecting the most valuable provenance and ensure that the strategy for querying provenance is efficient. We present Angus, an active learning framework for provenance-based intrusion detection. We propose two novel active learning strategies: the most similar graph query strategy and the maximum difference query strategy. They either select samples to update the training set according to similarities of provenance graphs or preferentially select samples with low redundancy and large differences from the current training set. Besides, we also improve the above query strategies by using the parallel query to reduce detection time overheads. The experiments on various real-world applications demonstrate their performance and efficiency.

The Zoned Namespace (ZNS) interface transfers most storage maintenance responsibilities from the underlying Solid-State Drives (SSDs) to the host. This shift creates new opportunities to ensure fairness and high performance in multi-tenant cloud computing environments at both hardware and software levels. However, when applications with different workloads share a single ZNS SSD hardware, traditional fair queueing schedulers fail to achieve fairness due to their limited awareness of workload characteristics. Moreover, allowing multiple outstanding requests to access the device simultaneously improves resource utilization but often leads to significant I/O interference among these requests. This interference results in over-throttling, which subsequently degrades the performance of existing fair queueing schedulers. To address the above problems, this article proposes an efficient and high-performance fair queueing scheduling scheme for ZNS SSD (ZNSFQ) on the host side. Firstly, ZNSFQ introduces a workload-aware fair scheduler that enhances fairness by accurately estimating the I/O cost for each application based on its workload characteristics. Secondly, to optimize performance while ensuring fairness, ZNSFQ designs a request dispatch parallelism adjuster. This adjuster manages the channel-level request dispatch parallelism for each application to minimize I/O interference. Finally, ZNSFQ employs a global adaptive coordinator to alleviate device-level I/O blocking, reducing tail latency and CPU consumption while satisfying fairness and performance. A comprehensive evaluation demonstrates that ZNSFQ significantly enhances fairness and performance compared to the latest fair queuing schedulers. In sequential access scenarios, ZNSFQ enhances fairness by over 38.13% and increases I/O bandwidth by more than 49.24%. Furthermore, in random access scenarios, it reduces CPU utilization by 70.22% while maintaining both fairness and high performance.

Emerging Zoned Namespace (ZNS) provides hosts with fine-grained, performance-predictable storage management. ZNS organizes the address space into zones composed of fixed-size, sequentially written, non-overwritable blocks, making it suitable for log-structured file systems. However, our experimental analysis reveals that ZNS’s write restrictions introduce notable persistence overhead. Firstly, out-of-place updates of data blocks require frequent small modifications to file metadata blocks, which are typically much smaller than a block, to record the latest logical block address. Secondly, some files, such as databases’ Write-Ahead Logging files, frequently execute synchronous small writes, with I/O sizes typically smaller than a logical block. The persistence of these file metadata and file data requires writing back the entire block even if it is only partially updated. This significantly increases the I/O latency and potentially reduces device lifespan. This article proposes exZNS , an innovative extension of ZNS, designed to provide both regular zones and byte-loggable zones . By exposing the persistent write buffer of the opened zones on the device to the application, the byte-loggable zone allows for appending at byte granularity through a new set of APIs. To reduce the persistence overhead described above, we built exBlzFS , a novel high-performance file system for exZNS. exBlzFS selectively records the partial updates of metadata blocks to the byte-loggable zone to ensure metadata persistence, and persists file data to the byte-loggable zone at byte granularity to absorb the frequent small writes. Evaluations show that exBlzFS increases the IOPS of RocksDB by 42.7% and 76.3%, and reduces the device’s write traffic by 86% and 94%, compared with BlzFS and F2FS, respectively. 1

Data security (including confidentiality, integrity, and availability) and crash consistency guarantees are essential for building trusted persistent memory (PM) systems. Security and consistency metadata are added to enable the guarantees. Recent studies show that errors in security metadata have the amplified effect, which significantly affects data availability. However, the impact of consistency metadata errors on data availability has rarely been discussed. We identify the crash-consistency error amplification (CCEA) problem, several errors in consistency metadata can make a large portion of data in PM possibly inconsistent. The error sensitivity of consistency metadata is higher than data and security metadata, thus requiring special attention. It is inefficient to address this problem by using the methods that are proposed to alleviate the amplified effect of security metadata errors, because security metadata are generally designed for a single purpose (e.g., integrity verification), while consistency metadata are designed for multiple purposes, including inconsistency locating and recovery. To effectively and efficiently alleviate the CCEA problem, we propose a c rash c o nsistency ver ification approach (COVER) that decouples inconsistency locating and recovery. COVER provides three design options that support different tradeoffs between effectiveness and efficiency. Experimental results show that COVER effectively alleviates the problem with only about 1.0% performance degradation on average compared with the state-of-the-art secure PM design.

Securing systems’ main memory is important for building trusted data centers. To ensure memory security, encryption and integrity verification techniques update the security metadata (e.g., encryption counters and integrity trees) during memory data writes. Existing studies are optimistic about the effect of data writes on system performance since they regard all data writes as background operations. However, we show that security metadata updates significantly increase data write latency. High-latency data writes frequently fill up write buffers in the system, forcing the system to perform the writes in the critical path. As a result, performance-critical data reads need to wait for the execution of these writes, which increases data read latency and degrades system performance. In this paper, we propose SEED that improves the performance of secure memory systems by speculatively updating security metadata in the background before data writes arrive. To enable speculative updates, SEED predicts which dirty cache lines will be written to memory through natural evictions. We find that cache evictions depend on multiple factors. To decouple the dependencies for accurate predictions, we devise a two-step eviction prediction method based on our observation that the next eviction victim rarely changes in a set. The first step predicts which cache sets will evict cache lines, while the second step predicts which cache lines will be evicted by finding the next eviction victims in the sets. For predicted evictions, we develop a speculative updater to perform speculative updates. We analyze the invariants that must be followed by the updater to ensure the correctness of speculative updates. The updater rolls back the speculatively updated security metadata of inaccurate predictions. To reduce the rollback overhead, we devise a rollback batching and an update pausing optimization for the updater. Experimental results show that SEED reduces data write latency by 39.8%, data read latency by 44.9%, and improves performance by 40.0% on average compared with the state-of-the-art secure memory design.

Key-Value Stores (KVS) based on log-structured merge-trees (LSM-trees) are widely used in storage systems but face significant challenges, such as high write amplification caused by compaction. KV-separated LSM-trees address write amplification but introduce significant space amplification, a critical concern in cost-sensitive scenarios. Garbage collection (GC) can reduce space amplification, but existing strategies are often inefficient and fail to account for workload characteristics. Moreover, current key-value (KV) separated LSM-trees overlook the space amplification caused by the index LSM-tree. In this paper, we systematically analyze the sources of space amplification in KV-separated LSM-trees and propose Scavenger+, which achieves a better performance-space tradeoff. Scavenger+ introduces (1) an I/O-efficient garbage collection scheme to reduce I/O overhead, (2) a space-aware compaction strategy based on compensated size to mitigate index-induced space amplification, and (3) a dynamic GC scheduler that adapts to system load to make better use of CPU and storage resources. Extensive experiments demonstrate that Scavenger+ significantly improves write performance and reduces space amplification compared to state-of-the-art KV-separated LSM-trees, including BlobDB, Titan, and TerarkDB.

Erasure codes(EC) have been widely adopted to provide high data reliability with low storage costs in clusters. Due to the deletion and out-of-place update operations, some data blocks are invalid, which unfortunately arouses the tediousgarbage collection(GC) problem. Several limitations still plague existing designs: substantial network traffic, unbalanced traffic load, and low read/write performance after GC. This paper proposes FastGC, a fast garbage collection method that merges the old stripes into a new stripe and reclaims invalid blocks. FastGC quickly generates an efficient merge solution by stripe grouping and bit sequences operations to minimize network traffic and maintains data block distributions of the same stripe to ensure read performance. It carefully allocates the storage space for new stripes during merging to eliminate the discontinuous free spaces that affect write performance. Furthermore, to accelerate the parity updates after merging, FastGC greedily schedules the transmission links for multi-stripe updates to balance the traffic load across nodes and adopts a maximum flow algorithm to saturate the bandwidth utilization. Comprehensive evaluation results show via simulations and Alibaba ECS experiments that FastGC can significantly reduce 10.36%-81.22% of the network traffic and 34.25%-72.36% of the GC time while maintaining read/write performance after GC.

QLC solid-state disks (SSDs) are increasingly deployed in large-scale storage systems. While achieving remarkable storage density and cost-effectiveness, QLC NAND exhibits degraded performance. To alleviate the issue, Independent Multi-Plane (IMP) read has been proposed to leverage the plane-level parallelism under random read workloads. However, compared to the previous generations of chips, the variation in read latency of QLC chip has widened significantly, and the number of planes in a QLC chip has increased. As a result, the idle time in IMP commands has escalated dramatically. Moreover, the conventional layout of data and parities in redundant array of independent NAND (RAIN) increases the probability of high latency read occureneces, exacerbating the contribution to idle time. Consequently, the incorporation of IMP is inherently inefficient in contemporary QLC NAND flash chips. In this paper, we propose an efficient independent read (EFFIR) scheme to tackle this challenge. EFFIR features a read latency variation aware transaction service that properly combines read transactions to minimize idle time and proactively transfers read data to eliminate unnecessary delays. Moreover, EFFIR incorporates a read latency variation aware RAIN that reorganizes the layout of data and parity to mitigate the impact of high-latency data access on idle time. Our comprehensive experimental results demonstrate and elucidate how EFFIR significantly enhances SSD responsiveness while consistently delivering favorable performance across a diverse range of read-intensive workloads.

Data migration strategies (DMS) improve the overall performance of hybrid memory systems by migrating frequently accessed (hot) data to faster memory. However, designing an efficient DMS is challenging since the key metrics of DMS -hot data selection, migration granularity, and migration frequency -are sensitive to access patterns of workloads. Most existing strategies focus on only one of these metrics and often overlook the crucial impact of access patterns, resulting in sub-optimal performance and unnecessary migration traffic. In this paper, we propose AdaptHM, a fully access-pattern-aware Adaptive data migration strategy for Hybrid Memory systems. AdaptHM achieves adaptability on all three metrics through its unique multi-level data framework. First, AdaptHM adopts a group-level competition policy to select hot blocks, which responds faster to access patterns than threshold-based policies. Second, AdaptHM enables segment-level dynamic migration granularity by decoupling migration from remapping, which shows better access pattern resilience than existing schemes with fixed-size global migration granularity. Third, AdaptHM adjusts the migration frequency at set-level by periodically assessing the migration benefit, avoiding unnecessary migrations. Experimental results demonstrate that AdaptHM improves the performance by an average of 12.78% and reduces energy consumption by up to 37.24% compared to the state-of-the-art scheme.

As the technology node develops toward its physical limit, lithographic hotspot detection has become increasingly important and ever-challenging in the computer-aided design (CAD) flow. In recent years, convolutional neural networks (CNNs) have achieved great success in hotspot detection. However, the interpretability of their hotspot prediction has yet to be considered. Compared with conventional lithography simulation and pattern matching-based methods, the black-box nature of CNNs wavers their practical applications with confidence. In this article, we propose the first interpretable CNN-based hotspot detector capable of providing high-detection accuracy and reliable explanations for hotspot identification. Specifically, we augment the training dataset with expanded error markers obtained and preprocessed from lithography simulation, which are then learned by an encoder-decoder architecture as intermediate features. We additionally introduce coordinate attention in the encoder to facilitate better-feature extraction. By learning these error markers and part of their surrounding metals as root cause hotspot features, our architecture achieves the highest-hotspot accuracy of 99.78% and the lowest-false positive rate of 5.29% compared to all prior work. Moreover, our method demonstrates the best visual and quantitative interpretability results when applying CNN interpretation methods.

Memristive accelerators (MAs), built with memristive crossbar arrays (MCAs), have gained significant attention for their ability to efficiently perform matrix-vector multiplication in diverse applications. Modeling and simulation are indispensable tools for exploring and evaluating architectural design. Specifically, for MAs, maintaining high computation accuracy has been challenging because of realistic nonidealities like wire resistance (i.e., IR drop), I–V nonlinearity, program variation, and so on. Thus, for the system architects, fast and exact analysis of the effects of nonidealities is highly desirable, especially during the vast early design space exploration of the architecture. However, the SPICE model and existing MCA compact models (CMs) do not offer acceptable speeds for accurate simulation purposes, making them less practical. Additionally, the existing MCA simplified circuit models and predictive models fail to produce effective results due to the complexity of IR drop, let alone the coupling of multiple nonidealities. To enable fast and accurate design exploration for MAs with nonidealities considered, in this article, we propose FADESIM which includes fast IR drop simulation methods and a processing chain for the joint simulation of multiple nonidealities. Starting with the analysis of the accurate CM for IR drop, we explore the special properties of the model to enable fast iterative methods as well as a method to skip invalid calculations. This significantly reduces the time complexity of the simulation from naive$O(n^{6})$to$O(n^{3})$. For less severe IR drop cases, a custom iterative update algorithm is presented for faster simulations as a supplement, specifically with a time complexity of near$O(n^{2})$and proven applicable conditions. To simulate multiple nonidealities, we introduce a processing chain to inject corresponding processing functions before, during, and after the proposed fast IR drop simulation process according to the stages at which nonidealities take effect. The array-level experimental results show that our method achieves accurate simulations, with$19.8 \times - 884.9 \times $faster and$276.7 \times - 8018.8 \times $reduced memory usage compared to SPICE simulation. Further experiments at the algorithm-level demonstrate the effectiveness of our method in assisting architects with evaluating their designs.

Emerging persistent memory (PM) can provide large persistent capacity with performance comparable to DRAM in modern memory systems. Persistent transactional memory (PTM) needs to ensure data consistency after unexpected power loss or crashes. Therefore, crash consistency strategies, such as persistent logging, are still required. However, the additional overhead introduced by these strategies, such as significant extra writes on PM, can lead to system performance degradation. In this article, we propose CodePM, a fault-tolerant PM transactional library that utilizes parity-based crash consistency to remove logging overhead while guaranteeing the correct state of data. CodePM reuses the decoding capability of parity to detect and recover inconsistent objects. To ensure consistency without logs when updating, CodePM employs fine-grained memory fences to carefully align potential inconsistency with the repairability of parity. To detect inconsistency without logs when recovering, CodePM utilizes optimistic speculative scanning recovery by reusing checksum and parity, which supports instant recovery with transient degraded reliability. Moreover, we study the memory fence blocking effects and further augment CodePM with pipelined encoding and persistent writing to hide update latency. We implemented CodePM on Pangolin, the state-of-the-art parity-based PTM for fault-tolerance. Evaluation results with real-world workloads on Intel Optane DCPMM show that CodePM can achieve up to$3.4\times $higher throughput than Pangolin.

Massive off-chip accesses in graphics processing units (GPUs) are the main performance bottleneck. We find that many writes are duplicate, and the duplication can beinter-dupandintra-dup. Whileinter-dupmeans different memory blocks are identical, andintra-dupmeans all the 4B elements in a line are the same. In this work, we propose a cache-assisted GPU memory deduplication architecture named cache-assisted GPU memory deduplicated (CMD) to reduce the off-chip accesses via utilizing the data duplication in GPU applications. CMD includes three key design contributions which aim to reduce the three kinds of accesses: 1) a novel GPU memory deduplication architecture that removes theintra-dupandinter-duplines. We design several techniques to manage duplicate blocks, reducing massive off-chip writes; 2) we propose a cache-assisted read scheme to reduce the reads to duplicate data. When an L2 cache miss wants to read the duplicate block, if the reference block has been fetched to L2 and it is clean, we can copy it to the L2 missed block without accessing off-chip DRAM. As for the reads tointra-dupdata, CMD uses the on-chip metadata cache to get the data; and 3) when a cache line is evicted, the clean sectors in the line are invalidated while the dirty sectors are written back. However, most read-only victims are rereferenced from DRAM more than twice. Therefore, we add a full-associate FIFO to accommodate the read-only (it is also clean) victims to reduce the rereference counts. Experiments show that CMD can decrease the off-chip accesses by 31.01%, reduce the energy by 32.78% and improve performance by 42.53%. Besides, CMD can improve the performance of memory-intensive workloads by 57.56%.

Graph embedding maps graph nodes to low-dimensional vectors and is widely used in machine learning tasks. The increasing availability of billion-edge graphs underscores the importance of learning efficient and effective embeddings on large graphs, such as link prediction on Twitter with over one billion edges. Most existing graph embedding methods fall short of reaching high data scalability. In this paper, we present a general-purpose, distributed, information-centric random walk-based, and pipeline-optimized graph embedding framework,$\sf{DistGER-Pipe}$DistGER−Pipe, which scales to embed billion-edge graphs.$\sf{DistGER-Pipe}$DistGER−Pipeincrementally computes information-centric random walks to reduce redundant computations for more effective and efficient graph embedding. It further leverages a multi-proximity-aware, streaming, parallel graph partitioning strategy, simultaneously achieving high local partition quality and excellent workload balancing across machines.$\sf{DistGER-Pipe}$DistGER−Pipealso improves the distributed$\sf{Skip-Gram}$Skip−Gramlearning model to generate node embeddings by optimizing access locality, CPU throughput, and synchronization efficiency. Finally,$\sf{DistGER-Pipe}$DistGER−Pipedesigns pipelined execution that decouples the operators in sampling and training procedures with an inter-round serial and intra-round parallel processing, attaining optimal utilization of computing resources. Experiments on real-world graphs demonstrate that compared to state-of-the-art distributed graph embedding frameworks, including$\sf{KnightKing}$KnightKing,$\sf{DistDGL}$DistDGL,$\sf{Pytorch-BigGraph}$Pytorch−BigGraph, and$\sf{DistGER}$DistGER,$\sf{DistGER-Pipe}$DistGER−Pipeexhibits 3.15×–1053× acceleration, 45% reduction in cross-machines communication, >10% effectiveness improvement in downstream tasks, and 38% enhancement in CPU utilization.

Convolutional neural networks (CNNs) successfully detect lithographic hotspots by learning from hand-designed features of layout patterns or entire layouts, as images, in an end-to-end fashion. However, compared to lithography simulation, CNN-based solutions demonstrate inferior hotspot detection accuracy and a high false-alarm rate. Moreover, the interpretability of the hotspot prediction process has yet to be considered due to the “black-box” nature of CNNs. In this work, inspired by conventional lithography simulation where defect regions are simulated as direct evidence for hotspot identification, we propose an explainable two-stage CNN-based hotspot detector that considers both the accuracy and interpretability of hotspot detection. Our architecture learns to locate the defect areas in the first stage as extracted hotspot features. In the second stage, we combine the strength of feature engineering and end-to-end learning, incorporating the original layout input, the learned defect location map from the first stage, and a fixed auxiliary region of interest (ROI) map for final hotspot detection. Experimental results for our technique exhibit the highest hotspot accuracy (98.1%) and the lowest false-alarm rate (4.0%) thus far compared to all prior CNN solutions. We also demonstrate the best overall qualitative and quantitative interpretability results with the highest increase in confidence (IC) and the lowest average drop (AD) in scores when CNN interpretation methods such as Grad-CAM-based approaches are applied. We further demonstrate use cases of our technique for successfully justifying and pinpointing hotspot mispredictions by examining the prediction evidence from our learned defect locations.

Delta compression can complement data deduplication by further minimizing redundancy through the compression of non-duplicate data chunks. When adding delta compression to deduplication-based backup systems, however, two primary challenges arise that degrade performance of inline deduplication. First, extra I/Os are introduced along the critical paths of backup and restoration for retrieving base chunks, slowing the system. Second, rewriting techniques prohibit specific data chunks from serving as base chunks for delta compression to improve restore performance, resulting in a loss of compression efficiency. In this paper, we introduce LoopDelta, a framework that seamlessly integrates delta compression into inline deduplication for backup storage, addressing the aforementioned challenges by using three techniques: (1) dual-locality-based similarity tracking leverages both logical and physical locality to detect most of the similar chunks, which, due to their locality, can be prefetched by piggybacking on routine operations during deduplication, thereby eliminating extra I/Os during backup; (2) cache-aware filter identifies base chunks requiring extra I/Os during restore and prevents their referencing, thus eliminating extra restore I/Os; and (3) inversed delta compression, which reverses the roles of base and target chunks in the traditional delta compression approach, thereby allowing for the delta compression of data chunks that are otherwise prohibited as base chunks due to rewriting techniques. Experiments show that LoopDelta increases the compression ratio by 1.28 to 11.33 times over basic deduplication, without significantly affecting backup throughput, and enhances restore performance by up to 3.57 times.

Serverless computing relies on keeping functions alive or pre-warming them before invocation to mitigate the cold start problem, stemming from the overhead of initializing function startup environments. However, under constrained cloud resources, accurately predicting the invocation patterns of sparse functions remains challenging. This limits the formulation of effective pre-warm and keep-alive strategies, leading to frequent cold starts and degraded user satisfaction. To address these challenges, we proposeSPFaaS, a hybrid framework based on sparse function prediction. To enhance the learnability of sparse function invocation data,SPFaaStakes into account the characteristics of cloud service workloads along with the features of pre-warm and keep-alive strategies, transforming function invocation records into probabilistic data. It captures the underlying periodicity and temporal dependencies in the data through multiple rounds of sampling and the combined use of Gated Recurrent Units and Temporal Convolutional Networks for accurate prediction. Based on the final prediction outcome and real-time system states,SPFaaSdetermines adaptive pre-warm and keep-alive strategies for each function. Experiments conducted on two real-world serverless clusters demonstrate thatSPFaaSoutperforms state-of-the-art methods in reducing cold starts and improving user satisfaction.

In modern datacenters, memory disaggregation un-packs monolithic servers to build independent network-connected compute and memory pools, greatly improving resource uti-lization. Existing memory-disaggregated key-value stores adopt ownership sharing or ownership partitioning for request scheduling and they all show poor performance and scalability. The former generates multiple cache misses and multiple network round trips due to poor cache locality, while the latter fails to fully use resources on all Compute Nodes (CNs) due to load imbalance. In this paper, we propose Seastar, a fast and scalable key-value store that achieves high cache locality as well as load balancing. To enable high concurrency and ensure linearizability, Seastar exploits a self-verification version chain to provide out-of-place atomic updates and ensure consistent reads. To improve cache locality, Seastar leverages a pair-based ownership assigning scheme to assign the processing permission of requests corre-sponding to each key to a CN pair through two independent hash functions. To ensure load balancing among CNs, Seastar proposes a decentralized power-of-two-choices routing scheme to route the requests to the less-loaded CNs in CN pairs for handling. Experimental results demonstrate that Seastar improves the throughput by up to 6.7x and significantly reduces the latency compared with state-of-the-art memory-disaggregated key-value stores.

Modern SSDs achieve low latency and high throughput by utilizing multiple levels of SSD parallelism. Fairness is also a critical design consideration in cloud environments and has inspired great interest in recent years to study it. However, we find that excessive striving for SSD parallelism can inadvertently harm the SSD fairness due to the striped data layout. We observe from our experimental analysis that some common access patterns in cloud loads (e.g., high-intensity I/O) are not addressed by SSD parallelism, which can affect both SSD parallelism and fairness. In this paper, we analyze two address mapping policies with orthogonal characteristics from SSD parallelism and fairness perspectives. Based on extensive experiments and observations, we schedule a hybrid address mapping policy to achieve fairness of SSDs in cloud environments by distinguishing different I/O access modes and cold/hot data, while ensuring SSD performance. Experimental results on the latest block-level I/O traces show that our approach significantly improves performance and fairness by up to 77.3% and 62.8%, respectively, over the previous schemes with negligible cost. Meanwhile, it can also reduce the write amplification by 23.5% to 33.5%.

Backdoor attacks have been well-studied in visible light object detection (VLOD) in recent years. However, VLOD can not effectively work in dark and temperature-sensitive scenarios. Instead, thermal infrared object detection (TIOD) is the most accessible and practical in such environments. In this paper, our team is the first to investigate the security vulnerabilities associated with TIOD in the context of backdoor attacks, spanning both the digital and physical realms. We introduce two novel types of backdoor attacks on TIOD, each offering unique capabilities: Object-affecting Attack and Range-affecting Attack. We conduct a comprehensive analysis of key factors influencing trigger design, which include temperature, size, material, and concealment. These factors, especially temperature, significantly impact the efficacy of backdoor attacks on TIOD. A thorough understanding of these factors will serve as a foundation for designing physical triggers and temperature controlling experiments. Our study includes extensive experiments conducted in both digital and physical environments. In the digital realm, we evaluate our approach using benchmark datasets for TIOD, achieving an Attack Success Rate (ASR) of up to 98.21%. In the physical realm, we test our approach in two real-world settings: a traffic intersection and a parking lot, using a thermal infrared camera. Here, we attain an ASR of up to 98.38%.

Emerging resistive RAM (ReRAM) devices can in-situ execute vector-matrix-multiplication (VMM) and is able to achieve energy-efficient in-memory scientific computing. However, the peripheral separated S&Hs and ADCs for row buffering and sensing in conventional designs are the system bottleneck. We propose an ADC-less all-in-one processing-in-ReRAM design that enables the precharge once, readout multiple-bits (PORM) functionality for overlapping the tRCD latencies of different-significance result bits sensed out on the same bitline. Specifically, to support PORM sensing, we propose a cascaded-feedback bitline sensing architecture for VMM and a buffering-and-sensing-collocated sense amplifier elemental design with the bitline and the storage node fully decoupled for enabling conflict-free column accesses. We further propose cross-level inter-leaving mechanism for successive column VMM accesses to reduce the overall latency through improving the hardware spatiotemporal utilization. Experimental results show that our proposed design achieves 297% overall performance improvement and 85.8% energy reduction, compared with an aggressive baseline.

QLC flash-based SSDs has attracted growing interest and is expected to fit in read-intensive scenarios owing to its higher cost-effectiveness and shorter write endurance compared with the Triple-Level Cell (TLC) SSDs. Recently, new commands supporting independent reads such as Single Operation Multiple Locations (SOML) and Independent multi-plane (IMP) read are proposed to improve read performance. Unfortuantely, while independent read exhibits a significant performance improvement, we show in this paper that the exisiting approach fails to fully exploit its potential due to the larger read latency variation and more planes per die for current SSD architecture. Through a set of experiments, we demonstrate that the lack of a read latency variation aware machanism leads to low performance of independent read among a wide variety of workloads. To alleviate this issue, we propose LITA, which key idea is to combine read transactions with similar latency into one command. LITA includes (1) LIT, a latency variation aware transaction combination, (2) and TASAP, a latency variation aware transaction completion service. The experimental results show LITA can reduce read latency by 20.4% and 9.6% on average for 4-planes QLC SSDs under IMP and SOML, respectively.

Data-parallel deep neural networks (DNN) training systems deployed across nodes have been widely used in various domains, while the system performance is often bottlenecked by the communication overhead among workers for synchronizing gradients. Top-k sparsification compression is the de facto approach to alleviate the communication bottleneck, which truncates the gradient to its largest k elements before sending it to other nodes.

Persistent Memory (PM) offers both byte-address ability and non-volatility, making it well-suited for accelerating$\mathrm{B}^{+}$-Tree indexes. However, existing persistent$\mathrm{B}^{+}$-Tree indexes face significant performance challenges due to high structural modification operation (SMO) overhead. SMOs often result in costly item migrations and increased tail latency, which severely degrade the overall performance. In this paper, we present SSTree, a high-performance$\mathrm{B}^{+}$– Tree index specifically optimized to address SMO overhead. SSTree introduces three key innovations: (i) efficient leaf node expansion using a list of subnodes to postpone expensive node splits, (ii) delegated fingerprints to speed up search operations across subnodes, and (iii) proactive subnode compaction that employs out-of-place updates to optimize item organization. Our evaluation demonstrates that SSTree delivers up to$4.38\times$higher write throughput and up to$62\times$lower tail latency compared to state-of-the-art persistent$\mathrm{B}^{+}$-Tree indexes.

Hash indexes are widely used in key-value storage systems due to their ability to perform rapid single-point queries. The persistent memory (PM) technology has received significant attention in both academia and industry due to its high performance, non-volatility, and large capacity characteristics. Currently, hash indexes tailored for persistent memories have been extensively researched. However, through an in-depth experimental study, we have discovered that existing persistent hash indexes suffer from low query performance. This is primarily due to persistent memory's higher read latency than DRAM's, which reduces the performance of both positive and negative queries in persistent hash indexes. Additionally, the former's higher read lock overhead further diminishes query performance. To address the above problems, we propose in this paper a Read-Optimized Persistent Hash Index, referred to as ROPHI, based on fingerprint filtering and lock-free prefetching. By employing a fingerprint filtering method, ROPHI introduces a DRAM-based Cuckoo filter to store fingerprints of keys on top of the PM-based hash table, effectively mitigating the time-consuming access overhead of persistent memory hash tables by accessing only the DRAM-based filter. Additionally, ROPHI employs lock-free prefetching for positive query acceleration, utilizing lock-free optimistic concurrent read techniques to avoid read lock overhead and high-speed cache prefetching techniques to reduce access overhead to persistent memory. Experimental results on the Intel Optane DC Persistent Memory Module (DCPMM) platform demonstrate that ROPHI significantly improves query performance over existing persistent hash index schemes. Specifically, ROPHI achieves an improvement of 2.67×-13.59× in negative query performance and 1.72x-7.86x in positive query performance. ROPHI outperforms the state-of-the-art SmartHT in positive query throughput by 34.5%, and in insertion and deletion throughput by 9.20% and 19.87% respectively, while sacrificing only 1.93% of negative query throughput. Additionally, it achieves a 5.07x improvement in recovery efficiency.

Key- Value Stores (KVS) implemented with log- structured merge-tree (LSM-tree) have gained widespread ac-ceptance in storage systems. Nonetheless, a significant challenge arises in the form of high write amplification due to the compaction process. While KV-separated LSM-trees successfully tackle this issue, they also bring about substantial space am-plification problems, a concern that cannot be overlooked in cost-sensitive scenarios. Garbage collection (GC) holds significant promise for space amplification reduction, yet existing GC strategies often fall short in optimization performance, lacking thorough consideration of workload characteristics. Additionally, current KV-separated LSM-trees also ignore the adverse effect of the space amplification in the index LSM-tree. In this paper, we systematically analyze the sources of space amplification of KV- separated LSM-trees and introduce Scavenger, which achieves a better trade-off between performance and space amplification. Scavenger initially proposes an I/O-efficient garbage collection scheme to reduce I/O overhead and incorporates a space-aware compaction strategy based on compensated size to minimize the space amplification of index LSM-trees. Extensive experiments show that Scavenger significantly improves write performance and achieves lower space amplification than other KV-separated LSM-trees (including BlobDB, Titan, and TerarkDB).

The memristive Computing-in-Memory (CIM) sys-tem can efficiently accelerate matrix-vector multiplication (MVM) operations through in-situ computing. The data layout has a significant impact on the communication performance of CIM systems. Existing software-level communication opti-mizations aim to reduce communication distance by carefully designing static data layouts, while wear-leveling (WL) and error mitigation methods use dynamic scheduling to enhance system reliability, resulting in randomized data layouts and increased communication overhead. Besides, existing CIM compilers di-rectly map data to physical crossbars and generate instructions, which causes inconvenience for dynamic scheduling. To address these challenges of balancing communication performance and reliability while coordinating existing CIM compilers and dy-namic scheduling, we propose a disorder-resistant computation translation layer (DRCTL), which improves system lifetime and communication performance through co-optimization of data layout and dynamic scheduling. It consists of three parts: (1) We propose an address conversion method for dynamic scheduling, which updates the addresses in the instruction stream after dynamic scheduling, thereby avoiding recompilation. (2) Dynamic scheduling strategy for reliability improvement. We propose a hierarchical wear-leveling (HWL) strategy, which reduces communication by increasing scheduling granularity. (3) Communication optimization for dynamic scheduling. We propose data layout-aware selective remapping (LASR), which helps dynamic scheduling methods improve communication lo-cality and reduce latency by exploiting data dependencies. The experiments demonstrate that HWL extends lifetime by 100.3-205.9 x compared to not using WL. Even with a slight lifetime decrease compared to the state-of-the-art WL (TIWL), it still supports continuous neural network training for 7 years. After applying LASR to HWL, the number of execution cycles, energy consumption, on-chip and off-chip NoC accesses decrease by an average of 26.91 %, 26.88%, 36.41 %, and 80.62%, respectively.

RGB-Thermal Salient Object Detection (RGBT-SOD) plays a critical role in complex scene recognition applications, such as autonomous driving. However, security research in this domain is still in its infancy. This paper presents the first backdoor attack on RGBT-SOD systems, generating saliency maps on triggered inputs that depict non-existent salient objects chosen by the attacker or falsely mark an entire image as fully salient or entirely non-salient. We uncover that triggers have an influence range for generating non-existent salient objects, supported by a theoretical analysis. Extensive experiments show the effectiveness of our attack in both digital and physical-world scenarios. Notably, our dual-modality backdoor attack achieves an Attack Success Rate (ASR) of 86.72% with only five pairs of poisoned images in model training. After investigating potential countermeasures, we find them inadequate in mitigating our attacks, highlighting the urgent need for robust defenses against sophisticated backdoor attacks in RGBT-SOD systems.

It is essential to utilize a large number of I/O workloads to analyze commodity system performance or simulate scientific phenomena in high-performance scientific computing. I/O traces are often unavailable at scale due to trace storage overhead, privacy concerns, and the performance impact of trace instrumentation. We study how to generate sufficiently representative I/O workloads using Generative Adversarial Networks (GANs). The best GAN architecture can generate I/O workloads with maximum mean discrepancy (MMD) as low as 0.015-0.05, which implies the synthetic I/O workloads have successfully learned the potential distribution of real I/O traces. We demonstrate that the performance similarity between the original I/O trace and the generated I/O workload through trace replay can be 90.36%-97.32%.

A significant drawback of erasure-coding is suffering from the expensive update traffic. The analysis of real-world-production traces shows that partial updates, including partial-block-updates and partial-stripe-updates, are both common. Existing schemes cannot work adequately for partial updates. Raid-based scheme coordinates multiple updated entire blocks to update parity, yet it incurs significant network traffic for partial-block-updates. Delta-based scheme transmits the updated parts and independently updates parity, yet it cannot share computed-delta parts for partial-stripe-updates. We propose CoRD, which optimally combines Raid-based and Delta-based schemes to minimize the update traffic. It exploits the offset address intersections between multiple updated blocks and only transmits the updated parts to coordinate in parity updates. CoRD further address cross-block update scenarios by flipping some dedicated blocks to improve the performance. Comprehensive evaluations verify the effectiveness of CoRD for the latest traces, with the update traffic reduction of 37.02%-87.19% and the performance improvement of 36.54%-231.92% compared to state-of-the-art.

More and more storage systems use erasure code to tolerate faults. It takes pieces of data blocks as input and encodes a small number of parity blocks as output, where these blocks form a stripe. When reconsidering the recovery problem in the multi-stripe level and heterogeneous network clusters, quickly generating an efficient multi-stripe recovery solution that reduces recovery time remains a challenging and time-consuming task. Previous works either use a greedy algorithm that may fall into the local optimal and have low recovery performance or a meta-heuristic algorithm with a long running time and low solution generation efficiency. In this article, we propose a Stripe-schedule Aware Repair (SARepair) technique for multi-stripe recovery in heterogeneous erasure-coded clusters based on Reed–Solomon code. By carefully examining the metadata of blocks, SARepair intelligently adjusts the recovery solution for each stripe and obtains another multi-stripe solution with less recovery time in a computationally efficient manner. It then tolerates worse solutions to overcome the local optimal and uses a rollback mechanism to adjust search regions to reduce recovery time further. Moreover, instead of reading blocks sequentially from each node, SARepair also selectively schedules the reading order for each block to reduce the memory overhead. We extend SARepair to address the full-node recovery and adapt to the LRC code. We prototype SARepair and show via both simulations and Amazon EC2 experiments that the recovery performance can be improved by up to 59.97% over a state-of-the-art recovery approach while keeping running time and memory overhead low.

Persistent memory (PM) is promising to be the next-generation storage device with better I/O performance. Since the traditional I/O path is too lengthy to drive PM featuring low latency and high bandwidth, prior works proposed memory-mapped I/O (MMIO) to shorten the I/O path to PM. However, native MMIO directly maps files into the user address space, which puts files at risk of being corrupted by scribbles and non-atomic I/O interfaces, causing serious reliability issues. To address these issues, we propose RMMIO, an efficient user-space library that provides reliable MMIO for PM systems. RMMIO provides atomic I/O interfaces and lightweight snapshots to ensure the reliability of MMIO. Compared with existing schemes, RMMIO mitigates additional writes and extra software overheads caused by reliability guarantees, thus achieving MMIO-like performance. In addition, we also propose an automatic snapshot with efficient memory management for RMMIO to minimize data loss incurred by reliability issues. The experimental results of microbenchmarks show that RMMIO achieves 8.49x and 2.31x higher throughput than ext4-DAX and the state-of-the-art MMIO-based scheme, respectively, while ensuring data reliability. The real-world application accelerated by RMMIO achieves at most 7.06x higher throughput than that of ext4-DAX.

With the popularity of cloud services, cloud block storage (CBS) systems have been widely deployed by cloud providers. Cloud cache plays a vital role in maintaining high and stable performance in cloud block storage systems. In the past few decades, much research has been conducted on the design of cache replacement policies. Prior work frequently relies on manually-engineered heuristics to capture the most common cache access patterns, or predict the reuse distance and try to identify the blocks that are either cache-friendly or cache-averse. Researchers are now applying recent advances in machine learning to guide cache replacement policy, augmenting or replacing traditional heuristics and data structures. However, most existing approaches depend on a certain environment which restricted their application, e.g., some methods only consider the on-chip cache consisting of program counters (PCs). Moreover, those approaches with attractive hit rates are usually unable to deal with modern irregular workloads, due to the limited feature used. In contrast, we propose a cloud cache replacement framework to automatically learn the relationship between the probability distribution of different replacement policies and workload distribution by using deep reinforcement learning. We train an end-to-end cache replacement policy based on the requested address with two efficient and stable cache replacement policies. Furthermore, by using prioritized experience replay and setting parameter constraints, our framework can accelerate the offline training process without affecting the cloud application. We have evaluated our proposed framework by using block-based I/O traces collected from Alibaba Cloud and Tencent Cloud, two of the largest cloud providers in the world, and several open-source traces. Experimental results show that our method not only outperforms several state-of-the-art cache methods in hit rate, but also reduces request latency and data traffic to the backend storage.

Since cloud service providers adopt the sharing mode to improve the utilization of solid state drives (SSDs) and reduce management costs, fairness is a critical design consideration and has drawn great interest in recent years. There are many methods to achieve fairness at the SSD device level, including cache-based resource allocation and queue rescheduling of transaction scheduling unit (TSU). However, poor locality data in cloud environments makes these methods of achieving fairness through the cache level invalid. In addition, existing fairness approaches at the TSU level treat flash memory as a black box and do not consider some characteristics of flash memory, leading to room for performance improvement. In this work, we propose a novel collaborative SSD fairness scheme, named a coordinated SSD cache and TSU fair scheme (CoFS), that achieves end-to-end full path fairness at the device level. A reinforcement learning-assisted fairness management scheme is designed to coordinate the SSD cache and TSU considering both the cache space and bandwidth resources which are significant for fairness control. The key idea is to enable the front-end SSD cache to achieve workload-level fairness by recognizing workload patterns, while the back-end TSU achieves flash-level fairness by sensing SSD internal status. Finally, CoFS collaborates with them to achieve SSD device-level fairness. In addition, we have designed a flexible reward function mechanism in the cache to balance different optimization objectives and augment TSU queue scheduling to adapt to different types of SSDs. Experimental results show that CoFS improves the overall fairness and performance by 30.8% to 56.7% and 42.1% to 98.7% in different scenarios.

Disaggregated memory (DM) architecture separates CPU and DRAM into computing/memory resource pools and interconnects them with high-speed networks. Storage systems on DM locate data by distributed index. However, existing distributed indexes either suffer from prohibitive synchronization overhead of write operation or sacrifice the performance of read operation, resulting in low throughput, high tail latency, and challenging trade-off. In this paper, we present Marlin+, a scalable and optimized B+-tree on DM. Marlin+ provides atomic granularity synchronization between write operations via three strategies: 1) a concurrent algorithm that is friendly to IDU operations (Insert, Delete, and Update), enabling different clients to concurrently operate on the same leaf node, 2) shared-exclusive leaf node lock, effectively preventing conflicts between index structure modification operation (SMO) and IDU operations, and 3) critical path compression of write to reduce latency of write operation. Moreover, Marlin+ proposes an adaptive remote address cache to accelerate the access of hot data. Compared to the state-of-the-art schemes based on DM, Marlin achieves 2.21× higher throughput and 83.4% lower P99 latency under YCSB hybrid workloads. Compared to Marlin, Marlin+ improves the throughput by up to 1.58× and reduces the P50 latency by up to 50.5% under YCSB read-intensive workloads.

Consistent Hashing (CH) algorithms are widely adopted in networks and distributed systems for their ability to achieve load balancing and minimize disruptions. However, the rise of the Internet of Things (IoT) has introduced new challenges for existing CH algorithms, characterized by high memory usage and update overhead. This article presents DxHash, a novel CH algorithm based on repeated pseudo-random number generation. DxHash offers significant benefits, including a remarkably low memory footprint, high lookup throughput, and minimal update overhead. Additionally, we introduce a weighted variant of DxHash, enabling adaptable weight adjustments to handle heterogeneous load distribution. Through extensive evaluation, we demonstrate that DxHash outperforms AnchorHash, a state-of-the-art CH algorithm, in terms of the reduction of up to 98.4% in memory footprint and comparable performance in lookup and update.

The address allocation policy in SSD aims to translate the logical address of I/O requests into a physical address, and the static address allocation is widely used in modern SSD. Through extensive experiments, we find that there are significant differences in the utilization of SSD parallelism among different static address allocation policies. We also observe that the fixed address allocation design prevents SSDs from continuing to meet the challenges posed by cloud workloads and misses the possibility of further optimization. These situations stem from our excessive reliance on SSD parallelism over time. In this paper, we proposeHsaP, ahybridstatic addressallocationpolicy, that adaptively chooses the best static allocation policy to meet the SSD performance at runtime.HsaPis adynamicscheduling scheme based onstaticaddress allocation policy. The static policy ensures thatHsaPhas stable performance and light-weight overhead, while dynamic scheduling can effectively combine different allocation policies, selecting the best-performing static mapping mode for a given SSD state. Meanwhile,HsaPcan further improve the read and write performance of SSDs simultaneously through plane reallocation and data rewrite. Experimental results show thatHsaPachieves significant read and write performance gain of a wide range of the latest cloud block storage traces compared to several state-of-the-art address allocation approaches.

Emerging cross-point memory can in-situ perform vector-matrix multiplication (VMM) for energy-efficient scientific computation. However, parasitic-capacitance-induced row charging and discharging latency is a major performance bottleneck of subarray VMM. We propose a memory-timing-compliant bulk VMM processing-using-memory design with row access and column access co-optimization from rethinking of read access commands and µ-op timing. We propose row-level-parallelism-adaptive timing termination mechanism to reduce tail latency of tRCD and tRP by exploiting row nonlinear charging and bulk-interleaved row-column-cooperative VMM access mechanism to reduce tRAS and overlap CL without increasing column ADC precision. Evaluations show that our design can achieve 5.03× performance speedup compared with an aggressive baseline.

Since multiple flows (users) compete for a single, shared solid state drives (SSDs) concurrently in cloud environments, fairness has drawn great interest in recent years. Although many fair schemes are proposed on different modules of SSDs, such as resource allocation for the cache and chip queue reordering for the transaction scheduling unit (TSU), they fail to continue to provide fairness for the worse locality of I/O requests due to a deeper cloud storage stack that makes cache performance poor. Moreover, existing methods only treat back-end flash as a black box and do not exploit the differences in flash features across requests. This paper proposes CoFS, a novel coordinated SSD cache and TSU light-weight learning-based fair scheme to achieve complete fairness considering cache and bandwidth resources which are significant for fairness control. The key idea is to make the front-end SSD cache achieves fairness at the workload-level by recognizing dynamic I/O changes, while the back-end TSU achieves fairness at the flash-level by awareness of flash page-types, and finally CoFS collaborates with them to achieve fairness at the SSD device-level. Experimental results show that CoFS achieves significant fairness and performance gain of a wide range of the latest cloud block storage workloads, with an average improvement of 46.3% and 94.5% compared to state-of-the-art fairness policies, respectively.

Hybrid memory systems show strong potential to satisfy the growing memory demands of modern applications by combining different memory technologies. Due to the different performance characteristics of hybrid memories, a data migration strategy that migrates hot data to a faster memory is critical to the overall performance. Prior works have focused on identifying hot data and migration decisions. However, we find that the fix-sized global migration granularity in existing data migration schemes results in suboptimal performance on most workloads. The key observation is that the optimal migration granularity varies with access patterns. This paper proposes AGDM, an access-pattern-aware Adaptive Granularity Data Migration strategy for hybrid memory systems. AGDM tracks memory access patterns in runtime and accordingly adopts the most appropriate migration mode and granularity. The novel remapping-migration decoupled metadata organization enables AGDM to set local optimal gran-ularities for memory regions with different access patterns. Our evaluation shows that, compared to the state-of-the-art scheme, AGDM gets an average performance improvement of 20.06% with 29.98% energy savings.

As SSD failures seriously lead to data loss and service interruption, proactive failure prediction is often used to improve system availability. However, the unidimensional SMART-based prediction models hardly predict all drive failures. Some other features applied in data centers and enterprise storage systems are not readily available in consumer storage systems (CSS). To further analyze related failures in production SSD-based CSS, we study nearly 2.3 million SSDs from 12 drive models based on a dataset of SMART logs, trouble tickets, and error logs. We discover that SMART, Firmware Version, WindowsEvent, and BlueScreenof Death (SFWB) are closely related to SSD failures. We further propose a multidimensional-based failure prediction approach (MFPA), which is portable in algorithms, SSD vendors, and PC manufacturers. Experiments on the datasets show that SFWB-based MFPA achieves a high true positive rate (98.18%) and low false positive rate (0.56%), which is 4% higher and 86% lower than the SMART-based model. It is robust and can con-tinuously predict for 2–3 months without iteration, substantially improving the system availability.

ReRAM-based Processing-In-Memory (PIM) architectures have demonstrated high energy efficiency and performance in deep neural network (DNN) acceleration. Most of the existing PIM accelerators for DNN focus on offline batch learning (OBL) which requires the whole dataset to be available before training. However, in the real world, data instances arrive in sequential settings, and even the data pattern may change, which calls concept drift. OBL requires expensive retraining to solve concept drift, whereas online deep learning (ODL) is evidenced to be a better solution to keep the model evolving over streaming data. Unfortunately, when ODL optimizes models over a large-scale data stream in the PIM system, unbalanced writes are more severe than OBL, due to the heavier weight updates, resulting in the amplification of unbalanced writes and lifetime deterioration. In this work, we propose ODLPIM, an online deep learning PIM accelerator that extends the system lifetime through algorithm-hardware co-optimization. ODLPIM adopts a novel write-optimized parameter update (WARP) scheme that reduces the non-critical weight updates in hidden layers. Besides, a table-based inter-crossbar wear-leveling (TIWL) scheme is proposed and applied to the hardware controller to achieve wear-leveling between crossbars for lifetime improvement. Experiments show that WARP reduces weight updates on average to 15.25% and up to 24% compared to that without WARP, and eventually prolongs system lifetime on average to 9.65% and up to 26.81%, with a negligible rise in cumulative error rate (up to 0.31%). By combining WARP with TIWL, the lifetime of ODLPIM is improved by an average of$\mathbf{12}.\mathbf{59}\times$and up to$\mathbf{17}.\mathbf{73}\times$.

Modern clustered storage systems have commonly used erasure coding to maintain data durability against failures, yet it introduces significant update overhead for partial updates (e.g., only part of a block is updated). Recent studies propose the append-commit and buffer-logging techniques, which append multiple updated data to buffer-log and cache the corresponding old data from disk into memory, to reduce the update costs when committing the updates to parity. However, caching the entire old data block will introduce additional disk reads and memory overhead because caching the old part that not be updated, caching the partial old data block will incur the significant disk seeks for frequent partial updates. Our real-cloud experiments show that the unbalanced disk I/O may cause the bottleneck for updating, which is unfortunately overlooked by existing studies.This paper proposes LASC, a locality-aware speculative cache scheme for partial updates. LASC perceives the update locality of a data block from an update request stream within a period and speculatively caches the old data from the disk. It caches the entire old data block with high update locality to reduce the disk seeks. For a series of update requests to the same data block, LASC only performs one disk seek. Otherwise, it caches the old partial data to migrate the disk reads and memory overhead. We evaluate LASC via trace-driven simulations and Alibaba ECS experiments for two of the largest and latest public block-level I/O traces and show that LASC can effectively balance the disk I/O and improve the update performance while keeping memory overhead low.

Distributed file systems (DFS) are the cornerstone of modern mass data processing systems. In DFS, the metadata service, as the core component, often becomes a performance bottleneck. Existing metadata service solutions have implemented flattened metadata management and full-path indexing to achieve high scalability in terms of capacity and throughput. However, these approaches have limitations, such as conflicts with POSIX-style permission verification and flawed support for super directories, leading to high and unstable latency that cannot provide reliable service for latency-sensitive applications. To overcome these limitations, we propose Duplex, a scalable DFS metadata service based on full-path indexing, which aims for low and stable latency. Duplex incorporates three key designs: a fast access path featuring a centralized permission server for efficient permission verification, a permission merging algorithm to reduce the PMS's space footprint, and flattened metadata management based on double consistent hashing that enables low-latency access to super directories. Our evaluations demonstrate that, compared to state-of-the-art metadata solutions, Duplex significantly reduces the average lookup latency by up to 84% and the 99th percentile tail latency by up to 88.2% for metadataintensive benchmarks. Additionally, Duplex improves the lookup IOPS by up to 7.6 × /2.3× compared to CephFS and BeeGFS.

Outsourcing storage to the cloud can relieve pressure on storage capacity, wherein encrypted deduplication is often used to perform deduplication on encrypted data, for ensuring both storage efficiency and security. However, encrypted deduplication only removes duplicate encrypted data without considering non-duplicate but very similar ones. In this paper, we propose EDelta, which is the first to enable encrypted delta compression so as to further reduce redundancy of similar encrypted data atop encrypted deduplication, by means of preserving similarity after encryption. We also propose two methods to speed up the encryption process of encrypted delta compression. We prototype EDelta in cloud environments and our experimental results based on real-world datasets show that EDelta can achieve an additional data reduction ratio by up to 45.0% on top of encrypted deduplication.

Hashing is a widely used and efficient indexing mechanism for key-value storage. Persistent memory (PM) has attracted extensive attention in research due to its non-volatility and DRAM-like performance. Intel DCPMM, as a PM, can provide large capacity and low total cost of ownership, further promoting the research of PM-based hash index. However, based on real-world workloads, we found that negative searches of existing PM-based hash indexes significantly degrade system performance. A direct method to solve this problem is to use a PM-based Bloom filter to reduce negative searches, but at the cost of the decreased lifespan of PM due to extra PM writes. An alternative method is to use a DRAM-based Bloom filter, but it still faces increased multi-threaded insertion/deletion/positive-search scalability overhead as well as increased data consistency and recovery overhead.In this paper, we propose SmartHT, a small-size DRAM-based Bloom filter to accelerate hash table operations for PM while solving the aforementioned problems. SmartHT uses efficient merge write optimization with head insertion, lazy deletion, and shortened average chained length of head-bucket to provide high insertion/deletion/positive-search scalability, respectively. On the other hand, it utilizes a merged-flush mechanism based on an 8-byte failure-atomic write method to reduce flush instructions and extra PM writes to achieve low data consistency overhead. Experimental results on Intel Optane DCPMM show that, compared with the state-of-the-art persistent hash indexes, SmartHT improves multi-threaded negative queries under uniform and skewed distributions by 4.61x-13.86x and 2.76x-12.99x respectively, achieves high multi-threaded scalability and low data consistency overhead, at the modest cost of recovery time overhead.

Applications powered by Deep Neural Networks (DNNs) are shifting from cloud to edge computing due to the promising performance efficiency of Resistive Random Access Memory (RRAM) devices. However, modern RRAM devices have various possible faults, including hard and soft faults. Existing RRAM-based deep learning systems lack a comprehensive fault tolerance approach, leading to safety risks in deploying edge-end DNN inference. In this paper, we propose an effective fault recovery framework called TOSA to safeguard edge-based DNN accelerators from Stuck-At-Faults (SAFs). Two novel ideas are presented to enhance reliability in edge-based RRAM inference accelerators: (1) SAFs-aware priority-based bit-memristor mapping algorithm to match integrated SAF variations and DNN’s defect tolerance, (2) a new retraining method called joint retraining to recover model accuracy from multiple faulty edge devices. We thoroughly evaluate TOSA on multiple DNN models, demonstrating its effectiveness in reducing costs and improving efficiencies. Experimental results show that TOSA achieves a 45.28× speedup when compared to the per-device retraining methods while preserving inference accuracy better than the latest competitor, making it a reliable architecture for accelerating DNN inference on edge devices.

Memory disaggregation architecture can achieve higher resource utilization, independent scaling of CPUs and memory. Disaggregated memory systems manage memory resources and locate data by distributed index. However, existing distributed indexes suffer from high synchronization overhead of write, due to naive node lock, thus resulting in high tail latency, low throughput, and poor scalability.

Cloud storage systems have commonly used erasure coding that encodes data in stripes of blocks as a low-cost redundancy method for data reliability. Relative to traditional erasure coding, wide-stripe erasure coding that increases the stripe size has been recently proposed and explored to achieve lower redundancy. We observe that wide-stripe erasure coding makes multi-block failures occur much more frequently than traditional erasure coding in cloud storage systems.However, how to efficiently repair multiple blocks in wide-stripe erasure-coded storage systems remains unexplored. The conventional multi-block repair method sends available blocks from surviving nodes to one single new node to repair all failed blocks in a centralized way, which may cause the new node to be the bottleneck; recent multi-block repair methods follow pipelined single-block repair methods and the former are simply built on the latter in an independent way, which may cause the surviving nodes with limited bandwidth to be bottlenecks.In this paper, we first analyze the effects of both centralized and independent ways on the multi-block repair and then propose HMBR, a hybrid multi-block repair mechanism that combines centralized and independent multi-block repairs to tradeoff the bandwidth bottlenecks caused by the new and surviving nodes, thus optimizing the multi-block repair performance. We further extend HMBR for hierarchical network topology and multi-node failures. We prototype HMBR and show via Amazon EC2 that the repair time of a multi-block failure can be reduced by up to 64.8% over state-of-the-art schemes.

Object cloud storage systems are deployed with diverse applications that have varying latency service level objectives (SLOs), posting challenges for supporting quality of service with limited storage resources. Existing methods provide prediction-based recommendations for dispatching requests from applications to storage devices, but the prediction accuracy can be affected by complex system topology. To address this issue, Graph3PO is designed to combine storage device queue information with system topological information for forming a temporal graph, which can accurately predict device queue states. Additionally, Graph3PO contains the urgency degree model and cost model for measuring SLO violation risks and penalties of scheduling requests on storage device queues. When the urgency degree of a request exceeds a threshold, Graph3PO determines whether to schedule it in the queue or initiate a hedge request to another storage device. Experimental results show that Graph3PO outperforms its competitors, with SLO violation rates 2.8 to 201.1 times lower.

Abstract Accurate resource allocation and colocating jobs are effective ways to increase resource utilization and reduce costs in modern datacenters. The main challenges are fluctuations in resource consumption and interference among colocated jobs. Therefore, we propose a resource management schema based on the probability distributions of resource consumption and completion time for multitenant cloud clusters. First, we found that the characteristics of the task can be well described by the probability distribution of resource consumption and completion time, and the probability distribution function can be obtained by Gaussian fitting. Second, we propose a probability distribution based resource allocation (PDRA) strategy for batch jobs. Third, we design a tail latency aware allocation (TLAA) strategy to use transient resources efficiently while ensuring tail latency requirements. Finally, we design a cost‐effective resource revocation (CERR) strategy to revoke transient resources with minimal eviction costs. Experimental results demonstrate the efficiency of our resource management. Our resource allocation strategies (PDRA and TLAA) can effectively improve resource utilization and reduce job completion time. CERR can reduce the impact of resource revocation on batch jobs and perform better than existing resource revocation strategies.

Graph embedding maps graph nodes to low-dimensional vectors, and is widely adopted in machine learning tasks. The increasing availability of billion-edge graphs underscores the importance of learning efficient and effective embeddings on large graphs, such as link prediction on Twitter with over one billion edges. Most existing graph embedding methods fall short of reaching high data scalability. In this paper, we present a general-purpose, distributed, information-centric random walk-based graph embedding framework, DistGER, which can scale to embed billion-edge graphs. DistGER incrementally computes information-centric random walks. It further leverages a multi-proximity-aware, streaming, parallel graph partitioning strategy, simultaneously achieving high local partition quality and excellent workload balancing across machines. DistGER also improves the distributed Skip-Gram learning model to generate node embeddings by optimizing the access locality, CPU throughput, and synchronization efficiency. Experiments on real-world graphs demonstrate that compared to state-of-the-art distributed graph embedding frameworks, including KnightKing, DistDGL, and Pytorch-BigGraph, DistGER exhibits 2.33×--129× acceleration, 45% reduction in cross-machines communication, and >10% effectiveness improvement in downstream tasks.

Disaggregated memory architecture has risen in popularity for large datacenters with the advantage of improved resource utilization, failure isolation, and elasticity. Replicated state machines (RSMs) have been extensively used for reliability and consistency. In traditional RSM protocols, each replica stores replicated data and has the computing power to participate in some part of the protocols. However, traditional RSM protocols fail to work in the disaggregated memory architecture due to asymmetric resources on CPU nodes and memory nodes. This article proposes ECHO, a fast one-sided RDMA-based RSM protocol with lightweight log replication and remote applying, efficient linearizability guarantee, and fast coordinator failure recovery. ECHO enables all operations in the protocol to be efficiently executed using only one-sided RDMA, without the participation of any computing resource in the memory pool. To provide lightweight log replication and remote applying, ECHO couples the replicated log and the state machine to avoid dual-copy and performs remote applying by updating pointers. To enable efficient remote log state management, ECHO leverages a hitchhiked log state updating scheme to eliminate extra network round trips. To provide efficient linearizability guarantee, ECHO performs immediate remote applying after log replication and leverages the local locks at the coordinator to ensure linear consistency. Moreover, ECHO adopts a commit-aware log cache to make data visible immediately after being committed. To achieve fast failure recovery, ECHO leverages a commit point identification scheme to reduce the overhead of log consistency recovery. Experimental results demonstrate that ECHO outperforms the state-of-the-art RSM protocol (namely Sift) in multiple scenarios. For example, ECHO achieves 27%–52% higher throughput on typical write-intensive workloads. Moreover, ECHO reduces the consistency recovery time by three orders of magnitude for coordinator failure.

The Zoned Namespace (ZNS) Solid State Drive (SSD) is a nascent form of storage device that offers novel prospects for the Log Structured Merge Tree (LSM-tree). ZNS exposes erase blocks in SSD as append-only zones, enabling the LSM-tree to gain awareness of the physical layout of data. Nevertheless, LSM-tree on ZNS SSDs necessitates Garbage Collection (GC) owing to the mismatch between the gigantic zones and relatively small Sorted String Tables (SSTables). Through extensive experiments, we observe that a smaller zone size can reduce data migration in GC at the cost of a significant performance decline owing to inadequate parallelism exploitation. In this article, we present SplitZNS, which introduces small zones by tweaking the zone-to-chip mapping to maximize GC efficiency for LSM-tree on ZNS SSDs. Following the multi-level peculiarity of LSM-tree and the inherent parallel architecture of ZNS SSDs, we propose a number of techniques to leverage and accelerate small zones to alleviate the performance impact due to underutilized parallelism. (1) First, we use small zones selectively to prevent exacerbating write slowdowns and stalls due to their suboptimal performance. (2) Second, to enhance parallelism utilization, we propose SubZone Ring, which employs a per-chip FIFO buffer to imitate a large zone writing style; (3) Read Prefetcher, which prefetches data concurrently through multiple chips during compactions; (4) and Read Scheduler, which assigns query requests the highest priority. We build a prototype integrated with SplitZNS to validate its efficiency and efficacy. Experimental results demonstrate that SplitZNS achieves up to 2.77× performance and reduces data migration considerably compared to the lifetime-based data placement. 1

Graph embedding is becoming widely adopted as an efficient way to learn graph representations required to solve graph analytics problems. However, most existing graph embedding methods, owing to computation-efficiency challenges for large-scale graphs, generally employ a one-size-fits-all strategy to extract information, resulting in a large amount of redundant or inaccurate representations. In this work, we propose HuGE+, an efficient and scalable graph embedding method enabled by an entropy-driven mechanism. Specifically, HuGE+ leverages hybrid-property heuristic random walk to capture node features, which considers both information content of nodes and the number of common neighbors in each walking step. More importantly, to guarantee information effectiveness of sampling, HuGE+ adopts two heuristic methods to decide the random walk length and the number of walks per node, respectively. Extensive experiments on real-world graphs demonstrate that HuGE+ achieves both efficiency and performance advantages over recent popular graph embedding approaches. For three downstream graph tasks, our approach not only offers >10% average gains, but also exhibits 23×–127× speedup over existing sampling-based methods. In addition, HuGE+ significantly reduces memory footprint by an average of 68.9%, facilitating training for billion-node-scale graph embeddings.

A significant drawback of erasure codes is suffering from the expensive update overhead, and all parity blocks are regenerated once any update of one data block for consistency. Existing update techniques either neglect the multi-block updates scenario under update-intensive workloads or do not consider how to minimize the update cost in heterogeneous clusters. This paper presents the first systematic study on multi-block updates in heterogeneous clusters. We formulate the problem as a cost-based routing optimization model and propose a novelMulti-block Double Tree Update(MDTUpdate) technique. The key idea is to construct a double tree structure for multiple updated data blocks and all parity blocks, which avoids the congested link bottleneck. To reduce the update costs effectively, we exploit a hybrid update scheme that combines the data-delta and parity-delta schemes under the double tree structure. To accelerate the tree construction, we design a time-efficient greedy algorithm that timely determines the transmission route via perceiving the cost discrepancy among nodes while avoiding exhaustive enumeration. We further prove that our algorithm is optimal in minimizing the update costs. The experiments show that MDTUpdate can improve the update performance by up to 83.23% over the existing techniques while incurring extremely lightweight running overhead.

Modern computing systems and data-intensive applications are eager for larger and faster memory. Building hybrid memory systems with different memory technologies has become a dominant trend to satisfy these demands. For hybrid memory systems, page migration schemes that dynamically migrate frequently accessed hot pages into faster memory are crucial for improving performance. However, existing migration schemes are either too aggressive, resulting in unnecessary extra traffic, or too conservative to quickly adapt to changes in access patterns. Besides, the extra latency introduced by querying metadata is often ignored or handled in an unscalable manner. In this article, we propose a relative hotness page migration (RHPM) strategy, which discovers hot pages in a set of pages by competing with each other rather than comparing with a threshold. The migration is performed only when a new page wins the competition. To overlap latency due to access metadata, RHPM fetches metadata and data in parallel. In addition, it enables a small metadata buffer to speed up metadata access. Compared to the state-of-the-art scheme, RHPM requires only 1/512 of the on-chip capacity, significantly reducing on-chip hardware overhead. Evaluation of RHPM with simulations of 25 workloads shows that RHPM outperforms the state-of-the-art scheme by an average of 13.34% in performance and saves 44.19% on energy, demonstrating better resilience to changes in access patterns.

Due to high static power and low scalability, the traditional SRAM-based cache is not a good solution for image processing applications. Emerging spin transfer torque magnetic RAM (STT-MRAM) is a promising candidate for cache due to its low leakage power and high density. However, STT-MRAM suffers from high write energy. Therefore, by making use of the ability of tolerating minor errors in image processing applications, this work presents an STT-MRAM-basedAPProximatecachearchitecture (APPcache+) to write/read approximate data, which can largely reduce the cache energy and improve the STT-MRAM lifetime. APPcache+ includes three main designs. First, we find that there are many similar elements (e.g., pixels in images) in cache lines. Therefore, APPcache+ presents several lightweight similarity-based encoding techniques to remove redundant elements, thus, shortening the data size and reducing the energy of STT-MRAM cache. Second, we design a partial read scheme to reduce the read energy of the STT-MRAM cache. In the traditional decompression process, the whole line is fetched into the decompressor, leading to unnecessary read energy. The partial read scheme can largely reduce read energy while keeping the overhead low. Third, we observe the encoding schemes may lead to bit write imbalance. Therefore, we propose a lightweight Ping-Pong intraline wear-leveling scheme to improve the lifetime. Compared with the baseline, extensive evaluation results show that our APPcache+ can largely reduce the overall energy by 32.58%, improve lifetime by 40.7% with only 2.2% performance degradation, and 1.86% output quality loss.

Spin-transfer torque magnetic random access memory (STT-MRAM) is a promising cache memory candidate due to its high density, low leakage power, and nonvolatility. Multilevel cell (MLC) STT-MRAM can further increase density by storing 2 bits in one cell’s hard and soft domain, respectively. However, MLC STT-MRAM suffers two-step write, leading to high write energy, long latency, and severe lifetime degradation. Current encoding techniques propose to encode the new data to reduce the two-step data writes. However, they have two weaknesses: 1) high area overhead, e.g., recent work TSE (Hsieh et al., 2020) needs extra 37.5% MLCs and 2) prolong the write latency due to an extra read. Therefore, we propose enhanced one-step write (EOSwrite) to write data in one step. EOSwrite includes line bypassing and four intraline encoding techniques. Line bypassing schemes can bypass the writes to zero or clean lines, leading to low write/read latency. As for the intraline techniques, we propose four write modes. They utilize the data patterns and the clean data in cache lines to write data in one step, therefore reducing the data write latency. The key idea of one-step write is to write as much data as possible in the soft domain of MLC STT-MRAM. EOSwrite can greatly relieve the weaknesses of the current encoding schemes. Evaluation results show that EOSwrite can improve the lifetime of MLC STT-MRAM by 56.96%, reduce dynamic energy by 33.95%, reduce access latency by 36.95%, and improve system performance of MLC STT-MRAM by 4.30%, respectively. While the area overhead of EOSwrite is only 7.27%.

Social network events are related to the national economy and people’s livelihood, so timely perception and processing of massive social network events data are becoming increasingly important for public opinion analysis. How to fully exploit the access feature of event information to store and manage social network events is of great significance for accurate and real-time query analysis. We propose a social network event storage management method based on microblog text clustering and hot/cold data classification. First, for the microblog text data, we construct a keyword provenance graph by using the information entropy to measure the weight of the edge between keyword nodes. Then, we cluster the events using the provenance-based community partition (PCP) with local modularity to improve the event clustering accuracy. In addition, we can further filter noisy data via incremental clustering, enable hot/cold event data classification and dynamic migration, and compress cold data to save space on a hybrid storage architecture. The experimental results show that the clustering purity can reach more than 93% and the query time can be reduced by more than 70% using clustering and hybrid storage policy.

Identifying intrusion from massive and multi-source logs accurately and in real-time presents challenges for today's users. This article presents Paradise, a real-time, generalized, and distributed provenance-based intrusion detection method. Paradise introduces a novel extract strategy to prune and extract process feature vectors from provenance dependencies at the system log level, and it stores them in high-efficiency memory databases. Using this strategy, Paradise does not depend on the specific operating system type or provenance collection framework. Provenance-based dependencies are calculated independently during the detection phase, thus, Paradise can negotiate all detection results from multiple detectors without extra communication overhead between detectors. Paradise also employs an efficient load-balanced distribution scheme that enhances the Kafka architecture to efficiently distribute provenance graph feature vectors to the detectors. The experimental results demonstrate that our method has a high detection accuracy with a low time overhead.

The emergence of containers dramatically simplifies and facilitates the development and deployment of applications. More and more enterprises deploy their applications on the container cloud platform. For cloud service providers, an effective container workload prediction method is a must to achieve efficient utilization of cloud resources. However, the existing methods are either rarely based on container load characteristics or cannot make accurate real-time predictions. In this paper, we propose a Docker container workload proactive prediction method using a hybrid model combining triple exponential smoothing and long short-term memory (LSTM), which not only can capture both short-term and long-term dependencies in container resource time series but also smooth the container resource utilization data. In order to improve the prediction accuracy of the hybrid model, those two single models are combined using the mean absolute percentage error (MAPE) method. Besides, we design a real-time Docker workload prediction system for the hybrid model. Our experiments show that the mean absolute percentage error of the hybrid model is decreased by an average of 3.24%, 12.18%, 13.42%, 43.45%, and 50.69% compared with the LSTM, the triple exponential smoothing, ES-ARIMA, Bayesian Ridge Regression and BiLSTM with an acceptable time and computational cost overhead.

Large-scale storage systems have introduced erasure codes to guarantee high data reliability, yet inevitably at the expense of high repair costs. In practice, storage nodes are usually divided into different racks, and data blocks in nodes are organized into multiple stripes independently manipulated by erasure code. Due to the scarcity and heterogeneity of the cross-rack bandwidth, the cross-rack transmission dominates the entire repair costs. When erasure code is deployed in rack architectures, existing repair techniques are limited in different aspects: neglecting the heterogeneous cross-rack bandwidth, less consideration for multi-stripe failure, and no special treatment on repair-link scheduling. In this paper, we present CMRepair, aCross-rack Multi-stripe Repairtechnique that aims to reduce the repair time for multi-stripes failure repair in heterogeneous erasure-coded clusters. CMRepair first carefully chooses the nodes for reading/repairing blocks and searches for the multi-stripe repair solution. It adopts different algorithms to adjust the solution, including the Computation Time Priority (CTP) algorithm based on the greedy idea and the Repair Time Priority (RTP) algorithm based on the meta-heuristics idea. Furthermore, CMRepair selectively schedules the execution orders of cross-rack links, with the primary objective of saturating the unused upload/download bandwidth resources and avoiding network congestion. The experiments show that CMRepair with the CTP algorithm can reduce 27.59%-58.12% of the repair time while only introducing negligible computation overhead, and CMRepair with the RTP algorithm can reduce 33.52%-97.75% of the repair time in an acceptable computation time, over existing repair techniques.

As the principal failure in data centers, disk failure may pose the risk of data loss, increase the maintenance cost, and affect system availability. As a proactive fault tolerance technology, disk failure prediction can minimize the loss before a failure occurs. Whereas, a weak prediction model with a low Failure Detection Rate (FDR) and high False Alarm Rate (FAR) may substantially increase the system cost due to inadequate consideration or misperception of the misclassification cost. To address these challenges, we propose a cost-sensitive learning engine CSLE for disk failure prediction, which combines a two-phase feature selection based on Cohen's D and Genetic Algorithm, a meta-algorithm based on cost-sensitive learning, and an adaptive optimal classifier for heterogeneous and homogeneous disk series. Experimental results on real datasets show that the AUC of CSLE is increased by 2%-42% compared with the commonly used rank-sum test. CSLE can reduce the misclassification cost by 52 %-96 % compared with the rank model. Besides, CSLE has a better pervasiveness than the traditional prediction model, it can reduce both the misclassification cost and the FAR by 16%-70% for heterogeneous disk series, and increase the FDR by 3%-29% for homogeneous disk series.

More and more commodity storage systems use erasure code to tolerate faults. When reconsidering the recovery problem in a multi-stripe level and heterogeneous network for the erasure-coded cluster, efficiently generating an optimal multi-stripe recovery solution that reduces recovery time remains a challenging and time-consuming task. Previous works either use a greedy algorithm that may fall into the local-optimal and has low performance or use a meta-heuristic algorithm with a long search time and low efficiency.In this paper, we propose a Stripe-schedule Aware Repair (SARepair) technique for multi-stripe recovery in heterogeneous erasure-coded clusters. By carefully examining the metadata of blocks, SARepair intelligently adjusts the recovery solution for each stripe and obtains another multi-stripe solution with less recovery time in a computationally efficient manner. It then tolerates worse solutions to overcome the local-optimal and uses a rollback mechanism to adjust search regions to further reduce recovery time. Moreover, instead of reading blocks sequentially from each node, SARepair also selectively schedules the reading order for each block to reduce the memory overhead. We prototype SARepair and show via both simulations and Amazon EC2 experiments that the recovery time can be reduced by up to 68% over a state-of-the-art recovery approach while keeping time complexity and memory overhead low.

The byte-addressable persistent memory (PM) is coming to be the next-generation storage device for better I/O performance. As the traditional I/O path is too lengthy to drive PM featuring low latency and high bandwidth, prior works have proposed memory-mapped I/O (MMIO) to shorten the I/O path to PM. However, native MMIO directly maps files into the user address space, which puts files at risk of user-space scribbles and non-atomic I/O interfaces, termed reliability issues. Since existing reliability schemes cause significant extra overheads, we propose RMMIO, an efficient user-space library that provides reliable memory-mapped I/O interfaces for PM systems. RMMIO achieves a good balance between efficiency and reliability by introducing a memory-mapped cache layer upon kernel file systems. The cache layer accelerates I/O requests and carries the file system’s responsibility for data reliability by data isolation. In addition, RMMIO further employs lightweight snapshots and efficient atomic I/O interfaces to guarantee the integrity and consistency of the data in the cache layer at low costs. The experimental results show that RMMIO achieves 8.49x higher throughput than ext4-DAX and 2.31x higher throughput than state-of-the-art MMIO-based schemes for PM while ensuring data reliability.

As a vital method for computer system design, multilevel cache technology is still a research hotspot in the field of storage. Recently, researchers have designed many multilevel cache replacement algorithms by analyzing the historical trajectories of data blocks between different cache levels but suffer from high overhead and poor adaptability. In addition, these methods still try to determine the most suitable victim to be replaced, given a new block to be loaded into the cache but do not consider how to cooperate between different cache levels. To overcome these challenges, we propose a multi-factor adaptive multi-level cooperative cache replacement policy in block storage systems that leverages the hierarchical characteristics of multilevel cache and multi-factor orthogonality by the probability distribution. This adaptation is mainly reflected in two aspects: each cache level automatically adapts different cache policies based on I/O access patterns, and different cache levels cooperate by adjusting the dynamic frequency threshold. In this work, we choose as our target for optimization the replacement policy of the cloud block cache, because rich storage stacks in cloud environments can be well used as multi-level cache scenarios. We have evaluated our proposed framework by using block-based I/O traces collected from Alibaba Cloud, one of the largest cloud providers in the world, and several open-source traces. Experimental results show that our method improves the hit ratios by 28.9% and reduces the response time by 9.6% to the state-of-the-art cache replacement policies for multi-level cache across different workloads and cache sizes.

Erasure coding is commonly used as a storage-efficient redundancy method for fault tolerance in cold storage. Recent studies have begun to explore the use of erasure coding in hot storage, which requires fast online recovery to preserve read performance. However, existing erasure-coded repair strategies cannot effectively handle frequent and rapidly-changing network congestions in hot storage clusters. In this paper, we present the notion of pivots, which refer to the storage nodes with sufficient available downlink and uplink bandwidths in a congested hot storage network. We propose PivotRepair, a pivot-based pipelined single-chunk repair technique that leverages pivots for enabling the fast construction of a pipelined repair tree that bypasses congested links. We further propose an adaptive scheduling strategy to improve full-node repair performance. We prototype PivotRepair and show that the repair time of a single-chunk repair and a full-node repair can be reduced by up to 71.27% and 16.50%, respectively, over state-of-the-art repair schemes.

Data security is a key issue that non-volatile memory (NVM) system designers must consider. However, this is challenging because implementing security mechanisms such as bonsai merkle tree (BMT) in NVM needs to ensure crash recovery due to the non-volatile property of NVM. Existing schemes fail to efficiently guarantee the atomic BMT root update and instant system recovery required for BMT crash recovery, resulting in large write traffic and performance overhead. In this paper, we propose DR-TREE, a dynamic and recoverable BMT scheme for secure NVM, which reduces the update overhead of BMT root and achieves fast crash recovery with low write traffic. DR-TREE dynamically builds BMT and adjusts the updated BMT levels according to memory write requests, thus reducing unnecessary update overhead of BMT root. Next, based on the locality of memory write requests, DR-TREE merges repeated BMT updates, further decreasing the update overhead of BMT root. Moreover, DR-TREE achieves fast crash recovery with extremely low write traffic by delaying the partial recovery process. Experiments show that compared to the state-of-the-art design, DR-TREE improves the performance by 44.6%, decreases write traffic by 78.2% and achieves the system recovery in 5ms.

High-density storage servers (HDSSes), which pack many disks into single servers, are currently used in data centers to save costs (power, cooling, etc). Erasure coding, which stripes data and provides high availability guarantees, is also commonly deployed in data centers at lower cost than replication. However, when applying erasure coding to a single HDSS, we find that erasure coding’s state-of-the-art studies that improve repair performance in parallel mainly use multiple servers’ sufficient footprint, which is yet quite limited in the single HDSS, thus leading to a memory-competition issue for disk failure recovery.

Large-scale distributed storage systems have introduced erasure code to guarantee high data reliability, yet inevitably at the expense of high repair costs. In practice, storage nodes are usually divided into different racks, and data blocks in storage nodes are often organized into multiple stripes independently manipulated by erasure code. Due to the scarcity and heterogeneity of the cross-rack bandwidth, the cross-rack network transmission dominates the entire repair costs. We argue that when erasure code is deployed in a rack architecture, existing repair techniques are limited in different aspects: neglecting the heterogeneous cross-rack bandwidth, less consideration for multi-stripe failure, no special treatment on repair link scheduling, and only targeting specific erasure code constructions.

Persistent memory (PM) exhibits a huge potential to provide B+-tree indexes with high performance, efficient persistence, and instant recovery. A large number of PM-optimized B+-tree indexes have been proposed, but most of them fail to provide high performance for both read and write operations because: (1) their designs of search optimization and insert improvement are often traded off against each other, and (2) they overlook the read/write interference problem of PM which incurs unpredictable performance degradation. In this paper, we propose ROWE-tree, a read-optimized and write-efficient B+-tree for PM. The designs of our ROWE-tree consist of three key points. First, we propose two techniques to make a good trade-off between write and read performance: self-verifying insertion, which reduces consistency overhead by using the key itself as a persist mark instead of additional metadata, and semi-sorted leaf nodes, which use append-only insertion to avoid the shifting overhead of sorting nodes but keep intra-cache-line items sorted to accelerate the lookup. Second, based on the observation that data accesses are highly skewed in real-world workloads, we build an in-DRAM cache of hot items to outsource accesses to hot items to DRAM. By doing so, we can alleviate the read/write interference of PM and significantly improve overall performance. Third, to cope with the dynamic changes of hot items, we exploit a lightweight mechanism to track such changes at run-time. Using Intel Optane DCPMM, our evaluations show that ROWE-tree obtains up to 3.86 × higher performance than the state-of-the-art PM B+-tree indexes under YCSB workloads.

Abstract Data deduplication is a widely used technique to remove duplicate data to reduce the storage overhead. However, deduplication typically cannot eliminate the redundancy among nonidentical but similar data chunks. To reduce the storage overhead further, delta compression is often applied to compress the post‐deduplication data. While the two techniques are effective in saving storage space, they introduce complex references among data chunks, which inevitably undermines the system reliability and introduces fragmentation that may degrade the restore performance. In this paper, we observe that the delta compressed chunks (DCCs) are much smaller than regular chunks (non‐DCCs). Also, most fragmentation caused by the base chunk of DCCs remain fragmented in consecutive backups. Based on these observations, we introduce a framework called , which combines replication and erasure coding and uses History‐aware Delta Selection to ensure high reliability and restore performance. Specifically, uses a delta‐utilization‐aware filter and a cooperative cache scheme (CCS) to maintain cache locality and avoid unnecessary container reads, respectively. Moreover, the system selectively performs delta compression by historical information to avoid cyclic fragmentation in consecutive backups. Experimental results based on four real‐world datasets demonstrate that significantly improves the restore performance by 58.3%–76.7% with a low storage overhead.

Abstract Flash‐based key‐value (KV) caching has received increasing attention in recent years with the advantages of flash‐based solid state drives (SSDs) in capacity and cost. By caching most data in SSD, the caching system can eliminate lots of time‐consuming requests to back‐end data stores to provide low‐latency services. To adapt to the unique technical constraints of flash memory, flash‐based KV caching adopts a slab‐based log‐structured management scheme in which the slab is the basic storage unit, and uses a small memory space as a write buffer to eliminate small random writes to SSD for consistent performance and increased lifetime of SSD. However, we have observed that under update‐intensive workloads with strong temporal locality, the slab‐based management in flash‐based KV caching introduces substantial SSD write traffic because of indistinguishable SSD flushing of hot items in slabs, which shortens the SSD lifetime and degrades the performance with increased erase operations. In this article, we first analyze the SSD write traffic in the flash‐based KV caching, and then propose a novel slab popularity‐based storage management scheme‐SPOPB, to extend SSD lifetime and improve performance. Our scheme identifies hot items using a self‐adaptive threshold to reorganize and classify slabs with both the hotness and size of items. Then SPOPB filters and retains the popular slabs containing hot items in the write buffer with redesigned replacement policy to reduce the SSD write traffic. Our experiments show that our design can effectively reduce the SSD write traffic by 63.6%, the erase counts by 55.6%, and improve the performance by 42%.

Abstract The evolution of persistent memory (PM) has significantly affected the design of today's indexing structures. Hashing‐based structures are widely used in storage systems to achieve fast query responses. Recently, several concurrent and failure‐atomic hashing schemes for PM have been proposed to improve the scalability. However, these works still suffer from limited scalability, especially under write‐intensive workloads or at a high number of threads. Our empirical study concludes three issues harm the scalability of PM hashing schemes: the NUMA effects, the resizing operations, and the inter‐thread interference overhead in PM. Based on the above scalability issues, we present SPHT, a scalable and persistent hashing scheme for hybrid DRAM‐PM memory. To eliminate the NUMA effects, SPHT maintains a hash subtable for every NUMA node and stores the key‐value items in the designated nodes. By doing so, all operations can be executed in the local memory without cross‐node communication. The hash subtable consists of two components: the search layer in DRAM for fast accesses and the data layer in PM for efficient persistence. The data layer is organized in a log‐structured way and its log chunks own separate PM space, thus avoiding the resizing operations in PM. Meanwhile, the compacted log structure also supports batching multiple small items, which effectively reduces the persistence overhead. Furthermore, to maximize concurrency, we assign threads to different partitions in the data layer to reduce the inter‐thread interference overhead. On Intel Optane DCPMM, our evaluations show that SPHT scales well and achieves up to 2.7 higher performance than state‐of‐the‐art PM hashing schemes under YCSB workloads.

Data security is an indispensable part of non-volatile memory (NVM) systems. However, implementing data security efficiently on NVM is challenging, since we have to guarantee the consistency of user data and the related security metadata. Existing consistency schemes ignore the recoverability of the SGX style integrity tree (SIT) and the access correlation between metadata blocks, thereby generating unnecessary NVM write traffic. In this article, we propose SecNVM, an efficient and write-friendly metadata crash consistency scheme for secure NVM. SecNVM utilizes the observation that for a lazily updated SIT, the lost tree nodes after a crash can be recovered by the corresponding child nodes in NVM. It reduces the SIT persistency overhead through a restrained write-back metadata cache and exploits the SIT inter-layer dependency for recovery. Next, leveraging the strong access correlation between the counter and DMAC, SecNVM improves the efficiency of security metadata access through a novel collaborative counter-DMAC scheme. In addition, it adopts a lightweight address tracker to reduce the cost of address tracking for fast recovery. Experiments show that compared to the state-of-the-art schemes, SecNVM improves the performance and decreases write traffic a lot, and achieves an acceptable recovery time.

Three-dimensional (3-D) integrated cross-point memory arrays can be used to build high-density storage-class memory systems. However, the coupled network topology caused by sharing word lines or bit lines between adjacent memory layers significantly enlarges the memory space overhead and time cost of memory operation simulations on mega-scale 3-D cross-point memory arrays. We observe that different components of the 3-D cross-point array have different contribution significance to the key metrics of write or read operations, and the distribution patterns of principal components in the arrays are different for write and read operations. We propose an operation-adaptive array modeling framework that exploits the impact of the applied operation on the distribution of array principal components for 3-D cross-point memory arrays. Based on the modeling framework, we propose two array network compaction methods for efficient write and read operation simulations on 3-D cross-point arrays, respectively: 1) pruning zero-biased unselected cells and 2) group-merging neighboring half-selected cells with a specific granularity. Also, serially connected line segments are merged, and floating segments are deleted. Evaluations show that the proposed methods significantly reduce the memory space overhead and time cost of memory operation simulations for various layer sizes and different cell-level access parallelism in an array. Besides, the proposed methods can efficiently simulate memory operations on multilayered 3-D cross-point memory arrays with up to$4096\times 4096$layer size under the 16-GB memory space constraint, achieving a 64-times improvement in layer size that can be simulated compared with the conventional complete 3-D cross-point array network model.

As we all know, file and free space fragmentation negatively affect file system performance. F2FS is a file system designed for flash memory. However, it suffers from severe fragmentation due to out-of-place updates and the highly synchronous, multithreaded writing behaviors of applications. The adaptive reserved space (ARS) scheme chooses some files instead of all files to update in the reserved space which collects file features associated with fragmentation to construct datasets and uses decision trees to pick reserved files. However, ARS processes file features according to writefds. On the one hand,fdsand files are mapped many-to-many, so there is an issue in mapping reserved files according to reservedfdsthat a normal file is selected as a reserved file. On the other hand, the mapping of files tofdsfor big data traces is confusing. We propose ARST that traces the file of writefdto optimize ARS. Moreover, by selecting time-independent file features, ARST predicts whether files with little historical information are reserved to maximize the performance improvement brought by reserving space. Besides, adjustable reserved space and dynamic reservation strategies are adopted. We implement ARST on a HiKey960 development platform and a commercial smartphone with slight space and file creation time overheads. Experimental results show that ARST reduces file and free space fragmentation dramatically and improves file system performance. ARST reduces the running time of realistic workloads by up to 94.28% than F2FS with in-place updates and outperforms ARS by up to 49.06% for Wechat.

In distributed data stores, performance fluctuations generally occur across servers, especially when the servers are deployed in a cloud environment. Hence, the replica selected for a reading request will directly affect the response latency. However, replica selection is challenging in latency-critical data stores (e.g., key-value stores). Such data stores generally deal with small size data, and clients have to select replicas independently. Even the state-of-the-art algorithm of replica selection (C3) still has considerable room for improving the latency. According to our experiments, compared with C3, using the ideal replica selection (Oracle) reduces the 99th latency by about 34-60 percent. In this article, we propose NetRS to address the fundamental factors that prevent replica selection algorithms from being effective. NetRS is a framework that enables in-network replica selection for distributed data stores. It exploits emerging network devices, including programmable switches and network accelerators, to select replicas for requests. NetRS supports diverse algorithms of replica selection and is suited to the network topology of modern data centers. According to our extensive evaluations, compared with the conventional scheme of clients selecting replicas for requests, NetRS reduces the mean latency by up to 50.3 percent, and the 99th latency by up to 56.3 percent. Moreover, NetRS could effectively cut the response latency even when unexpected events (e.g., workload changes, network device failures), and network congestion occur.

More and more enterprises are beginning to use Docker containers to build cloud platforms. Predicting the resource usage of container workload has been an important and challenging problem to improve the performance of cloud computing platform. The existing prediction models either incur large time overhead or have insufficient accuracy. This article proposes a hybrid model of the ARIMA and triple exponential smoothing. It can accurately predict both linear and nonlinear relationships in the container resource load sequence. To deal with the dynamic Docker container resource load, the weighting values of the two single models in the hybrid model are chosen according to the sum of squares of their predicted errors for a period of time. We also design and implement a real-time prediction system that consists of the collection, storage, prediction of Docker container resource load data and scheduling optimization of CPU and memory resource usage based on predicted values. The experimental results show that the predicting accuracy of the hybrid model improves by 52.64, 20.15, and 203.72 percent on average compared to the ARIMA, the triple exponential smoothing model and ANN+SaDE model respectively with a small time overhead.

Container-based virtualization has gradually become a main solution in today‘s cloud computing environments. Detecting and analyzing anomaly in containers present a major challenge for cloud vendors and users. This paper proposes an online container anomaly detection system by monitoring and analyzing multidimensional resource metrics of the containers based on the optimized isolation forest algorithm. To improve the detection accuracy, it assigns each resource metric a weight and changes the random feature selection in the isolation forest algorithm to the weighted feature selection according to the resource bias of the container. In addition, it can identify abnormal resource metrics and automatically adjust the monitoring period to reduce the monitoring delay and system overhead. Moreover, it can locate the cause of the anomalies via analyzing and exploring the container log. The experimental results demonstrate the performance and efficiency of the system on detecting the typical anomalies in containers in both simulated and real cloud environments.

Disk failure has always been a major problem for data centers, leading to data loss. Current disk failure prediction approaches are mostly offline and assume that the disk labels required for training learning models are available and accurate. However, these offline methods are no longer suitable for disk failure prediction tasks in large-scale data centers. Behind this explosive amount of data, most methods do not consider whether it is not easy to get the label values during the training or the obtained label values are not completely accurate. These problems further restrict the development of supervised learning and offline modeling in disk failure prediction. In this article, Active Semi-supervised Learning Disk-failure Prediction ( ASLDP ), a novel disk failure prediction method is proposed, which uses active learning and semi-supervised learning. According to the characteristics of data in the disk lifecycle, ASLDP carries out active learning for those clear labeled samples, which selects valuable samples with the most significant probability uncertainty and eliminates redundancy. For those samples that are unclearly labeled or unlabeled, ASLDP uses semi-supervised learning for pre-labeled by calculating the conditional values of the samples and enhances the generalization ability by active learning. Compared with several state-of-the-art offline and online learning approaches, the results on four realistic datasets from Backblaze and Baidu demonstrate that ASLDP achieves stable failure detection rates of 80–85% with low false alarm rates. In addition, we use a dataset from Alibaba to evaluate the generality of ASLDP . Furthermore, ASLDP can overcome the problem of missing sample labels and data redundancy in large data centers, which are not considered and implemented in all offline learning methods for disk failure prediction to the best of our knowledge. Finally, ASLDP can predict the disk failure 4.9 days in advance with lower overhead and latency.

Erasure codes offer a storage-efficient redundancy mechanism for maintaining data availability guarantees in storage clusters, yet also incur high network traffic consumption and recovery time in failure repair. Extensive research has been carried out to reduce the recovery time. However, previous works either target specific erasure code constructions which are not commonly used in today’s distributed storage clusters or neglect the heterogeneous bandwidth property in real network environments. Since erasure-coded clusters are typically composed of multi-node with heterogeneous bandwidth and accessed in parallel, the whole recovery time is mainly restricted by the low-bandwidth links. In this article, we propose SMFRepair, a single-node multi-level forwarding repair technique that is designed to improve the performance in heterogeneous networks based on Reed-Solomon codes for general fault tolerance. SMFRepair carefully selects the helper nodes and uses idle nodes to bypass low-bandwidth links. Idle nodes have sufficient and unused network bandwidth. It also pipelines the repair links that are optimized by idle nodes. Furthermore, a multi-node scheduling repair technique, called MSRepair, is proposed. MSRepair carefully schedules the multi-node repair link to saturate the most unoccupied bandwidth and transfers data from as large-bandwidth links as possible, with the primary objective of minimizing the recovery time. Large-scale simulation and Amazon EC2 real experiments show that compared to state-of-the-art repair techniques, SMFRepair can accelerate the single-node recovery by up to 47.69%, and MSRepair can reduce the multi-node recovery time by 33.78%$\sim$67.53%.

Approximate computing applications lead to large energy consumption and performance demand for the memory system. However, traditional SRAM based cache cannot satisfy these demands due to high leakage power and limited density. Spin Transfer Torque Magnetic RAM (STT-MRAM) is a promising candidate of cache due to low leakage power and high density. However, STT-MRAM suffers from high write energy. To leverage the ability of tolerating acceptable quality loss via approximations to data, we propose an STT-MRAM based APProximate cache architecture (APPcache) to write/read approximate data thus largely reducing energy. We find many similar elements (e.g. pixels in images) existing in cache lines while running approximate computing applications. Therefore, APPcache uses several lightweight similarity-based encoding schemes to eliminate the similar elements to reduce the data size thus reducing the write energy of STT-MRAM based cache. Besides, we design a software interface to manually control the output quality. APPcache can significantly eliminate similar elements, thus improving energy efficiency. Experimental results show that our scheme can reduce write energy and improve the image raw data compression ratio by 21.9% and 38.0% compared with the state-of-the-art scheme with 1 % error rate, respectively. As for the output quality, the losses of all benchmarks are within 5% with 1 % error rate.

Memory encryption can enhance the security of Non-volatile memories (NVMs), but it significantly increases the data bits written to NVMs and leads to severe lifetime and performance degradation. Current encryption techniques aim to reduce the re-encryption to many existing clean words, which unfortunately suffer from high encryption overheads (i.e. latency and energy) and many unnecessary writes. In the meantime, compression techniques can reduce the writes of encrypted NVM. However, we find that they may destroy the data patterns and increase the modified words, resulting in many encryptions in secure NVM. In this paper, we propose the MORphable Encryption and Encoding (MORE2) scheme to address these problems. Our MORphable Encryption (MORE) technique aims to reduce the full-line re-encryption and avoid clean line encryption. Besides, MORE proposes a prediction-based write scheme to avoid the encryption of clean lines, and pre-encrypt the lines that are predicted as dirty. Therefore, MORE can remove the encryption from the critical path of NVM. Furthermore, MORE2proposes the Morphable Selective Encoding (MSE) scheme to compress the modified words while preserving clean words. MORE2encrypts all metadata with the line counter to guarantee high security. Experimental results show that MORE2reduces the bit flips of encrypted NVM by 53.5 %, decreases the access latency by 27.32%, improves the IPC performance by 12.1 %, and reduces the write energy by 29.1 % compared with the state-of-the-art design.

Erasure coding has been widely deployed in modern large-scale storage systems for storage-efficient fault tolerance by storing stripes of data and parity chunks. Recently, enterprises explore the notion of wide stripes to suppress the fraction of parity chunks in each stripe to achieve extreme storage savings. However, how to efficiently generate wide stripes remains a non-trivial issue. In particular, re-encoding the currently stored stripes (termed narrow stripes) into wide stripes triggers substantial bandwidth overhead in relocating and regenerating the chunks for wide stripes. We propose StripeMerge, a wide-stripe generation mechanism that selects and merges narrow stripes into wide stripes, with the primary objective of minimizing the wide-stripe generation bandwidth. We prove the existence of an optimal scheme that does not incur any data transfer for wide-stripe generation, yet the optimal scheme is computationally expensive. To this end, we propose two heuristics that can be efficiently executed with only limited wide-stripe generation bandwidth overhead. We prototype StripeMerge and show via both simulations and Amazon EC2 experiments that the wide-stripe generation time can be reduced by up to 87.8% over a state-of-the-art storage scaling approach.

Graph embedding is becoming widely adopted as an efficient way to learn graph representations required to solve graph analytics problems. However, most existing graph embedding methods, owing to computation-efficiency challenges for large-scale graphs, generally employ a one-size-fits-all strategy to extract information, resulting in a large amount of redundant or inaccurate representations. In this work, we propose HuGE, an efficient and scalable graph embedding method enabled by an entropy-driven mechanism. Specifically, HuGE leverages hybrid-property heuristic random walk to capture node features, which considers both node degree and the number of common neighbors in each walking step. More importantly, to guarantee information effectiveness of sampling, HuGE adopts two heuristic methods to decide the random walk length and the number of walks per node, respectively. Extensive experiments on real-world graphs demonstrate that HuGE achieves both efficiency and performance advantages over recent popular graph embedding approaches. For link prediction and multi-label classification, our approach not only offers >10% average gains, but also exhibits 22×-126× speedup compared with existing sampling-based methods.

For performance benefits, recent trends of modern data centers tend to use NVM (Non-volatile Memory) as storage and utilize one-sided primitives of RDMA (Remote Direct Memory Access) to directly access NVM for I/O requests. However, one-sided RDMA doesn’t have durability semantics currently, and data may partially exist in the NVM when crashes happen, thus causing unexpected data inconsistency.

Data security is a very important issue in non-volatile memory (NVM) systems. Due to high security and low decryption latency, counter mode encryption (CME) is often used, which unfortunately suffers from extra efforts to guarantee the crash consistency of counters. Existing schemes fail to efficiently guarantee the crash consistency of counters and data, thus resulting in inefficient recovery and high costs. In this paper, we propose a Crash-Consistency-Aware Encryption scheme (CCAE) for NVM systems. CCAE is inspired by two key observations. First, we observe that all used log entries in a transaction have the same number of encryption times before being recycled due to the write appending feature. This motivates our shared counter optimization for log encryption. Second, we observe that the counters related to uncommitted data blocks only need to be restored to the latest or newer value before the crash to avoid OTP reuse. This motivates our delayed counter persistency scheme for data encryption. Evaluation results show that compared with the state-of-the-art design, CCAE reduces NVM write traffic caused by counters by 67%, improves system performance by 14%, and decreases NVM energy consumption by 35%.

It is common to deploy multiple workloads in one cluster to achieve high resource utilization, which tends to bring more resource contentions and performance interferences. If the allocable resources cannot satisfy the resource requirements of a task, the task should wait for resources, significantly increasing its scheduling latency. The inappropriate resource requirements may make a running task become a swollen task or a straggler task, which makes many allocated resources underutilized or the task be processed slowly. Therefore, how to guarantee the QoS of various services in the mixed workload deployment cluster is a challenge. Existing solutions preempt the resources from batch jobs to guarantee the resource requirements of latency-sensitive tasks without taking into account the underutilized resources in swollen tasks, which inevitably compromises the performance of batch jobs. Thus, we try to meet the resource requirements of newly incoming latency-sensitive tasks and straggler tasks with the underutilized resources instead of directly preempting the resources of batch jobs.

Erasure codes offer a storage-efficient redundancy mechanism for maintaining data availability guarantees in storage clusters, yet also incur high network traffic consumption and recovery time in failure repair. Exiting studies aim to reduce the recovery time in the heterogeneous network. However, the recovery time is always limited by the link with the low bandwidth between nodes, due to bandwidth heterogeneity. Recently, Bai et al. proposed a parallel pipeline tree technique, called PPT, to reduce recovery time by utilizing a special bandwidth gap to bypass the low-bandwidth link. But we find that PPT’s gap-based bypassing method will cause network congestion and competition. In this paper, we propose SMFRepair, a single-node multi-level forwarding repair technique that uses idle nodes to bypass low-bandwidth links without incurring network congestion and competition. Furthermore, a multi-node scheduling repair technique, called MSRepair, is proposed. MSRepair finds a recovery solution that schedules the parallel repair of multi-node and transfers data from as large-bandwidth links as possible, with the primary objective of minimizing the recovery time. Large-scale Mininet simulation and Amazon EC2 real experiments show that compared to state-of-the-art repair techniques, the single-node recovery time can be reduced by up to 36.65%, and the multi-node recovery time can be reduced by up to 55.10%.

Disk failure has always been a major problem for data centers, leading to data loss. Current research works used supervised learning to offline training through a large number of labeled samples. However, these offline methods are no longer suitable for disk failure prediction tasks in the current big data environment. Behind this explosive amount of data, most methods do not take into account the label values used in the model training phase may not be easy to obtain, or the obtained label values are not completely accurate. These problems further restrict the development of supervised learning and offline modeling in disk failure prediction. In this paper, ASLDP, a novel disk failure prediction method is proposed, which uses active learning and semi-supervised learning. According to the characteristics of data in the disk life cycle, ASLDP carries out active learning for those clear labeled samples, which selects valuable samples and eliminates redundancy. For those samples that are unclearly labeled or unlabeled, ASLDP combines with semi-supervised learning for pre-labeled, and enhances the generalization ability by active learning. The results on three realistic datasets demonstrate that ASLDP achieves stable failure detection rates of 80-85% with low false alarm rates, compared to current online learning methods. Furthermore, ASLDP can overcome the problems of the sample label missing and data redundancy in the massive data environment, compared to current offline learning methods.

Big data applications increasingly rely on the analysis of large graphs. In order to analyze and process the large graphs with high cost efficiency, researchers have developed a number of out-of-core graph processing systems in recent years based on just one commodity computer. On the other hand, with the rapidly growing need of analyzing graphs in the real-world, graph processing systems have to efficiently handle massive concurrent graph processing (CGP) jobs. Unfortunately, due to the inherent design for single graph processing job, existing out-of-core graph processing systems usually incur redundant data accesses and storage and severe competition of I/O bandwidth when handling the CGP jobs, thus leading to very long waiting time experienced by users for the computing results. In this paper, we propose an I/O-efficient out-of-core graph processing system, GraphCP, to support the processing of CGP jobs. GraphCP proposes a benefit-aware sharing execution model that shares the I/O access and processing of graph data among the CGP jobs and adaptively schedules the loading of graph data, which efficiently overcomes above challenges faced by existing out-of-core graph processing systems. In addition, GraphCP organizes the graph data with a Source-Sorted Sub-Block graph representation for better processing capacity and I/O access locality. Extensive evaluation results show that GraphCP is 10.3x and 4.6x faster than two state-of-the-art out-of-core graph processing systems GridGraph and GraphZ respectively, and 2.1x faster than a CGP-oriented graph processing system Seraph.

File systems for mobile devices usually preserve data consistency by ordered I/Os. However, maintaining I/O ordering prevents applications from fully exploiting device parallelism and thus degrades the storage performance. In this paper, we propose NBStack to eliminate ordered I/Os without compromising data consistency. First, we augment the existing block interface to expose the Flash out-of-band area to file systems. Second, we build an enhanced block device prototype that supports the new interface. Third, we develop NBFS, a Linux file system, that leverages the new block interface to achieve atomic writes without enforcing I/O orderings. Experimental results show that NBStack doubles the performance of F2FS while providing strong consistency and durability guarantees. If applications are willing to trade-off durability, NBStack can further aggressively improve performance.

In-memory key-value stores are often used to speed up Big Data workloads on modern HPC clusters. To maintain their high availability, erasure coding has been recently adopted as a low-cost redundancy scheme instead of replication. Existing erasure-coded update schemes, however, have either low performance or high memory overhead. In this paper, we propose a novel parity logging-based architecture, HybridPL, which creates a hybrid of in-place update (for data and XOR parity chunks) and log-based update (for the remaining parity chunks), so as to balance the update performance and memory cost, while maintaining efficient single-failure repairs. We realize HybridPL as an in-memory key-value store called LogECMem, and further design efficient repair schemes for multiple failures. We prototype LogECMem and conduct experiments on different workloads. We show that LogECMem achieves better update performance over existing erasure-coded update schemes with low memory overhead, while maintaining high basic I/O and repair performance.

Resistive cross-point memory arrays can be used to construct high-density storage-class memory. However, coupled IR drop and sneak currents cause multidimensional non-uniformity of cell effective voltage in cross-point arrays. The voltage non-uniformity significantly degrades write performance on cross-point memory if only adopting the worst-case write latency at partial dimensions. Furthermore, the non-uniformity of cell effective voltage in cross-point arrays depends on multidimensional dynamic write operation parameters: row, column as well as layer address, the number of selected cells, and the number of half-selected low-resistance state cells. In this article, we aim to improve the write performance by leveraging multidimensional non-uniformity of cell effective voltage. First, we analyze the impact of multidimensional write parameters on effective voltage and write latency. Then, we design the memory array write scheme that measures the write parameters and sets the write latency accordingly. We further analyze the features and effects of interlayer sneak currents and extend the scheme to 3D cross-point memory. The evaluation shows that the proposed memory array write scheme can reduce the memory access latency by 75.6 and 64.1 percent, and improve the system performance by 4.5 times and 3.4 times on average, compared with the baseline and the state-of-the-art approach, respectively.

Resistive memory is promising to be constructed as a high-density storage-class memory. Multilevel cell, access-transistor-free cross-point array structure, and 3-D array integration are three approaches to scale up the density of resistive memory. However, composing the three approaches together strengthens the interactions between array-level and cell-level nonidealities (interconnect resistance-induced IR drop, sneak current, and device variability) of resistive memory arrays during write operations and significantly degrades write performance and reliability. In this article, we analyze the dynamic voltage-dividing effect along a selected write current path in 3-D cross-point memory arrays. We propose a nonideality-tolerant high-density resistive memory (HD-RRAM) architecture, that can weaken the interactions between nonidealities and mitigate their degradation effects on the performance and reliability of array multilevel write operations. HD-RRAM is equipped with a double-transistor array architecture with two-transistor- n-resistor (2TnR) cell organization along pillars to reduce the current driving requirement and the large undesired voltage drop across each vertical pillar access transistor. Moreover, multiside asymmetric bias improves the resistive switching velocity by leveraging current-dividing effects. Variability-aware multilevel state partition reduces the worst-case write error rate by leveraging target state dependency of variability. Proportional-control multilevel state tuning reduces the average number of required write-and-verify iterations by leveraging pulse amplitude dependency of variability. Multilevel cell parallel writing improves the cell-level parallelism by leveraging the pass-through feature of intermediate resistance states. The evaluations show that HD-RRAM reduces both memory access latency and energy consumption over an aggressive baseline.

It is increasingly important and a big challenge to detect intrusion behavior variants in today's world. Previous host-based intrusion detection methods typically explore the sequence of system calls or unix shell commands to detect the intrusion behavior. This article abstracts the detection of intrusion behavior variants as the comparison between different sequences when the sequence order or length transforms. To overcome the impact of sequence transformation on the detection accuracy, we propose P-Gaussian, a provenance-based Gaussian distribution detection scheme which comprises two key design features: (1) it utilizes provenance to describe and identify intrusion behavior variants, and eliminates the impact of sequence order transformation on the detection accuracy. (2) it adopts Gaussian distribution principle to accurately compute the similarity between intrusion behavior and its variant, and eliminates the impact of intrusion behavior sequence length increase on the detection accuracy. To improve the detection performance, P-Gaussian employs a Redis memory database with multiple Redis instances and multiple threads to enable the parallelism of provenance processing in multi-core environments. It also classifies hot and cold provenance to provide high-efficient long-term forensic analysis. Experimental results on widely-used real world applications demonstrate the performance and efficiency of our system.

The integrity tree is a crucial part of the secure non-volatile memory (NVM) system design. For NVM with large capacity, the SGX-style integrity tree (SIT) is practical due to its parallel updates and variable arity. However, employing SIT in secure NVM is not easy. This is because the secure metadata SIT must be strictly persisted or restored after a sudden power¬loss, which unfortunately incurs unacceptable run-time overhead or recovery time. In this paper, we propose PSIT, a metadata persistency solution for SIT-protected secure NVM with high performance and fast restoration. PSIT utilizes the observation that for a lazily updated SIT, the lost tree nodes after a crash can be recovered by the corresponding child nodes in the NVM. It reduces the persistency overhead of the SIT nodes through a restrained write-back meta-cache and leverages the SIT inter¬layer dependency for recovery. Experiments show that compared to ASIT, a state-of-the-art secure NVM using SIT, PSIT decreases write traffic by 47% and improves the performance by 18% on average while maintaining an acceptable recovery time.

As we all know, file and free space fragmentation negatively affect file system performance. F2FS is a file system designed for flash memory. However, it suffers from severe fragmentation due to its out-of-place updates and the highly synchronous, multi-threaded writing behaviors of mobile applications. We observe that the running time of fragmented files is 2.36× longer than that of continuous files and that F2FS's in-place update scheme is incapable of reducing fragmentation. A fragmented file system leads to a poor user experience.(p)(/p)Reserving space to prevent fragmentation is an intuitive approach. However, reserving space for all files wastes space since there are a large number of files. To deal with this dilemma, we propose an adaptive reserved space (ARS) scheme to choose some specific files to update in the reserved space. How to effectively select reserved files is critical to performance. We collect file characteristics associated with fragmentation to construct data sets and use decision trees to accurately pick reserved files. Besides, adjustable reserved space and dynamic reservation strategy are adopted. We implement ARS on a HiKey960 development platform and a commercial smartphone with slight space and file creation time overheads. Experimental results show that ARS reduces file and free space fragmentation dramatically, improves file I/O performance and reduces garbage collection overhead compared to traditional F2FS and F2FS with in-place updates. Furthermore, ARS delivers up to 1.26× transactions per second under SQLite than traditional F2FS and reduces running time of realistic workloads by up to 41.72% than F2FS with in-place updates.

Emerging Non-Volatile Memory (NVM) techniques suffer from many attacks on endurance and data confidentiality. Oblivious RAM (ORAM) can prevent both types of attacks by address obfuscation and encryption. However, ORAM also introduces a lot of redundant memory operations, worsening the NVM endurance and system performance. Furthermore, the slower NVM increases the response latency of target blocks compared to DRAM. As the read requests are on the critical path of program execution, the increased latency further reduces system performance. To alleviate endurance and performance problems, we propose H2ORAM, a low response latency optimized ORAM scheme for NVM. Hybrid DRAM/NVM system is introduced to eliminate 57% read/write traffic to NVM, which improves the lifetime of NVM devices. To reduce response latency of read requests, we securely manipulate data distributions in hybrid memory to speed up hot data access. The manipulation leverages a frequency-based eviction scheme and a data duplication scheme to keep more hot data accessible from DRAM earlier and faster. Moreover, we utilize the empty slots in DRAM region to further improve performance. The experimental results show that 44% response latency reduction and 18% performance improvement are achieved over the basic Ring ORAM with hybrid memory architecture integrated.

As users' demand for large-capacity storage continues to grow, the widely used NAND flash memory always adopts multi-bit per cell and 3D-stacking technology to improve the storage density however impairing flash performance. Modern flash chips allow flash blocks to switch between multi-bit per cell and one-bit per cell, which gives the potential to benefit from the high performance of Single-Level Cells (SLC, i.e. one-bit per cell). However, the semantic gap caused by the block I/O interface and flash translation layer (FTL) prevents the performance of the underlying flash memory from being fully utilized. In this paper, we propose a log-structured file system, called DualFS. DualFS allows flash blocks to switch between the original-mode and SLC-mode in a free manner and uses these SLC blocks to accelerate critical requests. DualFS dynamically adjusts the capacity of the SLC-mode area according to the total size of valid data to maintain the device a negligible capacity degradation. Besides, DualFS draws advantages from the Open-Channel solid-state disk (SSD) by using software semantic information to guide data placement and management. In particular, since writing the same amount of data to the SLC block will impair more endurance, DualFS proposes a novel life management scheme which throttles the ratio of writing to SLC-mode area within a certain window, thereby accurately control the endurance of SSD. Last, DualFS integrates the garbage collection (GC) procedure, which is directly driven by the file system and merges the GC of the SLC-mode area and the original-mode area. The experiment results show that DualFS offers an average 22.2% performance improvement than the state of art flash file system based on the open-channel SSD. Compared with other storage systems based on flash block dual-mode switching feature, the average performance of DualFS also exhibits considerable improvements by 13.3% to 24.6%. Moreover, DualFS effectively reduces the read response time under different read/write ratios and precisely manage the device endurance.

Flash memory-based SSDs are popular across a wide range of data storage markets, while the flash memory as a storage medium is becoming increasingly unreliable. To improve reliability, modern SSDs employ many in-device reliability enhancement techniques, such as ECC and RAID. However, existing intra-SSD reliability techniques do not consider both the multilevel reliability problem of flash memory and the imbalance of wear between blocks. Therefore either too much redundancy is provided for low wear blocks, which increases the write amplification, or multiple errors cannot be tolerated for high wear blocks and chips. We proposed AetEC, an adaptive error-tolerant erasure coding scheme within SSDs. AetEC adaptively applies RS (Reed Solomon) erasure codes with different error correction capabilities according to the number of high-wear blocks in the superblock. To tolerate chip-level failures of high-wear blocks, AetEC maximizes the error tolerance of RS codes. To improve performance, AetEC applies the physical address-based stripe organization and implements a FPGA-based RS encoder and decoder. Furthermore, AetEC proposes a double-linked list write cache to both improve write performance and ensure the consistency of user data and check data. We model and analyze the reliability of AetEC on SSDsim. Experiment results show that, compared with the fixed RAID4, the average response time of AetEC is reduced by up to 32.8%, and write amplification is decreased by up to 37.8%. Compared with the latest wear-aware RAID, AetEC reduces the uncorrectable page error rate (UPER) by two orders of magnitude with 5.9% additional response time.

A cloud object store entails serving workloads of multi tenants. Different applications have different access characteristics and various performance requirements. Thus, it is necessary for the cloud object store to provide tenant-specific policies. OpenStack Swift develops a storage policy mechanism to enable separate storage configurations for different applications. However, the limited configurability of the policy mechanism makes it difficult to provide efficient and flexible policies to meet the evolving needs of the applications. First, original policies only control the frontend paths of the request forwarding without covering the backend paths of the request processing. Thus, they cannot provide sufficient optimizations for workload performance. Second, those policies need to be pre-defined and remain unchanged at runtime, so that they lack the flexibility to adapt to the possible workload changes during runtime. In addition, effective execution of tenant-specific policies is based on the premise that all storage layers are aware of the requests’ sources, while the storage server layer lacks the requisite information to distinguish requests from multi tenants.

Non-volatile memory (NVM) is emerging as a fast byte-addressable persistent memory (PM) that promises data persistence at the main memory level. One of the common choices for providing failure-atomic updates in PM is the write-ahead logging (WAL) technique. To mitigate logging overhead, recent studies propose WAL-based hardware logging designs that overlap log writes with transaction execution. However, existing hardware logging designs incur a large number of unnecessary log writes. Many log writes are still performed in the critical path, which causes high performance overhead, particularly for the multi-core systems with many threads.

Byte-addressable non-volatile memory (NVM) is emerging as an alternative for main memory. Non-volatile main memory (NVMM) systems are required to support atomic persistence and deal with the high overhead of programming NVM cells. To this end, recent studies propose hardware logging and data encoding designs for NVMM systems. However, prior hardware logging designs incur either extra ordering constraints or redundant log data. Moreover, existing data encoding designs are unaware of the characteristics of log data, resulting in writing unnecessary log bits.In this paper, we propose a morphable hardware logging design (MorLog) that only logs the data necessary for recovery and dynamically selects encoding methods with least write overhead. We observe that (1) only the oldest undo and the newest redo data in each transaction are necessary for recovery, and (2) the log data for clean bits are clean. The first motivates our morphable logging mechanism. This mechanism logs both undo and redo data for the first update to the data in a transaction, and then logs only redo data. Undo data are eagerly written to NVMM to ensure atomicity, while redo data are buffered in a volatile log buffer and L1 caches to write only the newest redo data to NVMM. The second motivates our selective log data encoding mechanism. This mechanism simultaneously encodes log data with different methods, and writes the encoded log data with the least write cost to NVMM. We devise a differential log data compression method to exploit the characteristics of log data. This method directly discards clean bits from log data and compresses remained dirty bits. Our evaluation shows that MorLog improves performance by 72.5%, reduces NVMM write traffic by 41.1%, and decreases NVMM write energy by 49.9% compared with the state-of-the-art design.

Provenance is a type of metadata that records the creation and transformation of data objects. It has been applied to a wide variety of areas such as security, search, and experimental documentation. However, provenance usually has a vast amount of data with its rapid growth rate which hinders the effective extraction and application of provenance. This paper proposes an efficient provenance management system via clustering and hybrid storage. Specifically, we propose a Provenance-Based Label Propagation Algorithm which is able to regularize and cluster a large number of irregular provenance. Then, we use separate physical storage mediums, such as SSD and HDD, to store hot and cold data separately, and implement a hot/cold scheduling scheme which can update and schedule data between them automatically. Besides, we implement a feedback mechanism which can locate and compress the rarely used cold data according to the query request. The experimental test shows that the system can significantly improve provenance query performance with a small run-time overhead.

The page size of NAND flash continuously grows as the manufacturing process advances. While larger pages can reduce the cost per bit and improve the throughput of NAND flash, it may waste the storage space and data transfer time, causing more frequent garbage collections when serving small write requests. The main methods solving the mismatch problem between the request size and the write unit are write buffer cache and flash page reprogramming. However, multi-level cell (MLC) chips impose additional constraints on page programming so reprogramming MLC pages is prohibited. We proposed a multiple subpage writing flash translation layer (MSPW-FTL) for MLC by exploiting single-level cell (SLC)/MLC dual mode and flash page reprogramming feature. By converting MLC mode blocks to SLC mode blocks, we store small data in subpages of the SLC mode block. Moreover, we proposed three management methods to improve system efficiency: 1) two-level mapping to serve requests of different sizes; 2) an allocation strategy determines how the subpages of different logical pages are mapped to physical pages; and 3) a data management module to deal with the data fragmentation caused by the subpage granularity allocation. We compared MSPW-FTL with some related state-of-the-art FTLs under different types of workloads. Experimental results show that in average, MSPW-FTL reduces the I/O response time by 57.2%, the write amplification by 52.1%, and the number of erasures by 34.1%.

Emerging nonvolatile memories (NVMs) are promising to replace DRAM as main memory. However, NVMs suffer from limited write endurance and high write energy. Encoding method reduces the bit flips of NVMs by exploiting additional tag bits to encode the data. The effect of the encoding method is limited by the capacity overhead of the tag bits. In this article, we propose to exploit the space saved by compression to store the tag bits of the encoding method. We observe that the saved space size of each compressed cache line varies, and different encoding methods have different tradeoffs between capacity overhead and effect. To fully exploit the space saved by compression for improving lifetime, we select the proper encoding method according to the saved space size. To improve the compression coverage and compression ratio, we select an efficient compression scheme from two compression algorithms and provide more space for data encoding. Still, some data patterns cannot be compressed by any compression technique. We use the Flip-N-Write with 3.1% capacity overhead to encode uncompressible cache lines. The experimental results show that our scheme reduces the bit flips by 32.5%, decreases the energy consumption by 22.6% and improves the lifetime by 69.9% with 3.5% capacity overhead.

Memristive crossbar array suffers from severe sneak currents that incur reliability issues and extra energy waste. Complementary resistive switches (CRSs) provide a new concept to address the sneak-current problem. But the destructive read of CRS results in an additional recovery write operation, which strongly restricts its further promotion. Exploiting the dual CRS/memristor mode of CRS devices, we propose Aliens, a novel reconfigurable architecture that introduces one alien cell (memristor mode) for each bitline in the crossbar. Aliens draws advantages from both modes: restrained sneak currents of the CRS mode and nondestructive read of the memristor mode. The simple and regular cell mode organization one bitline one memristor (OBOM) of Aliens enables an energy-saving read method. Further, by exploiting memory access locality, an effective mode switching strategy called Lazy-Switch is proposed to delay and merge the recovery write operations of the CRS mode. Moreover, an 1TnR crossbar structure is adopted to enable larger crossbar arrays as well as a higher ratio of memristor mode cells without going against the OBOM rule. The effects of the memristor mode cell ratio on the energy consumption, endurance, and access performance are studied. Also, we show the bank architecture of Aliens and analyze how to extend our designs to 3-D arrays. Due to fewer recovery write operations and negligible sneak currents, Aliens achieves improvements in energy, overall endurance, and access performance. The experimental results show that our design offers average energy savings of 19.1× compared with memristor-only memory, a memory lifetime 10.7× longer than CRS-only memory, and a competitive performance compared with memristor-only memory.

Efficient intrusion detection and analysis of the security landscape in big data environments present challenge for today's users. Intrusion behavior can be described by provenance graphs that record the dependency relationships between intrusion processes and the infected files. Existing intrusion detection methods typically analyze and identify the anomaly either in a single provenance path or the whole provenance graph, neither of which can achieve the benefit on both detection accuracy and detection time. We propose Pagoda, a hybrid approach that takes into account the anomaly degree of both a single provenance path and the whole provenance graph. It can identify intrusion quickly if a serious compromise has been found on one path, and can further improve the detection rate by considering the behavior representation in the whole provenance graph. Pagoda uses a persistent memory database to store provenance and aggregates multiple similar items into one provenance record to maximumly reduce unnecessary I/O during the detection analysis. In addition, it encodes duplicate items in the rule database and filters noise that does not contain intrusion information. The experimental results on a wide variety of real-world applications demonstrate its performance and efficiency.

Datacenter network load balancing schemes handle network traffic generated by massive different applications. Some packet-based or flowlet-based schemes capture traffic bursts for load balancing. But frequent rerouting within a flow can mix ACKs belonging to different paths in congestion control protocols, which adversely affects flow rate control. Besides, performance optimization effect of flowlet-based schemes may be less noticeable under smoother workloads. And several packet-based mechanisms implemented at end hosts can proactively reroute congested flows based on flow status even under a smooth workload, but fail to improve performance with the bursty nature of traffic. Therefore, existing schemes cannot adapt to different burst levels of dynamic traffic in datacenter networks and still have significant performance flaws in some ways. This paper proposes IntFlow, a novel load balancing scheme that integrates end-host based per-packet monitoring of flow status with flowlet switching in programable switches. IntFlow proactively reroutes flows experiencing network congestion or failures and avoids doing flowlet switching for small flows with high sending rate. IntFlow can provide excellent performance under both high burst and smooth workloads. Finally experimental results show IntFlow achieves up to 32% and 28% better performance than CONGA and Hermes under asymmetries, respectively.

Erasure codes are being extensively deployed in practical storage systems to prevent data loss with low redundancy. However, these codes require excessive disk I/Os and network traffic for recovering unavailable data. Among all erasure codes, Minimum Storage Regenerating (MSR) codes can achieve optimal repair bandwidth under the minimum storage during recovery, but some open issues remain to be addressed before applying them in real systems. Facing with the huge burden during recovery, erasure-coded storage systems need to be developed with high repair efficiency. Aiming at this goal, a new class of coding scheme is introduced—Hybrid Regenerating Codes (Hybrid-RC). The codes utilize the superiority of MSR codes to compute a subset of data blocks while some other parity blocks are used for reliability maintenance. As a result, our design is near-optimal with respect to storage and network traffic and shows great improvements in recovery performance.

Content-Defined Chunking (CDC) has been playing a key role in data deduplication systems recently due to its high redundancy detection ability. However, existing CDC-based approaches introduce heavy CPU overhead because they declare the chunk cut-points by computing and judging the rolling hashes of the data stream byte by byte. In this article, we propose FastCDC, a Fast and efficient Content-Defined Chunking approach, for data deduplication-based storage systems. The key idea behind FastCDC is the combined use of five key techniques, namely, gear based fast rolling hash, simplifying and enhancing the Gear hash judgment, skipping sub-minimum chunk cut-points, normalizing the chunk-size distribution in a small specified region to address the problem of the decreased deduplication ratio stemming from the cut-point skipping, and last but not least, rolling two bytes each time to further speed up CDC. Our evaluation results show that, by using a combination of the five techniques, FastCDC is 3-12X faster than the state-of-the-art CDC approaches, while achieving nearly the same and even higher deduplication ratio as the classic Rabin-based CDC. In addition, our study on the deduplication throughput of FastCDC-based Destor (an open source deduplication project) indicates that FastCDC helps achieve 1.2-3.0X higher throughput than Destor based on state-of-the-art chunkers.

In recent years, a number of out-of-core graph processing systems have been proposed to process graphs with billions of edges on just one commodity computer, due to their high cost efficiency. To obtain a better performance, these systems adopt a full I/O model that scans all edges during the computation to avoid the inefficiency of random I/Os. Although this model ensures good I/O access locality, it leads to a large number of useless edges to be loaded when running graph algorithms that only access a small portion of edges in each iteration. An intuitive method to solve this I/O inefficiency problem is the on-demand I/O model that only accesses the active edges. However, this method only works well for the graph algorithms with very few active edges, since the I/O cost will grow rapidly as the number of active edges increases due to the increasing amount of random I/Os. In this article, we present HUS-Graph, an efficient out-of-core graph processing system to address the above I/O issues and achieve a good balance between I/O traffic and I/O access locality. HUS-Graph adopts a hybrid update strategy including two update models, Row-oriented Push (ROP) and Column-oriented Pull (COP). It supports switching between ROP and COP adaptively, for the graph algorithms that have different computation and I/O features. For traversal-based algorithms, HUS-Graph also provides an immediate propagation-based vertex update scheme to accelerate the vertex state propagation and convergence speed. Furthermore, HUS-Graph adopts a locality-optimized dual-block representation to organize graph data and an I/O-based performance prediction method to enable the system to dynamically select the optimal update model between ROP and COP. To save the disk space and further reduce I/O traffic, HUS-Graph implements a space-efficient storage format by combining several graph compression methods. Extensive experimental results show that HUS-Graph outperforms two existing out-of-core systems GraphChi and GridGraph by 1.2x-52.8x.

Data deduplication, though being efficient in removing duplicate chunks, introduces chunk fragmentation which decreases restore performance. Rewriting algorithms are proposed to reduce the chunk fragmentation. Delta compression is often used as a complement for data deduplication to further improve storage efficiency. We observe that delta compression introduces a new type of chunk fragmentation stemming from improper delta compression for chunks of which the base chunks are fragmented. The new type of chunk fragmentation severely decreases restore performance and cannot be addressed by existing rewriting algorithms. To address this problem, we propose SDC, a scheme performing post-deduplication delta compression only for the chunks of which the bases can be directly found in the restore cache to eliminate additional disk reads for base chunks, thus avoiding the new type of chunk fragmentation. In addition, self-referenced chunks can be fragmented, which decrease restore performance, and these fragmented chunks can serve as bases to decrease the restore performance repeatedly. We propose a hybrid rewriting scheme for SDC to rewrite such fragmented chunks. Experimental results show that SDC improves the restore performance of the approach that directly performs delta compression after data deduplication by 2.9-16.9x, and achieves more than 95 percent of its compression gains.

Non-volatile memories (NVMs) are vulnerable to serious threat due to the endurance variation. We identify a new type of malicious attack, called Uniform Address Attack (UAA), which performs uniform and sequential writes to each line of the whole memory, and wears out the weaker lines (lines with lower endurance) early. Experimental results show that the lifetime of NVMs under UAA is reduced to 4.1% of the ideal lifetime. To address such attack, we propose a spare-line replacement scheme called Max-WE (Maximize the Weak lines' Endurance). By employing weak-priority and weak-strong-matching strategies for spare-line allocation, Max-WE is able to maximize the number of writes that the weakest lines can endure. Furthermore, Max-WE reduces the storage overhead of the mapping table by 85% through adopting a hybrid spare-line mapping scheme. Experimental results show that Max-WE can improve the lifetime by 9.5X with the spare-line overhead and mapping overhead as 10% and 0.016% of the total space respectively.

Data encoding methods have been proposed to alleviate the high write energy and limited write endurance disadvantages of Non-Volatile Memories (NVMs). Encoding methods are proved to be effective through theoretical analysis. Under the data patterns of workloads, existing encoding methods could become inefficient. We observe that the new cache line and the old cache line have many redundant (or unmodified) words. This makes the utilization ratio of the tag bits of data encoding methods become very low, and the efficiency of data encoding method decreases. To fully exploit the tag bits to reduce the bit flips of NVMs, we propose REdundant word Aware Data encoding (READ). The key idea of READ is to share the tag bits among all the words of the cache line and dynamically assign the tag bits to the modified words. The high utilization ratio of the tag bits in READ leads to heavy bit flips of the tag bits. To reduce the bit flips of the tag bits in READ, we further propose Sequential flips Aware Encoding (SAE). SAE is designed based on the observation that many sequential bits of the new data and the old data are opposite. For those writes, the bit flips of the tag bits will increase with the number of tag bits. SAE dynamically selects the encoding granularity which causes the minimum bit flips instead of using the minimum encoding granularity. Experimental results show that our schemes can reduce the energy consumption by 20.3%, decrease the bit flips by 25.0%, and improve the lifetime by 52.1%.

By exposing physical channels to host software, Open-Channel SSD shows great potential in future high performance storage systems. However, the existing scheme fails to achieve acceptable performance under heavy workloads. The main reasons reside not only in its single-buffer architecture, more importantly, but also in its line-based physical address management. Besides, the lock of address mapping table is also a performance burden under heavy workloads. We propose QBLK, an open source driver which tries to better exploit the parallelism of Open-Channel SSDs. Particularly, QBLK adopts four key techniques, namely (1) Multi-queue based buffering, (2) Per-channel based address management, (3) Lock-free address mapping, and (4) Fine-grained draining. Experimental results show that QBLK achieves up to 97.4% bandwidth improvement compared with the state-of-the-art PBLK scheme.

Recent advances in Resistive RAM (ReRAM) have explored the in-situ Matrix-Vector Multiplication (MVM) ability of crossbar arrays to achieve high energy-efficiency Process-In-Memory (PIM) architectures for Convolutional Neural Network (CNN), image processing, and so on. However, the existing ReRAM-based PIM architectures suffer from considerable additional auxiliary logic and device variations. In this work, we propose a novel analog computing architecture NB Engine for classification by implementing Naive Bayesian (NB) algorithm on ReRAM crossbar arrays. The two key steps of the NB algorithm, that is, probability calculation and electing the class that has the highest probability, are elaborately accomplished in our architecture. The ReRAM arrays are both used as storage and computation components. We store the pre-calculated prior probabilities and conditional probabilities of every class in crossbar arrays. Then the probability calculation step is completed in parallel through the MVM operation of the array. In general, the election step is a multiple-comparison procedure and is normally implemented by a comparison tree. Here, we reuse the max pooling module in a conventional CNN PIM architecture to realize a compatible comparison logic. However, neither of the two designs can avoid the overhead of costly high bit-precision Analog-to-Digital Converters (ADCs). So we introduce a novel analog parallel comparison design which does not need any ADCs or other computing logic with better energy-saving and area-efficiency. Our proposed NB Engine is tested by 11 various datasets. The influence of several non-ideal device properties is discussed and the NB Engine exhibits great tolerance to these variations. The experiment results show that our design offers a runtime speedup up to 2289.6x compared with the software-implemented NB classifier with negligible accuracy loss. In addition, the NB Engine saves 96.2% energy consumption and 45.2% array area compared with the CNN PIM compatible design.

Data deduplication is a widely deployed technique to remove duplicate content to save storage space, which is however incapable of eliminating the redundancy between nonidentical but similar data blocks. To achieve further space savings in deduplicated storage systems, delta compression is employed to compress post-deduplication data. Both deduplication and delta compression introduce content references among blocks, which inevitably undermines the reliability of deduplicated and delta compressed storage systems. To ensure better reliability, existing approaches utilize either replication or erasure codes to redundantly distribute data across multiple nodes. In deduplicated and delta compressed storage systems, we observe that delta compressed chunks (DCCs) are far smaller than regular chunks called non-DCCs. Motivated by this observation, we suggest a straightforward approach in which replication is used to protect DCCs and erasure code is deployed to protect non-DCCs. However, we need to address two critical challenges to ensure this solution effective. First, the random placement of DCCs replicas destroys cache locality. Second, the separate and individual recovery and restore cache could cause storage containers to be accessed repeatedly. To address these two challenges, in this paper, we propose RepEC-Duet which employs both replication and erasure codes to ensure high reliability and performance for deduplicated and delta-compressed storage systems. RepEC-Duet introduces a delta-utilization-aware filter to select and replicate containers based on the percentage of DCCs in the containers to maintain cache locality. Moreover, to avoid unnecessary container reads, we design a cooperative cache scheme that is aware of both failure recovery and regular restore cache. Our experimental results based on three real-world datasets demonstrate that RepEC-Duet significantly improves the restore performance by 26%-59%, and reduces the storage overhead by 54%-98% than the existing approaches.

Reservoir sampling is widely employed to characterize connectivity of large graph streams by producing edge samples. However, existing reservoir-based sampling methods mainly characterize large graph streams by a measure of counting triangles but perform poorly in accuracy when used to analyze the topological characteristics reflected by node degrees because they produce disconnected edge samples, making them ineffective in many applications that require both types of connectivity estimation simultaneously in real time. This paper proposes a new method, called triangle-induced reservoir sampling, or T-Sample, to produce connected edge samples. While every edge in a graph stream is still processed only once by T-Sample, a dual sampling mechanism performing both uniform sampling and non-uniform sampling is carefully designed with a base reservoir and an incremental reservoir. Specifically, the uniform sampling can be used to count triangles by employing the existing algorithms while the non-uniform sampling ensures that the edge samples are connected. Experimental results driven by real datasets show that T-Sample can obtain much more accurate estimations on the distributions of node degrees than the existing reservoir-based sampling methods.

NVMe SSDs have been wildly adopted to provide storage services in cloud platforms where diverse workloads (including latency-sensitive, throughput-oriented and capacity-oriented workloads) are colocated. To achieve performance isolation, existing solutions partition the shared SSD into multiple isolated regions and assign each workload a separate region. However, these isolation solutions could result in inefficient resource utilization and imbalanced wear. More importantly, they cannot reduce the interference caused by embedded cache contention. In this paper, we present CostPI to improve isolation and resource utilization by providing latency-sensitive workloads with dedicated resources (including data cache, mapping table cache and NAND flash), and providing throughput-oriented and capacity-oriented workloads with shared resources. Specifically, at the NVMe queue level, we present an SLO-aware arbitration mechanism which fetches requests from NVMe queues at different granularities according to workload SLOs. At the embedded cache level, we use an asymmetric allocation scheme to partition the cache (including data cache and mapping table cache). For different data cache partitions, we adopt different cache polices to meet diverse workload requirements while reducing the imbalanced wear. At the NAND flash level, we partition the hardware resources at the channel granularity to enable the strongest isolation. Our experiments show that CostPI can reduce the average response time by up to 44.2%, the 99% response time by up to 89.5%, and the 99.9% by up to 88.5% for latency-sensitive workloads. Meanwhile, CostPI can increase resource utilization and reduce wear-imbalance for the shared NVMe SSD.

Deduplication, as a space-saving technology, is widely deployed in the flash-based storage systems to address the capacity and endurance limitations of flash devices. In this paper, we find that deduplication changes the physical data layout, which raises the chances of the uneven read distribution. This uneven read distribution not only increases the access contention but also deteriorates the read parallelism, thus leading to the read performance degradation. To solve this issue, we propose an efficient read-leveling data distribution scheme (RLDDS), which scatters the highly-duplicated data into different parallel units, to improve the read performance for SSDs with deduplication for access-intensive workloads. RLDDS writes data into a parallel unit with lower potential read-hotness to balance the read distribution among all the parallel units. Extensive experimental results show that RLDDS effectively improves the read performance by up to 21.61% compared to deduplication with the conventional dynamic data allocation scheme. Additional benefits of RLDDS include the promoted write performance (up to 23.69%) in access-intensive workloads and the overall system performance improvement (up to 18.22%) with the same write traffic reduction.

Parity based RAID suffers from poor small write performance due to heavy parity update overhead. The recently proposed method EPLOG constructs a new stripe with updated data chunks without updating old parity chunks. However, due to skewness of data accesses, old versions of updated data chunks often need to be kept to protect other data chunks of the same stripe. This seriously hurts the efficiency of recovering system from device failures due to the need of reconstructing the preserved old data chunks on failed devices.

Big data applications increasingly rely on the analysis of large graphs. In recent years, a number of out-of-core graph processing systems have been proposed to process graphs with billions of edges on just one commodity computer, by efficiently using the secondary storage (e.g., hard disk, SSD). On the other hand, the vertex-centric computing model is extensively used in graph processing thanks to its good applicability and expressiveness. Unfortunately, when implementing vertex-centric model for out-of-core graph processing, the large number of random memory accesses required to construct subgraphs lead to a serious performance bottleneck that substantially weakens cache access locality and thus leads to very long waiting time experienced by users for the computing results. In this paper, we propose an efficient out-of-core graph processing system, LOSC, to substantially reduce the overhead of subgraph construction without sacrificing the underlying vertex-centric computing model. LOSC proposes a locality-optimized subgraph construction scheme that significantly improves the in-memory data access locality of the subgraph construction phase. Furthermore, LOSC adopts a compact edge storage format and a lightweight replication of vertices to reduce I/O traffic and improve computation efficiency. Extensive evaluation results show that LOSC is respectively 6.9x and 3.5x faster than GraphChi and GridGraph, two state-of-the-art out-of-core systems.

Recent research works on disk failure prediction achieve a high detection rate and a low false alarm rate with complex models at the cost of explainability. The lack of explainability is likely to hide bias or overfitting in the models, resulting in bad performance in real-world applications. To address the problem, we propose a new explanation method DFPE designed for disk failure prediction to explain failure predictions made by a model and infer prediction rules learned by a model. DFPE explains failure predictions by performing a series of replacement tests to find out the failure causes while it explains models by aggregating explanations for the failure predictions. A presented use case on a real-world dataset shows that compared to current explanation methods, DFPE can explain more about failure predictions and models with more accuracy. Thus it helps to target and handle the hidden bias and overfitting, measures feature importances from a new perspective and enables intelligent failure handling.

Resistive Memory (ReRAM) is promising to be used as high density storage-class memory by employing Triple-Level Cell (TLC) and crossbar structures. However, TLC crossbar ReRAM suffers from high write latency and energy due to the IR drop issue and the iterative program-and-verify procedure. In this paper, we propose Tiered-ReRAM architecture to overcome the challenges of TLC crossbar ReRAM. The proposed Tiered-ReRAM consists of three components, namely Tiered-crossbar design, Compression-based Incomplete Data Mapping (CIDM), and Compression-based Flip Scheme (CFS). Specifically, based on the observation that the magnitude of IR drops is primarily determined by the long length of bitlines in Double-Sided Ground Biasing (DSGB) crossbar arrays, Tiered-crossbar design splits each long bitline into the near and far segments by an isolation transistor, allowing the near segment to be accessed with decreased latency and energy. Moreover, in the near segments, CIDM dynamically selects the most appropriate IDM for each cache line according to the saved space by compression, which further reduces the write latency and energy with insignificant space overhead. In addition, in the far segments, CFS dynamically selects the most appropriate flip scheme for each cache line, which ensures more high resistance cells written into crossbar arrays and effectively reduces the leakage energy. For each compressed cache line, the selected IDM or flip scheme is applied on the condition that the total encoded data size will never exceed the original cache line size. The experimental results show that, on average, Tiered-ReRAM can improve the system performance by 30.5%, reduce the write latency by 35.2%, decrease the read latency by 26.1%, and reduce the energy consumption by 35.6%, compared to an aggressive baseline.

The following topics are dealt with: storage management; flash memories; parallel processing; cache storage; cloud computing; meta data; data compression; data handling; optimisation; learning (artificial intelligence).

Most high-performance computing (HPC) clusters use a global parallel file system to enable high data throughput. The parallel file system is typically centralized and its storage media are physically separated from the compute cluster. Compute nodes as clients of the parallel file system are often additionally equipped with SSDs. The node internal storage media are rarely well-integrated into the I/O and compute workflows. How to make full and flexible use of these storage media is therefore a valuable research question.

Minimum-Storage Regenerating (MSR) codes have become superior alternatives to traditional erasure codes as they can provide optimal repair bandwidth while the reliability and the storage overhead are still optimal. So far, the state-of-the-art MSR codes mainly focus on connecting to all the remaining nodes to repair a single failure. Since the recovery latency may be bottlenecked by the time taken to retrieve the slowest or straggling block, it is however impractical to have the highest connectivity in MSR codes. In this paper, we introduce a generic construction for all parameters in MSR codes, which allows for bandwidth-efficient repair of a single data node failure with an arbitrary (but fixed) number of accessed nodes d. Our method provides explicit and generic encoding and repairing processes, and such codes were not previously known to exist. When d = n - 1, we show that our codes are not inferior to the other MSR codes and retain the same recovery optimality. Furthermore, the arbitrary number of d allows our codes to adapt to the late binding strategy to avoid stragglers (overloaded sites) for efficient recovery performance. As a result, our codes outperform both traditional erasure codes and the state-of-the-art MSR codes on the aspect of the response time when the system is subjected to an imbalanced load.

Summary Performance isolation is highly desirable in cloud platforms where the virtual disks of virtual machines are simply large files on the shared and networked storage servers. However, existing isolation techniques cannot deal with the implications of the file system used by the networked storage servers, such that underlying resource usage is unpredictable (eg, the delayed write‐back mechanism could postpone writes, and the journaling mechanism could amplify writes). The lack of visibility on underlying resource usage leads to the predicament of being unable to meet isolation goals. In this paper, we present a software‐defined file system (SDFS) that exploits the underlying file system to allocate resources at per‐image‐file granularity and provide tenants with guaranteed throughput. The SDFS comprises two components: control plane and data plane. At the control plane, we provide a set of system calls to document tenant performance requirements into the metadata of image files. At the data plane, we construct a file‐based scheduler to manage memory and disk resources according to the tenant performance requirements. The SDFS design does not require a modification to guest operating systems, hypervisors, or file server protocols. Through a prototype implementation, we demonstrate that the SDFS can meet isolation goals and increase resource utilization with negligible overhead.

Emerging non-volatile byte-addressable memory (NVM) introduces many opportunities and challenges to memory system designs. As data become persistent at main memory level, persistent memory systems need to guarantee the consistent state of data in the event of system failures (i.e., crash consistency). Existing studies propose persistent memory designs with software-transparent crash consistency guarantee to reduce programmers' manual effort when taking advantage of persistent memory. However, these designs are suboptimal due to their performance overhead caused by creating checkpoints. In this paper, we propose a Non-Intrusive memory COntroller design (NICO) that uses backend operations for achieving software-transparent crash consistency with minimized checkpointing overhead. By moving data persist operations to the background, NICO fully decouples data persist operations from volatile execution and cache management. To efficiently enforce crash consistency, we design a lightweight checkpointing scheme which only needs to flush and modify a very small amount of data when creating a consistent snapshot of persistent memory data. Our results show that NICO reduces the percent of time spent on checkpointing to within 0.9 percent across different benchmarks, and improves performance by 2.04× compared with existing checkpoint-based designs on average.

Modern file systems employ journaling techniques to guarantee data consistency in case of unexpected system crashes or power failures. However, journaling file systems usually suffer from performance decrease due to the extra journal writes. Moreover, the emerging non-volatile memory technologies (NVMs) have the potential capability to reduce the journaling overhead by being deployed as the journaling storage devices. However, traditional journaling techniques, which are designed for hard disks, fail to perform efficiently in NVMs. In order to address this problem, we propose an NVM-based journaling scheme, called NJS. The basic idea behind NJS is to reduce the journaling overhead of traditional file systems while fully exploiting the byte-accessibility characteristic of NVM, and alleviating the slow write and endurance limitation of NVM. Our NJS consists of four major contributions: (1) In order to decrease the amount of journal writes, NJS only needs to write the file system metadata and over-write data to NVM as write-ahead logging, thus alleviating the slow write and endurance limitation of NVM. (2) NJS adopts a wear aware strategy for NVM journal block allocation in which each block can be evenly worn out, thus further extending the lifetime of NVM. (3) We propose a novel journaling update scheme in which journal data blocks can be updated in the byte-granularity based on the difference of the old and new versions of journal blocks, thus fully exploiting the unique byte-accessibility characteristic of NVM. (4) NJS includes a garbage collection mechanism that absorbs the redundant journal updates, and actively delays the checkpointing to the file system. Evaluation results show the efficiency and efficacy of NJS. For example, compared with Ext4 with a ramdisk-based journaling device, the throughput improvement of Ext4 with our NJS is up to 131.4 percent.

In this paper, we present a comprehensive study and evaluation of existing single image dehazing algorithms, using a new large-scale benchmark consisting of both synthetic and real-world hazy images, called REalistic Single Image DEhazing (RESIDE). RESIDE highlights diverse data sources and image contents, and is divided into five subsets, each serving different training or evaluation purposes. We further provide a rich variety of criteria for dehazing algorithm evaluation, ranging from full-reference metrics, to no-reference metrics, to subjective evaluation and the novel task-driven evaluation. Experiments on RESIDE shed light on the comparisons and limitations of stateof- the-art dehazing algorithms, and suggest promising future directions.

Emerging computational resistive memory is promising to overcome the challenges of scalability and energy efficiency that DRAM faces and also break through the memory wall bottleneck. However, cell-level and array-level nonideal properties of resistive memory significantly degrade the reliability, performance, accuracy, and energy efficiency during memory access and analog computation. Cell-level nonidealities include nonlinearity, asymmetry, and variability. Array-level nonidealities include interconnect resistance, parasitic capacitance, and sneak current. This review summarizes practical solutions that can mitigate the impact of nonideal device and circuit properties of resistive memory. First, we introduce several typical resistive memory devices with focus on their switching modes and characteristics. Second, we review resistive memory cells and memory array structures, including 1T1R, 1R, 1S1R, 1TnR, and CMOL. We also overview three-dimensional (3D) cross-point arrays and their structural properties. Third, we analyze the impact of nonideal device and circuit properties during memory access and analog arithmetic operations with focus on dot-product and matrix-vector multiplication. Fourth, we discuss the methods that can mitigate these nonideal properties by static parameter and dynamic runtime co-optimization from the viewpoint of device and circuit interaction. Here, dynamic runtime operation schemes include line connection, voltage bias, logical-to-physical mapping, read reference setting, and switching mode reconfiguration. Then, we highlight challenges on multilevel cell cross-point arrays and 3D cross-point arrays during these operations. Finally, we investigate design considerations of memory array peripheral circuits. We also portray an unified reconfigurable computational memory architecture.

Nowadays, high-bandwidth networks are more easily accessible than ever before. However, existing distributed graph-processing frameworks, such as GPS, fail to efficiently utilize the additional bandwidth capacity in these networks for higher performance, due to their inefficient computation and communication models, leading to very long waiting times experienced by users for the graph-computing results. The root cause lies in the fact that the computation and communication models of these frameworks generate, send and receive messages so slowly that only a small fraction of the available network bandwidth is utilized. In this paper, we propose a high-performance distributed graph-processing framework, called BlitzG, to address this problem. This framework fully exploits the available network bandwidth capacity for fast graph processing. Our approach aims at significant reduction in (i) the computation workload of each vertex for fast message generation by using a new slimmed-down vertex-centric computation model and (ii) the average message overhead for fast message delivery by designing a light-weight message-centric communication model. Evaluation on a 40Gbps Ethernet, driven by real-world graph datasets, shows that BlitzG outperforms GPS by up to 27x with an average of 20.7x.

In order to save human lives and reduce injury and property loss, Situation Awareness (SA) information is essential and important for rescue workers to perform the effective and timely disaster relief. The information is generally derived from the shared images via widely used smartphones. However, conventional smartphone-based image sharing schemes fail to efficiently meet the needs of SA applications due to two main reasons, i.e., real-time transmission requirement and application-level image redundancy, which is exacerbated by limited bandwidth and energy availability. In order to provide efficient image sharing in disasters, we propose a bandwidth- and energy- efficient image sharing system, called BEES. The salient feature behind BEES is to propose the concept of Approximate Image Sharing (AIS), which explores and exploits approximate feature extraction, redundancy detection, and image uploading to trade the slightly low quality of computation results in content-based redundancy elimination for higher bandwidth and energy efficiency. Nevertheless, the boundaries of the tradeoffs between the quality of computation results and efficiency are generally subjective and qualitative. We hence propose the energy-aware adaptive schemes in AIS to leverage the physical energy availability to objectively and quantitatively determine the tradeoffs between the quality of computation results and efficiency. Moreover, unlike existing work only for cross-batch similar images, BEES further eliminates in-batch ones via a similarity-aware submodular maximization model. The response time of querying similar images is reduced via leveraging a geographic coordinate based index partitioning scheme. We have implemented the BEES prototype which is evaluated via three real-world image datasets. Extensive experimental results demonstrate the efficacy and efficiency of BEES. We have released the source codes of BEES at GitHub.

SLO enforcement with the required strong SLO compliance and the desired low level of performance variability is necessary to ensure QoS for user applications with precisely differentiated service levels. However, for the cloud consolidating a large number of VMs rented by users, it is a great challenge to formulate an IO capacity allocation among consolidated VMs under the user-customized QoS constraints of SLO compliance and performance fluctuation for consolidated VMs. To address this challenge, we propose SASLO, an end-to-end VM-oriented control framework that supports users in customizing SLO targets and QoS constraints for each VM. SASLO can dynamically coordinate the throughput target and IO size limit for each VM adapting to the status of SLO enforcement so as to maximize the IO capacity allocation among consolidated VMs under QoS constraints. To accurately enforce time-varying throughput target, SASLO establishes a proportional-integral IO controller for each individual VM to converge the actual throughput to the target with an expected settling time. Our extensive evaluation driven by representative benchmarks demonstrates that SASLO is able to formulate a satisfactory IO capacity allocation plan for consolidated VMs under the constraints of SLO compliance and performance variability.

The recent development of CNN-based image dehazing has revealed the effectiveness of end-to-end modeling. However, extending the idea to end-to-end video dehazing has not been explored yet. In this paper, we propose an End-to-End Video Dehazing Network (EVD-Net), to exploit the temporal consistency between consecutive video frames. A thorough study has been conducted over a number of structure options, to identify the best temporal fusion strategy. Furthermore, we build an End-to-End United Video Dehazing and Detection Network (EVDD-Net), which concatenates and jointly trains EVD-Net with a video object detection model. The resulting augmented end-to-end pipeline has demonstrated much more stable and accurate detection results in hazy video.

This paper proposes a simple CNN model for creating general-purpose sentence embeddings that can transfer easily across domains and can also act as effective initialization for downstream tasks. Recently, averaging the embeddings of words in a sentence has proven to be a surprisingly successful and efficient way of obtaining sentence embeddings. However, these models represent a sentence, only in terms of features of words or uni-grams in it. In contrast, our model (CSE) utilizes both features of words and n-grams to encode sentences, which is actually a generalization of these bag-of-words models. The extensive experiments demonstrate that CSE performs better than average models in transfer learning setting and exceeds the state of the art in supervised learning setting by initializing the parameters with the pre-trained sentence embeddings.

Emerging Non-Volatile Memories (NVMs) such as Phase Change Memory (PCM) and Resistive RAM (RRAM) are promising to replace traditional DRAM technology. However, they suffer from limited write endurance and high write energy consumption. Encoding methods such as Flip-N-Write, FlipMin and CAFO can reduce the bit flips of NVMs by exploiting additional capacity to store the tag bits of encoding methods. The effects of encoding methods are limited by the capacity overhead of the tag bits. In this paper, we propose COE to COmpress cacheline for Extending the lifetime of NVMs. COE exploits the space saved by compression to store the tag bits of data encoding methods. Through combining data compression techniques with data encoding methods, COE can reduce the bit flips with negligible capacity overhead. We further observe that the saved space size of each compressed cacheline varies, and different encoding methods have different tradeoffs between capacity overhead and effects. To fully exploit the space saved by compression for improving lifetime, we select the proper encoding methods according to the saved space size. Experimental results show that our scheme can reduce the bit flips by 14.2%, decrease the energy consumption by 11.8% and improve the lifetime by 27.5% with only 0.2% capacity overhead.

Phase Change Memory (PCM) has the potential to replace traditional DRAM memory due to its better scalability and non-volatility. However, PCM also suffers from high write latency and energy consumption. To mitigate the write overhead of PCM-based main memory, we propose a Fine-grained Dirtiness Aware (FDA) last-level cache (LLC) victimization scheme. The key idea of FDA is to preferentially evict cachelines with fewer dirty words when victimizing dirty cachelines. The modified word is defined to be dirty. FDA exploits two key observations. First, the write service time of a cacheline is proportional to the number of dirty words. Second, a cacheline with fewer dirty words has the same or lower reference frequency compared with other dirty cachelines. Therefore, evicting cachelines with fewer dirty words can reduce the write service time of cachelines and will not increase the miss rate. To reduce the write service time of cachelines, FDA evicts the cacheline with the fewest dirty words when victimizing dirty cachelines. We also present FDARP to decrease the miss rate by further synergizing the number of dirty words with Re-reference Prediction Value. Experimental results show that FDA (FDARP) can improve the IPC performance by 8.3% (14.8%), decrease the write service time of cachelines by 37.0% (36.3%) and reduce write energy consumption of PCM by 27.0% (32.5%) under the mixed benchmarks.

Passive crossbar arrays of resistive random-access memory (RRAM) have shown great potential to meet the demands of future memory. By eliminating transistor per cell, the crossbar array possesses a higher memory density but introduces sneak currents which incur extra energy waste and reliability issues. The complementary resistive switch (CRS), consisting of two anti-serially stacked memristors, is considered as a promising solution to the sneak current problem. However, the destructive read of the CRS results in an additional recovery write operation which strongly restricts its further promotion. Exploiting the dual CRS/memristor mode of CRS devices, we propose Aliens, a novel hybrid architecture for resistive random-access memory which introduces one alien cell (memristor mode) for each wordline in the crossbar to provide a practical hybrid memory without operating system's intervention. Aliens draws advantages from both modes: restrained sneak current of CRS mode and non-destructive read of memristor mode. The simple and regular cell mode organization of Aliens enables an energy-saving read method and an effective mode switching strategy called Lazy-Switch. By exploiting memory access locality, Lazy-Switch delays and merges the recovery write operations of the CRS mode. Due to fewer recovery write operations and negligible sneak currents, Aliens achieves improvement in energy, overall endurance, and access performance. The experiment results show that our design offers average energy savings of 13.9× compared with memristor-only memory, a memory lifetime 5.3× longer than CRS-only memory, and a competitive performance compared with memristor-only memory.

Nowadays, there are lots of disks from various disk models in datacenter. It is a challenge to make failure prediction for all disk models with high precision and high coverage. One-for-one modeling, transfer learning modeling and one-for-all modeling are proposed to address the challenge. However, none of them works well for all disk models and the automation problem for method selection and parameter tuning still persists. In this paper, we propose OME, an optimized modeling engine for disk failure prediction in heterogeneous datacenter. It builds a basis predictive model with one-for-all modeling and searches for the optimized with one-for-one and transfer learning modeling for every disk model. To achieve automation, OME employs a simple but effective transfer learning method, does cross-validation for comparison, prunes the tuning space, and constructs a directed acyclic graph for parallelism. Evaluation on a dataset from a real-world datacenter shows that OME outperforms a one-for-all predictive model from previous work by 18.5% overall, and the improvement for 43.3% disk models reaches over 30%.

Parity-based RAID provides system-level fault tolerance. However, parity updates caused by small writes introduce lots of extra I/Os, degrading I/O performance and wearing SSDs out. It has been proposed to use Non-Volatile Memory (NVM) as a parity cache on an SSD RAID to postpone parity updates until the whole stripe has been updated. However, this often fails because of skewed distribution of hot data chunks within a stripe. In real workloads, it is often difficult to achieve a full-stripe update even after a long delay. In this paper, we propose a Correlation aware parity caching scheme, called Cap, for SSD-based RAIDs. The key idea behind Cap is to periodically reconstruct correlated hot data chunks into a new stripe. Since these data chunks have a strong correlation, they tend to be updated together within a short time span. This co-update within a stripe more efficiently utilizes the parity cache to convert partial-stripe updates into a full-stripe update. We have implemented Cap on a RAID-5 SSD array in Linux Kernel 4.3. Experimental results show that Cap improves the I/O bandwidth by 54%~145% compared with the Linux software RAID. Compared with the state-of-the-art parity caching scheme PPC, Cap improves the I/O bandwidth by 14%~31%.

Data deduplication, a lossless data compression technique, has been widely deployed in backup systems to save storage space. However, data fragmentation introduced by data deduplication seriously degrades restore performance. To alleviate the fragmentation problem, rewriting algorithms such as CBR, CAP, and HAR have been proposed. In certain backup datasets containing packed files like UNIX tar files, we have observed that a large amount of rewritten fragmented chunks tend to remain fragmented in following backups and thus get rewritten repeatedly in consecutive backups. Such repeatedly rewritten chunks are referred to as persistent fragmented chunks (PFCs), and they severely impact the efficiency of rewriting algorithms and restore performance. In this paper, we propose PFCG, an efficient scheme to enhance the efficiency of rewriting algorithms and restore performance. The central idea of PFCG is to identify and differentiate PFCs from regular fragmented chunks, i.e., non-PFCs, and store these two types of fragmented chunks into separate containers during backups to prevent the proliferation of PFCs in following backups. Our experimental results based on six real-world datasets demonstrate that PFCG significantly improves the restore performance by 21% to 47% over the state-of-the-art rewriting algorithms, without sacrificing deduplication efficiency.

In distributed key-value stores, performance fluctuations generally occur across servers, especially when the servers are deployed in a cloud environment. Hence, the replica selected for a request will directly affect the response latency. In the context of key-value stores, even the state-of-the-art algorithm of replica selection still has considerable room for improving the response latency. In this paper, we present the fundamental factors that prevent replica selection algorithms from being effective. We address these factors by proposing NetRS, a framework that enables in-network replica selection for key-value stores. NetRS exploits emerging network devices, including programmable switches and network accelerators, to select replicas for requests. NetRS supports diverse algorithms of replica selection and is suited to the network topology of modern data centers. Compared with the conventional scheme of clients selecting replicas for requests, NetRS could effectively cut the response latency according to our extensive evaluations. Specifically, NetRS reduces the average latency by up to 48.4%, and the 99th latency by up to 68.7%.

In recent years, a number of out-of-core graph processing systems have been proposed to process graphs with billions of edges on just one commodity computer, due to their high cost efficiency. To obtain the better performance, these systems adopt a full I/O model that accesses all edges during the computation to avoid the ineffectiveness of random I/Os. Although this model ensures good I/O access locality, it loads a large number of useless edges when running graph algorithms that only require a small portion of edges in each iteration. A natural method to solve this problem is the on-demand I/O model that only accesses the active edges. However, this method only works well for the graph algorithms with very few active edges, since the I/O cost will grow rapidly as the number of active edges increases due to larger amount of random I/Os.

The development of non-volatile memory technologies (NVMs) has attracted interest in designing data structures that are efficiently adapted to NVMs. In this context, several NVM-friendly hashing schemes have been proposed to reduce extra writes to NVMs, which have asymmetric properties of reads and writes and limited write endurance compared with traditional DRAM. However, these works neither consider the cost of cacheline flush and memory fence nor provide mechanisms to maintain data consistency in case of unexpected system failures. In this paper, we propose a write-efficient and consistent hashing scheme, called group hashing. The basic idea behind group hashing is to reduce the consistency cost while guaranteeing data consistency in case of unexpected system failures. Our group hashing consists of two major contributions: (1) We use 8-byte failure-atomic write to guarantee the data consistency, which eliminates the duplicate copy writes to NVMs, thus reducing the consistency cost of the hash table structure. (2) In order to improve CPU cache efficiency, our group hashing leverages a novel technique, i.e., group sharing, which divides the hash table into groups and deploys a contiguous memory space in each group to deal with hash collisions, thus reducing CPU cache misses to obtain higher performance in terms of request latency. We have implemented group hashing and evaluated the performance by using three real-world traces. Extensive experimental results demonstrate that our group hashing achieves low request latency as well as high CPU cache efficiency, compared with state-of-the-art NVM-based hashing schemes.

In datacenter networks, multipath exists to facilitate parallel data transmission. Taking deployment challenges into account, some optimized alternatives (e.g. CLOVE, Hermes) to ECMP balance load at the virtual edge or hosts. However inaccuracies of congestion detection and reaction exist in these solutions. They either detect congestion through ECN and coarse-grained RTT measurements or are congestion-oblivious. These congestion feedbacks are not sufficient enough to indicate the accurate congestion status under asymmetry. And when rerouting events occur on multiple paths, ACKs with congestion feedback of other paths can improperly influence the current sending rate. Therefore, we explore how to balance load by solving above inaccuracy problems while ensuring good adaptation to commodity switches and existing network protocols. We propose ALB, an adaptive load-balancing mechanism based on accurate congestion feedback running at end hosts, which is resilient to asymmetry. ALB leverage a latency-based congestion detection to precisely route flowlets to lighter load paths, and an ACK correction method to avoid inaccurate flow rate adjustment. In large-scale simulations ALB achieves up to 7% and 40% better flow completion time (FCT) than CONGA and CLOVE-ECN under asymmetry.

Storage hierarchy is widely used to mitigate the vast performance gap between different storage components economically, and the cache plays an important role in increasing the efficiency of memory access. However, the in-memory data organization of traditional graph computing framework is not well-optimized for various caches, especially the CPU cache, since classical caches are effectiveless towards irregular access pattern of graph applications. This work presents a cache- friendly graph data layout strategy to improve the efficiency of graph processing. By both considering the parameters of cache line and the pattern of access to adjacent list, we sort the edges to generate a sequential layout, and use BFS (Breadth First Search) algorithm to reorder the vertices for improving locality, thus benefit the CPU cache, without altering the code of graph processing toolkits. The efficiency improvement of Sort layout ranges from 11% up to 78.76% on BGL and SNAP with CC (Connected Components [1]). The BFS layout can benefit 3 classical algorithms on GraphChi: CC, TC (Triangle Counting) and PageRank [2], with the ratios of 15%-20%.

As promising alternatives in building future main memory systems, emerging non-volatile memory (NVM) technologies can increase memory capacity in a cost-effective and power-efficient way. However, NVM is facing security threats due to its limited write endurance: a malicious adversary can wear out the cells and cause the NVM system to fail quickly. To address this issue, several wear-leveling schemes have been proposed to evenly distribute write traffic in a security-aware manner. In this study, we present a new type of timing attack, remapping timing attack (RTA), based on information leakage from the remapping latency difference in NVM. Our analysis and experimental results show that RTA can cause three of the latest wear-leveling schemes (i.e., region-based start-gap, security refresh, and multi-way wear leveling) to lose their effectiveness in several days (even minutes), causing failure of NVM. To defend against such an attack, we further propose a novel wear-leveling scheme called the ‘security region-based start-gap (security RBSG)’, which is a two-stage strategy using a dynamic Feistel network to enhance the simple start-gap wear leveling with level-adjustable security assurance. The theoretical analysis and evaluation results show that the proposed security RBSG not only performs well when facing traditional malicious attacks, but also better defends against RTA.

Emerging Resistive Random Access Memory (ReRAM) is a promising candidate as the replacement for DRAM due to its low standby power, high density, high scalability, and nonvolatility. By employing the unique crossbar structure, ReRAM can be constructed with extremely high density. However, the crossbar ReRAM faces some serious challenges in terms of performance, reliability, and energy consumption. First, ReRAM’s crossbar structure causes an IR drop problem due to wire resistance and sneak currents, which results in nonuniform access latency in ReRAM banks and reduces its reliability. Second, without access transistors in the crossbar structure, write disturbance results in serious data reliability problem. Third, the access latency, reliability, and energy use of ReRAM arrays are significantly influenced by the data patterns involved in a write operation. To overcome the challenges of the crossbar ReRAM, we propose a novel circuit architecture co-optimization framework for improving the performance, reliability, and energy use of ReRAM-based main memory system, called CACF. The proposed CACF consists of three levels, including the circuit level, circuit architecture level, and architecture level. At the circuit level, to reduce the IR drops along bitlines, we propose a double-sided write driver design by applying write drivers along both sides of bitlines and selectively activating the write drivers. At the circuit architecture level, to address the write disturbance with low overheads, we propose a RESET disturbance detection scheme by adding disturbance reference cells and conditionally performing refresh operations. At the architecture level, a region partition with address remapping method is proposed to leverage the nonuniform access latency in ReRAM banks, and two flip schemes are proposed in different regions to optimize the data patterns involved in a write operation. The experimental results show that CACF improves system performance by 26.1%, decreases memory access latency by 22.4%, shortens running time by 20.1%, and reduces energy consumption by 21.6% on average over an aggressive baseline. Meanwhile, CACF significantly improves the reliability of ReRAM-based memory systems.

Deduplication has been widely used to improve storage efficiency in modern primary and secondary storage systems, yet how deduplication fundamentally affects storage system reliability remains debatable. This paper aims to analyze and compare storage system reliability with and without deduplication in primary workloads using public file system snapshots from two research groups. We first study the redundancy characteristics of the file system snapshots. We then propose a trace-driven, deduplication-aware simulation framework to analyze data loss in both chunk and file levels due to sector errors and whole-disk failures. Compared to without deduplication, our analysis shows that deduplication consistently reduces the damage of sector errors due to intra-file redundancy elimination, but potentially increases the damages of whole-disk failures if the highly referenced chunks are not carefully placed on disk. To improve reliability, we examine a deliberate copy technique that stores and repairs first the most referenced chunks in a small dedicated physical area (e.g., 1 percent of the physical capacity), and demonstrate its effectiveness through our simulation framework.

Hybrid storage is widely implemented as it satisfies the requirements of capacity and performance in an economically viable fashion. With the fast technical improvement, Hybrid storage systems consisting of several types of SSDs will be adopted gradually. Existing works mostly concentrate on thoroughly utilizing high-performance device but neglect the capability of low-performance device. This paper proposes a device crossfire method to boost hybrid storage performance by efficiently leveraging both high-performance and low-performance devices. Performance-critical data are appropriately off-loaded to low-performance device to exploit access parallelism. The implemented storage system CFStore exhibits good performance during experiments. Compared to famous hybrid storage system Hystor, CFStore improves throughput by 17.9%-42.6%, and reduces latency by 15.9%-35.0%.

Emerging Resistive Memory (ReRAM) is a promising candidate as the replacement for DRAM because of its low power consumption, high density and high endurance. Due to the unique crossbar structure, ReRAM can be constructed with a very high density. However, ReRAM's crossbar structure causes an IR drop problem which results in non-uniform access latency in ReRAM banks and reduces its reliability. Besides, the access latency and reliability of ReRAM arrays are greatly influenced by the data patterns involved in a write operation. In this paper, we propose a performance and reliability efficient ReRAM-based main memory structure. At the circuit level, we propose a double-sided write driver design to reduce the IR drops along bitlines. At the architecture level, a region partition with address remapping method and two flip schemes are proposed to reduce the access latency and improve the reliability of ReRAM arrays. The experimental results show that the proposed design can improve the system performance by 30.3% on average and reduce the memory access latency by 25.9% on average over an aggressive baseline, meanwhile the design improves the reliability of ReRAM-based memory system.

The page size of NAND flash continuously grows as the manufacturing process advances. While larger page can reduce the cost per bit and improve the throughput of NAND flash, it may waste the storage space and data transfer time. Meanwhile, it causes more frequent garbage collections when serving small write requests. To address the issues, we proposed a Mapping Granularity Adaptive FTL (MGA-FTL) based on flash page re-programming feature. MGA-FTL enables a finer granularity NAND flash space management and exploits multiple subpage writes on a single flash page without erase. 2-Level Mapping is introduced to serve requests of different sizes in order to control the overhead of DRAM requirement. Meanwhile, the allocation strategy determines whether different logical pages can be mapped to a single physical page to balance the space utilization and performance. Subpage merging limits the number of associated physical pages to a logical page, which could reduce data fragmentation and improves the performance of read operations. We compared MGA-FTL with some typical FTLs, including page-level mapping FTL and sector-log mapping FTL. Experimental results show that MGA-FTL reduces the I/O response time, write amplification and the number of erasures by 53%, 30% and 40% respectively. Despite the overhead of finegrained management, MGA-FTL increases no more than 16.5% DRAM requirement compared with a page-level mapping FTL. Unlike the subpage-level mapping, MGA-FTL only needs one third of DRAM space for storing mapping tables.

Counter-mode encryption has been widely used to resist NVMs from malicious attacks, due to its proved security and high performance. However, this scheme suffers from the counter size versus re-encryption problem, where per-line counters must be relatively large to avoid counter overflow, or re-encryption of the entire memory is required to ensure security. In order to address this problem, we propose a novel wear-leveling-aware counter mode for data encryption, called Resetting Counter via Remapping (RCR). The basic idea behind RCR is to leverage wear-leveling remappings to reset the line counter. With carefully designed procedure, RCR avoids counter overflow with much smaller counter size. The salient features of RCR include low storage overhead of counters, high counter cache hit ratio, and no extra re-encryption overhead. Compared with state-of-the-art works, RCR obtains significant performance improvements, e.g., up to a 57% reduction in the IPC degradation, under the evaluation of 8 memory-intensive benchmarks from SPEC 2006.

NAND flash-based Solid State Drives (SSDs) employ the Flash Translation Layer (FTL) to perform logical-to-physical address translation. Modern page-level FTLs selectively cache the address mappings in the limited SRAM while storing the mapping table in flash pages (called translation pages). However, many extra accesses to the translation pages are required for address translation, which decreases the performance and lifetime of an SSD. In this paper, we propose a Log-aware Synergized scheme for page-level FTL to reduce the extra overheads, called LSFTL. The contribution of LSFTL consists of two key elements: (i) By exploiting the partial programmability of SLC flash, in-place logging decreases garbage collection overhead via reserving a small portion of each translation page as a logging area to hold multiple updates to the entries of that translation page. (ii) Log-aware flush back reduces the number of translation page updates by evicting multiple dirty cache lines that share the same translation page in a single transaction. Extensive experimental results of trace-driven simulations show that LSFTL decreases the system response time by 39.40% on average, and up to 58.35%, and reduces the block erase count by 37.55% on average, and up to 39.99%, compared to the well-known DFTL.

3D architectures are considered the most promising approach to continuously increasing memory density and reducing cost/bit for NAND flash memory by stacking more layers. However, 3D MLC flash memory brings two serious problems, referred to as cell-to-cell program disturbance and big block problem. To solve the disturbance problem for better reliability, we proposed a Disturbance Compensation Programming Scheme (DCPS). Based on quantitatively analyzing the disturbance from each direction in 3D flash memory, the scheme accordingly set the verify voltage (VVFY) a little lower than the original value when Incremental Step Pulse Programming (ISPP) is performed. After disturbance compensation, the threshold voltage of flash cells can shift towards the ideal distribution. Moreover, Read reference Voltage Shifting (RVS) and Artificial Compensation (AC) strategies on margin pages are introduced to adapt to the three-dimensional structures to further enhance reliability. To solve big block problem, Multiple-Level-Queue page allocation (MLQ) is proposed. We use multiple queues to filter the logical addresses of different update counts and choose different data blocks to respond. The stored data are gradually well organized and generate less data migration when performing garbage collection. Experimental results show that our design reduces the disturbed BER by at least 82% with respect to a FTL with traditional allocation and garbage collection scheme. Besides, we demonstrate MLQ can achieve more effective results than the state of the art scheme in the big block environment. The write amplification, I/O response time and the number of erasures are reduced by 31.2%, 22.2% and 14.6% on average, respectively.

Resistive random access memory (RRAM) is promising to be used as high density storage-class memory by employing crossbar structure. However, the wire resistance in crossbar array causes the IR drop problem, which makes nonuniformity of write latency throughout the array. In large crossbar array, the write latency differs greatly even in the same row. Since the write latency of a region is determined by its slowest write-unit, the conventional group-by-row region partition and addressing scheme is suboptimal for improving the overall performance of RRAM. In this work, we present DAWS, a novel RRAM architecture that exploits intrinsic features of crossbar structure. We first build a circuit model to analyze the voltage distribution and write latency distribution in a crossbar array. Then we propose a voltage bias scheme to optimize write latency via minimizing the IR drop path. We further present block diagonal partition to narrow the variance of write latency within each region, thus the write latency of each region is reduced. Moreover, we provide block diagonal addressing to make the write latency monotonically increase with the physical address, which is in favor of address mapping and memory allocation. We also design diagonal writing and diagonal swapping to overlap SET and RESET operations by applying a particular voltage bias pattern that can exploit row level parallelism, thus the number of write operations is halved. The experimental results show that DAWS can reduce memory access latency by 24.0% and improve system performance by 29.7% over an aggressive baseline.

Resistive Random Access Memory (RRAM) technology is proposed as a promising replacement candidate for DRAM-based main memory due to its good scalability, low standby power, and non-volatility. The structure of Triple-Level Cell (TLC) can offer higher data density over Single-Level Cell (SLC). However, TLC RRAM suffers from high write energy and latency. Data compression techniques can reduce the size of the data to store. In contrast, data encoding methods such as Incomplete Data Mapping (IDM) can 'expand' the size for latency and energy reduction. We observe that the compression ratio of each cacheline varies, and therefore the saved space of each compressed cacheline is different. On the other hand, we find that different IDMs have different tradeoffs in capacity and write latency/energy. To fully exploit the space saved by compression for reducing the write latency/energy, and improving the performance of TLC RRAM-based main memory system, Compression-Ratio-Aware Data Encoding (CRADE) is proposed. The key idea of CRADE is to dynamically select the best-performing IDM according to the compression ratio of each cacheline. The cacheline is compressed first, and then the compressed cacheline is encoded by IDM. For each compressed cacheline, the IDM which uses the fewest states to encode is applied on the condition that the encoded data size will not exceed the cacheline size. Experimental results show that CRADE can reduce the write energy by 15%, decrease the write latency by 19%, reduce the read latency by 4%, and improve the IPC performance by 2% compared with the state-of-the-art scheme.

Multi Level Cell (MLC) Spin Transfer Torque RAM (STT-RAM) provides higher density than Single Level Cell (SLC) STT-RAM by storing two digital bits in a single cell, and is proposed as a promising candidate for on-chip cache. However, MLC STT-RAM suffers from high write energy. We observe that general encoding methods, which map the frequent data patterns to the energy-efficient resistance states, cannot reduce the write energy of MLC STT-RAM. To reduce the write energy of MLC STT-RAM, we propose a novel encoding method, i.e., Encoding Separately (ES). The key idea of ES is to encode the hard bits and soft bits of MLCs separately. The hard bits are encoded for fewer hard-bit writes (hard transitions) and soft bits are encoded for fewer soft-bit writes (soft transitions). Specifically, existing encoding methods commonly used in SLC can be applied to MLC STT-RAM when encoding the two bits separately. We further apply two encoding methods for SLC to MLC STT-RAM through encoding separately, and experimental results show that the proposed scheme can reduce the writes to hard bits and soft bits by 28% and 16%, and achieve an energy reduction of 25%.

Modern file systems employ journaling techniques to guarantee data consistency in case of unexpected system crashes or power failures. However, journaling file systems usually suffer from performance decrease due to the extra journal writes. Moreover, the emerging non-volatile memory technologies (NVMs) have the potential capability to improve the performance of journaling file systems by being deployed as the journaling storage devices. However, traditional journaling techniques, which are designed for hard disks, fail to perform efficiently in NVMs. In order to address this problem, we propose an NVM-based journaling scheme, called NJS. The basic idea behind NJS is to reduce the journaling overhead of traditional file systems while fully exploiting the byte-accessibility characteristic, and alleviating the relatively slow write and endurance limitation of NVM. Our NJS consists of three major contributions: (i) In order to minimize the amount of journal writes, NJS only needs to write the metadata of file systems and over-write data to NVM as write-ahead logging, thus alleviating the relatively slow write and endurance limitation of NVM. (ii) We propose a novel journaling update scheme in which the journaling data blocks can be updated in the byte-granularity based on the difference of the old and new versions of journal blocks, thus fully exploiting the unique byte-accessibility characteristic of NVM. (iii) NJS includes a garbage collection mechanism that absorbs the redundant journal updates, and actively delays the checkpointing to the file system. Evaluation results show the efficiency and efficacy of NJS. For example, compared with original Ext4 with a ramdisk-based journaling device, the throughput improvement of Ext4 with our NJS is up to 137.1%.

This paper proposes an image dehazing model built with a convolutional neural network (CNN), called All-in-One Dehazing Network (AOD-Net). It is designed based on a re-formulated atmospheric scattering model. Instead of estimating the transmission matrix and the atmospheric light separately as most previous models did, AOD-Net directly generates the clean image through a light-weight CNN. Such a novel end-to-end design makes it easy to embed AOD-Net into other deep models, e.g., Faster R-CNN, for improving high-level tasks on hazy images. Experimental results on both synthesized and natural hazy image datasets demonstrate our superior performance than the state-of-the-art in terms of PSNR, SSIM and the subjective visual quality. Furthermore, when concatenating AOD-Net with Faster R-CNN, we witness a large improvement of the object detection performance on hazy images.

In order to save human lives and reduce injury and property loss, Situation Awareness (SA) information is essential and important for rescue workers to perform the effective and timely disaster relief. The information is generally derived from the shared images via widely used smartphones. However, conventional smartphone-based image sharing schemes fail to efficiently meet the needs of SA applications due to two main reasons, i.e., real-time transmission requirement and application-level image redundancy, which is exacerbated by limited bandwidth and energy availability. In order to provide efficient image sharing in disasters, we propose a bandwidth-and energy-efficient image sharing system, called BEES. The salient feature behind BEES is to propose the concept of Approximate Image Sharing (AIS), which explores and exploits approximate feature extraction, redundancy detection, and image uploading to trade the slightly low quality of computation results in content-based redundancy elimination for higher bandwidth and energy efficiency. Nevertheless, the boundaries of the tradeoffs between the quality of computation results and efficiency are generally subjective and qualitative. We hence propose the energy-aware adaptive schemes in AIS to leverage the physical energy availability to objectively and quantitatively determine the tradeoffs between the quality of computation results and efficiency. Moreover, unlike existing work only for cross-batch similar images, BEES further eliminates in-batch ones via a similarity-aware submodular maximization model. We have implemented the BEES prototype which is evaluated via three real-world image datasets. Extensive experimental results demonstrate the efficacy and efficiency of BEES.

As a fundamental cloud service for modern Web applications, the cloud object storage system stores and retrieves millions or even billions of read-heavy data objects. Serving for a massive amount of requests each day makes the response latency be a vital component of user experiences. Due to the lack of suitable understanding on the response latency distribution, current practice is to use overprovision resources to meet Service Level Agreement (SLA). Hence we build a performance model for the cloud object storage system to predict the percentiles of requests meeting SLA (response latency requirement), in the context of complicated disk operations and event-driven programming model. Furthermore, we find that the waiting time for being accept()-ed at storage servers may introduce significant delay. And we quantify the impacts on system response latency, due to requests waiting for being accept()-ed. In a variety of scenarios, our model reduces the prediction errors by up to 73% compared to baseline models, and the prediction error of our model is 4.44% on average.

Modern distributed storage systems often store redundant data in multiple replications or erasure coding according to their access frequencies. Multiple replications scheme is well-performance for hot data while erasure coding scheme is storage-efficient for warm and cold data. When hot data turn cold, an encoding procedure starts to do the conversion. However, due to sequential striping, current conversion methods do not perform well for different data layouts, and cause risky blocks and expensive network consumption.In this paper, we propose Sice, a new encoder which deploys non-sequential striping. It constructs non-sequential stripes according to the data layout, performs conversion quickly with low overheads and ends to no reduction of system reliability. The results of both simulation and evaluation show that Sice gains almost the same good performance for different data layouts and has a great scalability. Sice helps HDFS-RAID reduce network consumption by about 65% and reduce influence on concurrent I/O-intensive applications by about 60%.

Nowadays, high-bandwidth networks are easily accessible in data centers. However, existing distributed graph-processing frameworks fail to efficiently utilize the additional bandwidth capacity in these networks for higher performance, due to their inefficient computation and communication models, leading to very long waiting times experienced by users for the graph-computing results. The root cause lies in the fact that the computation and communication models of these frameworks generate, send and receive messages so slowly that only a small fraction of the available network bandwidth is utilized. In this paper, we propose a high-performance distributed graph-processing framework, called BlitzG, to address this problem. This framework fully exploits the available network bandwidth capacity for fast graph processing. Our approach aims at significant reduction in (i) the computation workload of each vertex for fast message generation by using a new slimmed-down vertex-centric computation model and (ii) the average message overhead for fast message delivery by designing a lightweight message-centric communication model. Evaluation on a 40Gbps Ethernet, driven by real-world graph datasets, shows that BlitzG outperforms the state-of-the-art distributed graph-processing frameworks by up to 27x with an average of 20.7x.

Live migration of virtual machines (VM) is one of the key characteristics of virtualization for load balancing, system maintenance, power management, etc., in data centers or clusters. In order to reduce the data transferred and shorten the migration time, the compression techniques have been widely used to accelerate VM migration. However, different compression approaches have different compression ratios and speeds. Because there is a trade-off between compression and transmission, the migration performance improvements obtained from different compression approaches are differentiated, and the improvements vary with the network bandwidth. Besides, the compression window sizes used in most compression algorithms are typically much larger than a single page size, so the traditional single page compression loses some potential compression benefits. In this paper, we design and implement a Bandwidth-Aware Compression (BAC) scheme for VM migration. BAC chooses suitable compression approach according to the network bandwidth available for the migration process, and employs multi-page compression. These features make BAC obtain more migration performance improvements from compression. Experiments under various network scenarios demonstrate that, compared with conventional compression approaches, BAC shortens the total migration time while achieving comparable performance for the total data transferred and the downtime.

Cloud datacenters use limited network resources to host complex and diverse applications that requires us to treat upper applications as a black box and provide low latency for latency-sensitive short flows. Many good performance schemes need to preliminarily obtain flow information (flow size, deadline or traffic distribution) or modify hardware or software, which leads to difficult use and inefficient in practice.

With the development of technology and price decline, flash-based Solid state drives (SSDs) are rapidly used to construct RAIDs by storage vendors. SSD does not need to seek and rotate, therefore, its read performance is much better than that of HDD. However, the small write performance of SSD is limited by its inherent characteristics such as out- of-place updates and garbage collection. The traditional parity-based RAID also has small write problem because of parity updating. SSD-based RAID, which is called RAIS, is generally based on the traditional RAID design and implementation. Consequently, handling small write requests is a serious challenge when SSD is used to construct parity-based RAID. In RAIS storage system, small write requests not only result in poor performance, but also shorten the lifetime of each SSD. In this paper, we propose a novel write through cache method, called CRAIS5, which uses a RAM as the write cache of RAIS5, and adopts the write-through mode to delay the parity update. The write-through cache method makes full use of the flash characteristics, and removes the pre-read operation. CRAIS5 improves the small write performance and reduces the erase time. We have implemented the CRAIS5 prototype in Disksim simulator, and used the real traces to evaluate the performance. The evaluations demonstrate that our CRAIS5 outperforms RAIS5, and PPC, on average, by 42.82%, and 34.49% respectively.

Chunk-level deduplication, while robust in removing duplicate chunks, introduces chunk fragmentation which decreases restore performance. Rewriting algorithms are proposed to reduce the chunk fragmentation and accelerate the restore speed. Delta compression can remove redundant data between non-duplicate but similar chunks which cannot be eliminated by chunk-level deduplication. Some applications use delta compression as a complement for chunk-level deduplication to attain extra space and bandwidth savings. However, we observe that delta compression introduces a new type of chunk fragmentation stemming from delta compressed chunks whose base chunks are fragmented. We refer to such delta compressed chunks as base-fragmented chunks. We found that this new type of chunk fragmentation has a more severely impact on the restore performance than the chunk fragmentation introduced by chunk-level deduplication and cannot be reduced by existing rewriting algorithms. In order to address the problem due to the base-fragmented chunks, we propose SDC, a scheme that selectively performs delta compression after chunk-level deduplication. The main idea behind SDC is to simulate a restore cache to identify the non-base-fragmented chunks and only perform delta compression for these chunks, thus avoiding the new type of chunk fragmentation. Due to the locality among the backup streams, most of the non-base-fragmented chunks can be detected by the simulated restore cache. Experimental results based on real-world datasets show that SDC improves the restore performance of the delta compressed and deduplicated backup system by 1.93X-7.48X, and achieves 95.5%-97.4% of its compression, while imposing negligible impact on the backup throughput.

Erasure codes are widely used in practical storage systems to prevent disk failure and data loss. However, these codes require excessive disk I/Os and network traffic for recovering unavailable data. As a result, the recovery performance of erasure codes is suboptimal. Among all erasure codes, Minimum Storage Regenerating (MSR) codes can achieve optimal repair bandwidth under the minimum storage during recovery, but some open issues remain to be addressed before applying them in real systems. In this paper, we present Hybrid Regenerating Codes (Hybrid-RC), a new set of erasure codes with optimized recovery performance and low storage overhead. The codes utilize the superiority of MSR codes to compute a subset of data blocks while some other parity blocks are used for reliability maintenance. As a result, our design is near-optimal with respect to storage and network traffic. We show that Hybrid-RC reduces the reconstruction cost by up to 21% compared to the Local Reconstruction Codes (LRC) with the same storage overhead. Most importantly, in Hybrid-RC, each block contributes only half the amount of data when processing a single block failure. Therefore, the number of I/Os consumed per block is reduced by 50%, which is of great help to balance the network load and reduce the latency.

The size of write unit in PCM, namely the number of bits allowed to be written concurrently at one time, is restricted due to high write energy consumption. It typically needs several serially executed write units to finish a cache line service when using PCM as the main memory, which results in long write latency and high energy consumption. To address the poor write performance problem, we propose a novel PCM write scheme called Min-WU (Minimize the number of Write Units). We observe data access locality that some frequent zero-extended values dominate the write data patterns in typical multi-threaded applications (more than 40 and 44.9 percent of all memory accesses in PARSEC workloads and SPEC 2006 benchmarks, respectively). By leveraging carefully designed chip-level data redistribution method, the data amount is balanced and the data pattern is the same among all PCM chips. The key idea behind Min-WU is to minimize the number of serially executed write units in a cache line service after data redistribution through sFPC (simplified Frequent Pattern Compression), eRW (efficient Reordering Write operations method) and fWP (fine-tuned Write Parallelism circuits). Using Min-WU, the zero parts of write units can be indicated with predefined prefixes and the residues can be reordered and written simultaneously under power constraints. Our design can improve the performance, energy consumption and endurance of PCM-based main memory with low space and time overhead. Experimental results of 12 multi-threaded PARSEC 2.0 workloads show that Min-WU reduces 44 percent read latency, 28 percent write latency, 32.5 percent running time and 48 percent energy while receiving 32 percent IPC improvement compared with the conventional write scheme with few memory cycles and less than 3 percent storage space overhead. Evaluation results of 8 SPEC 2006 benchmarks demonstrate that Min-WU earns 57.8/46.0 percent read/write latency reduction, 28.7 percent IPC improvement, 28 percent running time reduction and 62.1 percent energy reduction compared with the baseline under realistic memory hierarchy configurations.

Chunk-level deduplication plays an important role in backup storage systems. Existing Content-Defined Chunking (CDC) algorithms, while robust in finding suitable chunk boundaries, face the key challenges of (1) low chunking throughput that renders the chunking stage a serious deduplication performance bottleneck, (2) large chunk size variance that decreases deduplication efficiency, and (3) being unable to find proper chunk boundaries in low-entropy strings and thus failing to deduplicate these strings. To address these challenges, this paper proposes a new CDC algorithm called the Asymmetric Extremum (AE) algorithm. The main idea behind AE is based on the observation that the extreme value in an asymmetric local range is not likely to be replaced by a new extreme value in dealing with the boundaries-shifting problem. As a result, AE has higher chunking throughput, smaller chunk size variance than the existing CDC algorithms, and is able to find proper chunk boundaries in low-entropy strings. The experimental results based on realworld datasets show that AE improves the throughput performance of the state-of-the-art CDC algorithms by more than 2.3×, which is fast enough to remove the chunking-throughput performance bottleneck of deduplication, and accelerates the system throughput by more than 50 percent, while achieving comparable deduplication efficiency.

With the rapid growth of application migration, the anonymity in data center networks becomes important in breaking attack chains and guaranteeing user privacy. However, existing anonymity systems are designed for the Internet environment, which suffer from high computational and network resource consumption and deliver low performance, thus failing to be directly deployed in data centers. In order to address this problem, this paper proposes an efficient and easily deployed anonymity scheme for software defined networking-based data centers, called mimic channel (MIC). The main idea behind MIC is to conceal the communication participants by modifying the source/destination addresses, such as media access control (MAC) and Internet protocol (IP) address at switch nodes, so as to achieve anonymity. Compared with the traditional overlay-based approaches, our in-network scheme has shorter transmission paths and less intermediate operations, thus achieving higher performance with less overhead. We also propose a collision avoidance mechanism to ensure the correctness of routing, and three mechanisms to enhance the traffic-analysis resistance. To enhance the practicality, we further propose solutions to enable MIC co-existing with some MIC-incompatible systems, such as packet analysis systems, intrusion detection systems, and firewall systems. Our security analysis demonstrates that MIC ensures unlinkability and improves traffic-analysis resistance. Our experiments show that MIC has extremely low overhead compared with the base-line transmission control protocol (TCP) (or secure sockets layer (SSL)), e.g., less than 1% overhead in terms of throughput. Experiments on MIC-based distributed file system show the applicability and efficiency of MIC.

Repair performance in hierarchical data centers is often bottlenecked by cross-rack network transfer. Recent theoretical results show that the cross-rack repair traffic can be minimized through repair layering, whose idea is to partition a repair operation into inner-rack and cross-rack layers. However, how repair layering should be implemented and deployed in practice remains an open issue. In this article, we address this issue by proposing a practical repair layering framework called DoubleR . We design two families of practical double regenerating codes (DRC), which not only minimize the cross-rack repair traffic but also have several practical properties that improve state-of-the-art regenerating codes. We implement and deploy DoubleR atop the Hadoop Distributed File System (HDFS) and show that DoubleR maintains the theoretical guarantees of DRC and improves the repair performance of regenerating codes in both node recovery and degraded read operations.

Cloud service is being adopted as a utility for large numbers of tenants by renting Virtual Machines (VMs). But for cloud storage, unpredictable IO characteristics make accurate Service-Level-Objective (SLO) enforcement challenging. As a result, it has been very difficult to support simple-to-use and technology-agnostic SLO specifying a particular value for a specific metric (e.g., storage bandwidth). This is because the quality of SLO enforcement depends on performance error and fluctuation that measure the precision of SLO enforcement . High precision of SLO enforcement is critical for user-oriented performance customization and user experiences. To address this challenge, this article presents V-Cup, a framework for VM-oriented customizable SLO and its near-precise enforcement. It consists of multiple auto-tuners, each of which exports an interface for a tenant to customize the desired storage bandwidth for a VM and enable the storage bandwidth of the VM to converge on the target value with a predictable precision. We design and implement V-Cup in the Xen hypervisor based on the fair sharing scheduler for VM-level resource management. Our V-Cup prototype evaluation shows that it achieves satisfying performance guarantees through near-precise SLO enforcement.

Erasure codes are widely used in modern distributed storage systems to prevent data loss and server failures. Regenerating codes are a class of erasure codes that trade storage efficiency and computation for repair bandwidth reduction. However, their nonunified coding parameters and huge computational overhead prohibit their applications. Hence, we first propose a family of General Functional Regenerating (GFR) codes with uncoded repair, balancing storage efficiency and repair bandwidth with general parameters. The GFR codes take advantage of a heuristic repair algorithm, which makes efforts to employ as little repair bandwidth as possible to repair a single failure. Second, we also present a scheduled shift multiplication (SSM) algorithm, which accelerates the matrix product over the Galois field by scheduling the order of coding operations, so encoding and repairing of GFR codes can be executed by fast bitwise shifting and exclusive-OR. Compared to the traditional table-lookup multiplication algorithm, our SSM algorithm gains 1.2 to 2 X speedup in our experimental evaluations, with little effect on the repair success rate.

Erasure codes are widely used in distributed storage systems to prevent data loss. Traditional codes suffer from a typical repair-bandwidth problem in which the amount of data required to reconstruct the lost data, referred to as the repair bandwidth, is often far more than the theoretical minimum. While many novel codes have been proposed in recent years to reduce the repair bandwidth, these codes either require extra storage and computation overhead or are only applicable to some special cases. To address the weaknesses of the existing solutions to the repair-bandwidth problem, we propose Z Codes, a general family of codes capable of achieving the theoretical lower bound of repair bandwidth versus storage. To the best of our knowledge, the Z codes are the first general systematic erasure codes that jointly achieve optimal repair bandwidth and storage. Further, we generalize the Z codes to the GZ codes to gain the Maximum Distance Separable (MDS) property. Our evaluations of a real system indicate that Z/GZ and Reed-Solomon (RS) codes show approximately close encoding and repairing speeds, while GZ codes achieve over 37.5% response time reduction for repairing the same size of data, compared to the RS and Cauchy Reed-Solomon (CRS) codes.

The raw error rate of a Solid-State drive (SSD) increases gradually with the increase of Program/Erase (P/E) cycles, retention time, and read cycles. Traditional approaches often use Error Correction Code (ECC) to ensure the reliability of SSDs. For error-free flash memory pages, time costs spent on ECC are redundant and make read performance suboptimal. This article presents a CRC-Detect-First LDPC (CDF-LDPC) algorithm to optimize the read performance of SSDs. The basic idea is to bypass Low-Density Parity-Check (LDPC) decoding of error-free flash memory pages, which can be found using a Cyclic Redundancy Check (CRC) code. Thus, error-free pages can be read directly without sacrificing the reliability of SSDs. Experiment results show that the read performance is improved more than 50% compared with traditional approaches. In particular, when idle time of benchmarks and SSD parallelism are exploited, CDF-LDPC can be performed more efficiently. In this case, the read performance of SSDs can be improved up to about 80% compared to that of the state-of-art.

In this article, we design and implement a cooperative shingle-aware file system, called CosaFS , on heterogeneous storage devices that mix solid-state drives (SSDs) and shingled magnetic recording (SMR) technology to improve the overall performance of storage systems. The basic idea of CosaFS is to classify objects as hot or cold objects based on a proposed Lookahead with Recency Weight scheme. If an object is identified as a hot (small) object, then it will be served by SSD. Otherwise, cold (large) objects are stored on SMR. For an SMR, large objects can be accessed in large sequential blocks, rendering the performance of their accesses comparable with that of accessing the same large sequential blocks as if they were stored on a hard drive. Small objects, such as inodes and directories, are stored on the SSD where “seeks” for such objects are nearly free. With thorough empirical studies, we demonstrate that CosaFS, as a cooperative shingle-aware file system, with metadata separation and cache-assistance, is a very effective way to handle the disk-based data demanded by the shingled writes and outperforms the device- and host-side shingle-aware file systems in terms of throughput, IOPS, and access latency as well.

With the rapid growth of the amount of information, cloud computing servers need to process and analyze large amounts of high-dimensional and unstructured data timely and accurately. This usually requires many query operations. Due to simplicity and ease of use, cuckoo hashing schemes have been widely used in real-world cloud-related applications. However, due to the potential hash collisions, the cuckoo hashing suffers from endless loops and high insertion latency, even high risks of re-construction of entire hash table. In order to address these problems, we propose a cost-efficient cuckoo hashing scheme, called MinCounter. The idea behind MinCounter is to alleviate the occurrence of endless loops in the data insertion by selecting unbusy kicking-out routes. MinCounter selects the “cold” (infrequently accessed), rather than random, buckets to handle hash collisions. We further improve the concurrency of the MinCounter scheme to pursue higher performance and adapt to concurrent applications. MinCounter has the salient features of offering efficient insertion and query services and delivering high performance of cloud servers, as well as enhancing the experiences for cloud users. We have implemented MinCounter in a large-scale cloud testbed and examined the performance by using three realworld traces. Extensive experimental results demonstrate the efficacy and efficiency of MinCounter.

Due to the high complexity in software hierarchy and the shared queue & lock mechanism for synchronized access, existing I/O stack for accessing the FCoE based SAN storage becomes a performance bottleneck, thus leading to a high I/O overhead and limited scalability in multi-core servers. In order to address this performance bottleneck, we propose a synergetic and efficient solution that consists of three optimization strategies for accessing the FCoE based SAN storage: (1) We use private per-CPU structures and disabling kernel preemption method to process I/Os, which significantly improves the performance of parallel I/O in multi-core servers; (2) We directly map the requests from the block-layer to the FCoE frames, which efficiently translates I/O requests into network messages; (3) We adopt a low latency I/O completion scheme, which substantially reduces the I/O completion latency. We have implemented a prototype (called FastFCoE, a protocol stack for accessing the FCoE based SAN storage). Experimental results demonstrate that FastFCoE achieves efficient and scalable I/O throughput, obtaining 1132.1K/836K IOPS (6.6/5.4 times as much as original Linux Open-FCoE stack) for read/write requests.

It is desirable but challenging to simultaneously support latency SLO at a pre-defined percentile, i.e., the Xth percentile latency SLO, and throughput SLO for consolidated VM storage. Ensuring the Xth percentile latency contributes to accurately differentiating service levels in the metric of the application-level latency SLO compliance, especially for the application built on multiple VMs. However, the Xth percentile latency SLO and throughput SLO enforcement are the opposite sides of the same coin due to the conflicting requirements for the level of IO concurrency. To address this challenge, this paper proposes PSLO, a framework supporting the Xth percentile latency and throughput SLOs under consolidated VM environment by precisely coordinating the level of IO concurrency and arrival rate for each VM issue queue. It is noted that PSLO can take full advantage of the available IO capacity allowed by SLO constraints to improve throughput or reduce latency with the best effort. We design and implement a PSLO prototype in the real VM consolidation environment created by Xen. Our extensive trace-driven prototype evaluation shows that our system is able to optimize the Xth percentile latency and throughput for consolidated VMs under SLO constraints.

Existing distributed graph-processing frameworks, e.g.,GPS, Pregel and Giraph, handle large-scale graphs in the memory of clusters built of commodity compute nodes for better scalability and performance. While capable of scaling out according to the size of graphs up to thousands of compute nodes, for graphs beyond a certain size, these frameworks usually require the investments of machines that are either beyond the financial capability of or unprofitable for most small and medium-sized organizations. At the other end of the spectrum of graph-processing frameworks research, the single-node disk-based graph-processing frameworks, e.g., GraphChi, handle large-scale graphs on one commodity computer, leading to high efficiency in the use of hardware but at the cost of low user performance and limited scalability. Motivated by this dichotomy, in this paper we propose a distributed disk-based graph-processing framework, called DD-Graph, that can process super-large graphs on a small cluster while achieving the high performance of existing distributed in-memory graph-processing frameworks.

Data compression is widely used in storage systems to reduce redundant data and thus save storage space. One challenge facing the traditional compression approaches is the limitation of compression windows size, which fails to reduce redundancy globally. In this paper, we present DEC, a Deduplication-Enhanced Compression approach that effectively combines deduplication and traditional compressors to increase compression ratio and efficiency. Specifically, we make full use of deduplication to (1) accelerate data reduction by fast but global deduplication and (2) exploit data locality to compress similar chunks by clustering the data chunks which are adjacent to the same duplicate chunks. Our experimental results of a DEC prototype based on real-world datasets show that DEC increases the compression ratio by 20% to 71% and speeds up the compression throughput by 17%~183% compared to traditional compressors, without sacrificing the decompression throughput by leveraging deduplication in traditional compression approaches.

Tencent, one of the biggest Internet companies in China, contains billions of users and over 600-PB data, and leverages thousands of SSDs in the storage system to improve system performance and obtain energy savings. Existing commercial SSDs however fail to meet the needs of the ultra largescale applications due to not matching the service patterns. In order to address this problem and deliver high performance, we propose an application-aware and software-defined SSD scheme for Tencent applications, called TSSD. TSSD explores and exploits the business characteristics of Tencent, which facilitates the efficient use of SSDs. TSSD is software-defined by packaging each flash chip as a fully independent and concurrent storage unit. Each concurrent unit can be mounted as a character device, which allows the application layer to manage the flash chips in a more efficient manner, while optimizing the data layout. TSSD further employs a host-target FTL (TFTL) that uses a dedicated interface in the application layer, which efficiently connects the application layer with flash chips. Application layer hence becomes more accurately by using the flash memory chip-level information from TFTL, including the storage utilization, the degree of wear, etc. Moreover, TFTL is a programmable FTL and provides a programmable interface to the application layer. According to the running states of SSDs and workload information, TSSD makes use of the programmable interface to efficiently improve the performance of the FTL, wear leveling, and garbage collection for the specified applications. Extensive experiments use the real-world datasets from the commercial storage systems of Tencent. The results demonstrate that TSSD significantly improves the storage system performance and meets the needs of the Tencent's large-scale business applications.

Phase Change Memory (PCM) has emerged as a promising candidate for building the future main memory systems. However, the limited write endurance is one of the major obstacles for PCM to be practically applied. Traditional wear-leveling techniques try to uniformly balance the write traffics under both general applications and malicious attacks to enhance the PCM lifetime. However, these techniques fail to consider the endurance variation in PCM chips, and result in severe lifespan degradation since uniform write distribution leads to the weakest cell to be worn out much earlier. In this paper, we propose a weight-based algebraic wear-leveling (WAWL) scheme to balance wear rates (i.e., write traffics/endurance) in a secure manner according to the endurance distribution. In WAWL, the entire memory space is divided into multiple regions. When the number of the writes to a region reaches a threshold (i.e., swapping interval), the region is swapped with a randomly chosen region. The basic idea behind WAWL is that the swapping interval and the chosen probability of each region are variable and associated with the endurance metric of the region. By deploying suitable swapping interval and chosen probability, WAWL achieves uniform wear-rate distribution across the entire memory in an undetectable way. In addition, to reduce space consumption and alleviate performance degradation during region swapping, we propose a fine-grained swapping scheme which migrates the lines one-by-one between the candidate regions. Experimental evaluation driven by the various attacks demonstrates that WAWL significantly increases the PCM lifespan and improves security with slight performance degradation and affordable hardware overhead.

Parity-based RAID storage systems have been widely deployed in production environments. However, they suffer from poor random write performance due to the parity update overhead, i.e., small write problem. With the increasing density and decreasing price, SSD-based caching offers promising opportunities for improving RAID storage I/O performance. However, as a cache device, frequent writes to SSD leads to being quickly worn out, which causes high costs and reliability problems. In this paper, we propose an efficient cache management scheme by Keeping Data and Deltas (KDD) in SSD. KDD dynamically partitions the cache space into Data Zone (DAZ) and Delta Zone (DEZ). DAZ stores data that are first admitted into SSD. On write hits, KDD writes the data to RAID storage without updating the parity blocks. Meanwhile, the deltas between old version of data and the currently accessed data are compactly stored in DEZ. In addition, KDD organizes the metadata partition on SSD as a circular log to make the cache persistent with low overhead. We evaluate the performance of KDD via both simulations and prototype implementations. Results show that KDD effectively reduces the small write penalty while significantly improving the lifetime of the SSD-based cache.

The noises at the power lines limit the charge pump to provide large instantaneous current to PCM cells, which results in the number of bits can be written concurrently, i.e. the size of write unit, is restricted in PCM. When implementing PCM as the main memory, the inequality of cache line's size and write unit's size may result in many consecutive executed write units, which greatly decreases the system performance. Existing PCM write schemes, however, consider the worst power and time cases of written data, and ignore the actual current consumption. It is assumed that all data bits are changed and the electric current of each data unit is under fully utilized. The write performance is blocked due to pessimistic estimates, i.e. the current is often excessively supplied but is not used effectively, which leads to huge energy consumption. As a result, the write parallelism is limited and therefore restricts the overall system performance. To address this problem, this paper proposes a novel PCM write scheme named Tetris Write to explore more write parallelism and reduce the critical number of write units in PCM chip. The key idea behind Tetris Write is to monitor the number of '1' and '0' changed in each data unit, and schedule the order of data units' write-1 and write-0 execution considering not only the time and power asymmetries, but also the number asymmetry between RET and SET operations, to allow a larger number of concurrent bit-writes and make the best use of power supply. Tetris Write tries to schedule the dominating long term write-1s first and attempts to steal interspaces remained by write-1s to put the extraessential short write-0s. 4-core PARSEC benchmarks' results show that Tetris Write can get 65% read latency reduction, 40% write latency reduction, 46% running time reduction and 2X IPC improvement compared with the baseline on average. In addition, Tetris Write earns 26%, 15% and 10% more read latency reduction, 15%, 7% and 5% more write latency reduction, and outperforms 22%, 12% and 7% more running time reduction, compared with the state-of-the-art Flip-N-Write, 2-Stage-Write and Three-Stage-Write schemes, whose IPC improvements are 1.4X, 1.6X and 1.8X, respectively.

With the rapid growth of application migration, the anonymity in data center networks becomes important in breaking attack chains and guaranteeing user privacy. However, existing anonymity systems are designed for the Internet environment, which suffer from high computational and network resource consumption and deliver low performance, thus failing to be directly deployed in data centers. In order to address this problem, this paper proposes an efficient and easily deployed anonymity scheme for SDN-based data centers, called MIC. The main idea behind MIC is to conceal the communication participants by modifying the source/destination addresses (such as MAC, IP and port) at switch nodes, so as to achieve anonymity. Compared with the traditional overlay-based approaches, our in-network scheme has shorter transmission paths and less intermediate operations, thus achieving higher performance with less overhead. We also propose a collision avoidance mechanism to ensure the correctness of routing, and two mechanisms to enhance the traffic-analysis resistance. Our security analysis demonstrates that MIC ensures unlinkability and improves traffic-analysis resistance. Our experiments show that MIC has extremely low overhead compared with the base-line TCP (or SSL), e.g., less than 1% overhead in terms of throughput.

As an emerging memory technology to build the future main memory systems, Phase Change Memory (PCM) can increase memory capacity in a cost-effective and power-efficient way. However, PCM is facing security threats for its limited write endurance: a malicious adversary could wear out the cells and cause the whole PCM system to fail within a short period of time. To address this issue, several wear-leveling schemes have been proposed to evenly distribute write traffic in a security-aware manner. In this work, we present a new type of timing attacknamed Remapping Timing Attack (RTA), based on the asymmetry in write time of PCM. Our analysis and experimental results show that the new revealed RTA can make two state-of-the-art wear-leveling schemes (Region Based Start-Gap and Security Refresh) lose effectiveness, failing PCM with these two techniques in several days (even minutes). In order to defend such attack, we further propose a novel wear-leveling scheme called Security Region Based Start-Gap (Security RBSG), which employs a two-stage strategy and uses a dynamic Feistel Network to enhance the simple Start-Gap wear leveling with level-adjustable security assurance. The theoretical analysis and evaluation results show that the proposed Security RBSG is the most robust wear-leveling methodology so far, which not only better defends the new RTA, but also performs well on the traditional malicious attacks, i.e., Repeated Address Attack and Birthday Paradox Attack.

Phase Change Memory (PCM) is an emerging non-volatile memory with the salient features of large-scale, high-speed, low-power and radiation resistance. It hence becomes an ideal candidate for the next-generation storage media of main memory. However, PCM suffers from inefficient I/O performance due to long write latency. Recent studies propose a multi-partition (or multi-subarray) architecture within each bank to enhance internal parallelism. However, conventional scheduling schemes fail to exploit the advantage of multiple partitions and incur inefficient bank utilization. In this paper, we propose a Write Priority overlap Read (WPoR) scheduling scheme which preferentially serves for a write request in one partition and allows other partitions to perform as many read requests as possible within this partition's program duration. Experimental results demonstrate that WPoR reduces the write latency by 24.7% (on average) compared with state-of-the-art scheduling algorithms. Meanwhile, the IPC indicator of WPoR scheduling increases respectively 6%, 7% and 26% (on average) compared with Read Priority, Write Pausing and Write Cancellation schemes.

With the rapid growth of data, communication overhead has become an important concern in applications of data centers and cloud computing. However, existing distributed graph-processing frameworks routinely suffer from high communication costs, leading to very long waiting times experienced by users for the graph-computing results. In order to address this problem, we propose a new computation model with low communication costs, called LCC-BSP. We use this model to design and implement a high-performance distributed graph-processing framework called LCC-Graph. This framework eliminates the high communication costs in existing distributed graph-processing frameworks. Moreover, LCC-Graph also minimizes the computation workload of each vertex, significantly reducing the computation time for each superstep. Evaluation of LCC-Graph on a 32-node cluster, driven by real-world graph datasets, shows that it significantly outperforms existing distributed graph-processing frameworks in terms of runtime, particularly when the system is supported by a high-bandwidth network. For example, LCC-Graph achieves an order of magnitude performance improvement over GPS and GraphLab.

Continuously enriched distributed systems in data centers generate much network traffic in push-style one-to-many group mode, raising new requirements for multicast transport in terms of efficiency and robustness. Existing reliable multicast solutions, which suffer from low robustness and inefficiency in either host-side protocols or multicast routing, are not suitable for data centers. In order to address the problems of inefficiency and low robustness, we present a sender-initiated, efficient, congestion-aware and robust reliable multicast solution mainly for small groups in SDN-based data centers, called MCTCP. The main idea behind MCTCP is to manage the multicast groups in a centralized manner, and reactively schedule multicast flows to active and low-utilized links, by extending TCP as the host-side protocol and managing multicast groups in the SDN-controller. The multicast spanning trees are calculated and adjusted according to the network status to perform a better allocation of resources. Our experiments show that, MCTCP can dynamically bypass the congested and failing links, achieving high efficiency and robustness. As a result, MCTCP outperforms the state-of-the-art reliable multicast schemes. Moreover, MCTCP improves the performance of data replication in HDFS compared with the original and TCP-SMO based ones, e.g., achieves 101% and 50% improvements in terms of bandwidth, respectively.

In a WSN, sink-node gathers data from surrounding nodes and sends it to outside world via a gateway. Therefore, its location information is important to both attacker and network operator. The former can launch attacks on a sink-node to steal information or damage it, while the latter must hide its location to ensure data's safety, and physical protection. Providing sink-node location information anonymity in WSN against a local and global adversary with minimal energy penalties, we propose a novel technique Location Privacy via Software Defined Networking (LPSDN), inspired by the concept of Software Defined Networking. LPSDN uses three kinds of nodes: a centralized controller, a special buffering and forwarding node, and a slave node. These nodes work together to hide sink-node location information against traffic analysis attack. LPSDN conserves more energy, because it does not generate fake packets to hide sink-node location information. A GUI/command-line can be used to monitor (energy status, traffic density) and control the whole network. Comparing with traditional WSNs, LPSDN efficiently reduces processing burden on slave nodes, increases overall network life, and creates on-demand traffic density zones to deceive an adversary. It also provides better maintainability, manageability, and control over all nodes and their behavior.

NAND flash memory has been widely used as storage medium in diverse environments due to its light weight, low power consumption, and high I/O performance. Page replacement is an important operation in NAND flash- based storage systems. However, traditional replacement algorithms designed for magnetic disks fail to meet the needs of NAND flash memory due to inherent features such as asymmetric I/O latencies and erase-before- write. In order to address this problem, this paper proposes a new page replacement algorithm, called Ghost buffer Assisted and Self-tuning Algorithm (GASA). GASA reduces expensive flash write operations by evicting cold clean pages preferentially and maintains reasonable buffer hit ratios via a ghost buffer. In addition, GASA is self-tuning due to the use of a simple learning scheme, thus adaptively matching different workloads and buffer sizes. Experimental results based on real-world OLTP traces demonstrate that GASA offers a good trade-off between the hit ratio and the flash write count, thus achieving better I/O performance than the state-of-the-art page replacement algorithms.

Shingled write disk (SWD) is a new kind of HDD, which increases areal density by adopting SMR technology. In the hybrid storage system consisting of SSDs and SWDs, there are some special characteristics such as large performance gap between SSDs and SWDs. In this paper we explore how to efficiently combine the advantages of SSDs and SWDs together. We propose Apas, an application aware hybrid storage system combining SSDs and SWDs, where SSDs are used as nonvolatile cache and SWDs work as lower level storage. In Apas we propose a cache partitioning policy to divide the cache space according to the application's characteristics. To mitigate the write amplification problem in SWDs, we first classify data bands in the SWDs into five types according to the distribution of valid data in the bands, and then design a new garbage collection policy combining the aggressive data cleaning and lazy data cleaning together according to the detected idle time and the five types of bands. Experiments are carried out and the results show that Apas has good performance.

In modern RAID-structured storage systems, reliability is guaranteed by the use of parity blocks. But the parity-update overheads upon each write request have become a performance bottleneck of RAID systems. In some ways, an attached log disk is used to improve the write performance by delaying the parity blocks update. However, these methods are data-block-oriented and they need more time to rebuild or synchronize the RAID system when a data disk or the log disk fails. In this paper, we propose a novel optimization method, called SWO, which can improve RAID write performance and reconstruction performance. Moreover, when handling a write request, the SWO chooses reconstruction- write or read-modify-write combining with the log information to further minimize the number of pre- read data blocks. We have implemented the proposed SWO prototype and carried out some performance measurements using IOmeter and RAIDmeter. We have implemented the main idea of RAID6L in RAID5 and call it RAID5L. At the same time, we have evaluated the reconstruction time and the synchronization time of the SWO. Our experiments demonstrate that the SWO significantly improves write performance and saves more time than Data Logging and RAID5L when rebuilding and synchronizing.

Data deduplication, an efficient approach to data reduction, has gained increasing attention and popularity in large-scale storage systems due to the explosive growth of digital data. It eliminates redundant data at the file or subfile level and identifies duplicate content by its cryptographically secure hash signature (i.e., collision-resistant fingerprint), which is shown to be much more computationally efficient than the traditional compression approaches in large-scale storage systems. In this paper, we first review the background and key features of data deduplication, then summarize and classify the state-of-the-art research in data deduplication according to the key workflow of the data deduplication process. The summary and taxonomy of the state of the art on deduplication help identify and understand the most important design considerations for data deduplication systems. In addition, we discuss the main applications and industry trend of data deduplication, and provide a list of the publicly available sources for deduplication research and studies. Finally, we outline the open problems and future research directions facing deduplication-based storage systems.

Phase Change Memory (PCM) is one of the promising memory technologies but suffers from some critical problems such as poor write performance and high write energy consumption. Due to the high write energy consumption and limited power supply, the size of concurrent bit-write is restricted inside one PCM chip. Typically, the size of concurrent bit-write is much less than the cache line size and it is normal that many serially executed write units are consumed to write down the data block to PCM when using it as the main memory. Existing state-of-the-art PCM write schemes, such as FNW (Flip-N-Write) and two-stage-write, address the problem of poor performance by improving the write parallelism under the power constraints. The parallelism is obtained via reducing the data amount and leveraging power as well as time asymmetries, respectively. However, due to the extremely pessimistic assumptions of current utilization (FNW) and optimistic assumptions of asymmetries (two-stage-write), these schemes fail to maximize the power supply utilization and hence improve the write parallelism. In this article, we propose a novel PCM write scheme, called MaxPB (Maximize the Power Budget utilization) to maximize the power budget utilization with minimum changes about the circuits design. MaxPB is a “think before acting” method. The main idea of MaxPB is to monitor the actual power needs of all data units first and then effectively package them into the least number of write units under the power constraints. Experimental results show the efficiency and performance improvements on MaxPB. For example, four-core PARSEC and SPEC experimental results show that MaxPB gets 32.0% and 20.3% more read latency reduction, 26.5% and 16.1% more write latency reduction, 24.3% and 15.6% more running time decrease, 1.32× and 0.92× more speedup, as well as 30.6% and 18.4% more energy consumption reduction on average compared with the state-of-the-art FNW and two-stage-write write schemes, respectively.

Data reduction has become increasingly important in storage systems due to the explosive growth of digital data in the world that has ushered in the big data era. One of the main challenges facing large-scale data reduction is how to maximally detect and eliminate redundancy at very low overheads. In this paper, we present DARE, a low-overhead deduplication-aware resemblance detection and elimination scheme that effectively exploits existing duplicate-adjacency information for highly efficient resemblance detection in data deduplication based backup/archiving storage systems. The main idea behind DARE is to employ a scheme, call Duplicate-Adjacency based Resemblance Detection (DupAdj), by considering any two data chunks to be similar (i.e., candidates for delta compression) if their respective adjacent data chunks are duplicate in a deduplication system, and then further enhance the resemblance detection efficiency by an improved super-feature approach. Our experimental results based on real-world and synthetic backup datasets show that DARE only consumes about 1/4 and 1/2 respectively of the computation and indexing overheads required by the traditional super-feature approaches while detecting 2-10 percent more redundancy and achieving a higher throughput, by exploiting existing duplicate-adjacency information for resemblance detection and finding the “sweet spot” for the super-feature approach.

We propose Rotated Logging (RoLo), a new logging architecture for parallel disk-based mirrored storage systems for enhanced energy efficiency, which is one of the key concerns in modern data centers. By spreading destaging I/O activities among short idle time slots and proactively reclaiming the stale logging space, RoLo rotates loggers among a logical logging space pool formed collectively from the free storage space available among mirrored disks. We develop three flavors of RoLo, that is, RoLo-P/R/E, to emphasize performance, reliability, and energy efficiency respectively. Without the extra dedicated log disks and the corresponding centralized destaging, RoLo eliminates the additional hardware and energy costs, potential single point of failure and performance bottleneck. Furthermore, RoLo-P/R/E, applied to specific scenes correctly, can prolong the lifecycle of the disks and improve the system's energy efficiency by reducing the disk spin up/down frequency. We propose RoLo-S to further alleviate the performance bottleneck and energy consumption caused by frequent disk head seeks in on-duty logger disks. We have implemented RoLo and RoLo-S on real disk systems. Extensive trace-driven evaluations demonstrate the advantages of the three RoLo schemes over both a RAID10 system with centralized logging architecture and a typical RAID10 system, and the advantages of RoLo-S over RoLo.

In order to efficiently achieve fault tolerance in cloud computing, large-scale data centers generally leverage remote backups to improve system reliability. Due to long-distance and expensive network transmission, the backups incur heavy communication overheads and potential errors. To address this important problem, we propose an efficient remote communication service, called Neptune. Neptune efficiently transmits massive data between long-distance data centers via a cost-effective filtration scheme. The filtration in Neptune is interpreted as eliminating redundancy and compressing similarity of files, which are generally studied independently in the existing work. In order to bridge their gap, Neptune leverages chunk-level deduplication to eliminate duplicate files and uses approximate delta compression to compress similar files. Moreover, in order to reduce the complexity and overheads, Neptune uses a locality-aware hashing to group similar files and proposes shortcut delta chains for fast remote recovery. We have implemented Neptune between two data centers and their distance is more than 1200 km via a 2 Mb/s network link. We examine the Neptune performance using real-world traces of Los Alamos National Laboratory (LANL), EMC, and Campus collection. Compared with state-of-the-art work, experimental results demonstrate the efficiency and efficacy of Neptune.

For years, the increasing popularity of flash memory has been changing storage systems. Flash-based solid-state drives (SSDs) are widely used as a new cache tier on top of hard disk drives (HDDs) to speed up data-intensive applications. However, the endurance problem of flash memory remains a concern and is getting worse with the adoption of MLC and TLC flash. In this article, we propose a novel cache management algorithm for flash-based disk cache named Lazy Adaptive Replacement Cache (LARC). LARC adopts the idea of selective caching to filter out seldom accessed blocks and prevent them from entering cache. This avoids cache pollution and preserves popular blocks in cache for a longer period of time, leading to a higher hit rate. Meanwhile, by avoiding unnecessary cache replacements, LARC reduces the volume of data written to the SSD and yields an SSD-friendly access pattern. In this way, LARC improves the performance and endurance of the SSD at the same time. LARC is self-tuning and incurs little overhead. It has been extensively evaluated by both trace-driven simulations and synthetic benchmarks on a prototype implementation. Our experiments show that LARC outperforms state-of-art algorithms for different kinds of workloads and extends SSD lifetime by up to 15.7 times.

In backup systems, the chunks of each backup are physically scattered after deduplication, which causes a challenging fragmentation problem. We observe that the fragmentation comes into sparse and out-of-order containers. The sparse container decreases restore performance and garbage collection efficiency, while the out-of-order container decreases restore performance if the restore cache is small. In order to reduce the fragmentation, we propose History-Aware Rewriting algorithm (HAR) and Cache-Aware Filter (CAF). HAR exploits historical information in backup systems to accurately identify and reduce sparse containers, and CAF exploits restore cache knowledge to identify the out-of-order containers that hurt restore performance. CAF efficiently complements HAR in datasets where out-of-order containers are dominant. To reduce the metadata overhead of the garbage collection, we further propose a Container-Marker Algorithm (CMA) to identify valid containers instead of valid chunks. Our extensive experimental results from real-world datasets show HAR significantly improves the restore performance by 2.84-175.36 × at a cost of only rewriting 0.5-2.03 percent data.

The challenges of handling the explosive growth in data volume and complexity cause the increasing needs for semantic queries. The semantic queries can be interpreted as the correlation-aware retrieval, while containing approximate results. Existing cloud storage systems mainly fail to offer an adequate capability for the semantic queries. Since the true value or worth of data heavily depends on how efficiently semantic search can be carried out on the data in (near-) real-time, large fractions of data end up with their values being lost or significantly reduced due to the data staleness. To address this problem, we propose a near-real-time and cost-effective semantic queries based methodology, called FAST. The idea behind FAST is to explore and exploit the semantic correlation within and among datasets via correlation-aware hashing and manageable flat-structured addressing to significantly reduce the processing latency, while incurring acceptably small loss of data-search accuracy. The near-real-time property of FAST enables rapid identification of correlated files and the significant narrowing of the scope of data to be processed. FAST supports several types of data analytics, which can be implemented in existing searchable storage systems. We conduct a real-world use case in which children reported missing in an extremely crowded environment (e.g., a highly popular scenic spot on a peak tourist day) are identified in a timely fashion by analyzing 60 million images using FAST. FAST is further improved by using semantic-aware namespace to provide dynamic and adaptive namespace management for ultra-large storage systems. Extensive experimental results demonstrate the efficiency and efficacy of FAST in the performance improvements.

Erasure codes are widely used in modern distributed storage systems to prevent data loss and server failures. Regenerating codes are a class of erasure codes that trades storage efficiency and computation for repair bandwidth reduction. However, their non-unified coding parameters and huge computation overhead prohibit their applications. Hence, we first propose a family of Functional Regenerating Codes (FRCs) with uncoded repair, balancing storage efficiency and repair bandwidth with general parameters. FRCs take advantage of a heuristic repair algorithm, which makes efforts to employ as little repair bandwidth as possible. Second, we optimize encoding by constructing the generator matrix with a bitmatrix, so encoding of FRCs can be executed by fast bitwise XORs. Further, we also optimize repairing with the Scheduled Shift Multiplication (SSM) algorithm, which accelerates the matrix product over the Galois field during repair. Compared to the traditional table-lookup multiplication algorithm, our SSM algorithm gains 1.2~2X speed-up.

In cloud services, users may frequently be required to reveal their personal private information which could be stored in the cloud to used by different parts for different purposes. However, in a cloud-wide storage network, the servers are easily under strong attacks and also commonly experience software/hardware faults. As such, the private information could be under great risk in such an untrusted environment. Given that the presented personal sensitive information is usually out of user's controlin most cloud-based services, ensuring data security and privacy protection with respect to untrusted storage network has become a formidable challenge in research. To address these challenges, in this paper we propose a self-destruction system, named CloudSky, which is able to enforce the security of user privacy over the untrusted cloud in a controllable way. CloudSky exploits a key control mechanism based on the attribute-based encryption (ABE) and takes advantage of active storage networks to allow the user to control the subjective life-cycle and the access control polices of the private data whose integrity is ensured by using HMAC to cope with untrusted environments. %and thereby adapting it to the cloud in terms of both performance and security requirements. The feasibility of the system in terms of its performance and scalability is demonstrated by experiments on a real large-scale storage network.

Modern solid state disk (SSD) has a buffer (SDRAM), which is used to store commonly used data and map in the near future. How to efficient management of this buffer is an important things of improving performance of SSD. Flash read and write speed have asymmetric characteristic. SSD buffer management algorithms must consider this characteristic of flash. Current page mapping SSD buffer management algorithms mainly use the Clean-First LRU (CFLRU) algorithm to first replace the clean buffer pages regardless of whether these pages will soon be used in the near future. At the same time, LRU buffer management algorithm of SSD does not consider file scanning. In order to solve these problems, we proposes a new SSD internal buffer management algorithm, called Two Queue Weight-Clock (2QW-Clock). This algorithm combines the advantages of 2Q and gives different weights to read page and write page to reflect the asymmetry of flash read and write speed. Therefore, it can get high write page hit ratios while maintaining high total page hit ratios. With the high write ratios, 2QW-Clock reduces the numbers of SSD write and erase operations. So it can greatly extend the life of the SSD. Conducting simulations with a variety of traces and a wide range of buffer sizes, we show that 2QW-Clock write hit ratios are significantly higher than CFLRU, LRU and 2Q in most cases while total hit ratios are almost as the 2Q. Simulation result shows that the numbers of 2QW-Clock write and erase counts reduced by up to 30% less than that of 2Q and CFLRU.

Due to the high complexity in software hierarchy and the shared queue & lock mechanism for synchronized access, existing I/O stack for remote target access in FCoE-based SAN storage becomes a performance bottleneck, thus leading to a high I/O overhead and limited I/O scalability in multi-core servers. For scalable performance, existing works focus on improving the efficiency of lock algorithm or reducing the number of synchronization points to decrease the synchronization overhead. However, the synchronization problem still exists and leads to a limited I/O scalability. In this paper, we propose Fast FCoE, a protocol stack framework for remote storage access in FCoE based SAN storage. Fast FCoE uses private per-CPU structures and disables the kernel preemption to process I/Os. This method avoids the synchronization overhead. For further I/O efficiency, Fast FCoE directly maps the requests from the block-layer to the FCoE frames. A salient feature of Fast FCoE is using the standard interfaces, thus supporting all upper softwares (such as existing file systems and applications) and offering flexible use in existing infrastructure (e.g., Adaptors, switches, storage devices). Our results demonstrate that Fast FCoE achieves efficient and scalable I/O throughput, obtaining 1107.3K/831.3K IOPS (5.43/4.88 times as much as Open-FCoE stack) for read/write requests.

Rapid disaster relief is important to save human lives and reduce property loss. With the wide use of smartphones and their ubiquitous easy access to the Internet, sharing and uploading images to the cloud via smartphones offer a nontrivial opportunity to provide information of disaster zones. However, due to limited available bandwidth and energy, smartphone-based crowdsourcing fails to support the real-time data analytics. The key to efficiently and timely share and analyze the images is to determine the value/worth of the images based on their significance and redundancy, and only upload those valuable and unique images. In this paper, we propose a near-realtime and cost-efficient scheme, called SmartEye, in the cloud-assisted disaster environment. The idea behind SmartEye is to implement QoS-aware in-network deduplication over DiffServ in the software-defined networks (SDN). Due to the ease of use, simplicity and scalability, DiffServ supports the in-network deduplication to meet the needs of differentiated QoS. SmartEye aggregates the flows with similar features via a semantic hashing, and provides communication services for the aggregated, not a single, flow. To achieve these goals, we leverage two main optimization schemes, including semantic hashing and space-efficient filters. Efficient image sharing is helpful to disaster detection and scene recognition. To demonstrate the feasibility of SmartEye, we conduct two real-world case studies in which the loss in Typhoon Haiyan (2013) and Hurricane Sandy (2012) can be identified in a timely fashion by analyzing massive data consisting of more than 22 million images using our SmartEye system. Extensive experimental results illustrate that SmartEye is efficient and effective to achieve real-time analytics in disasters.

Data deduplication, a space-efficient and bandwidth-saving technology, plays an important role in bandwidth-efficient data transmission in various data-intensive network and cloud applications. Rabin-based and MAXP-based Content-Defined Chunking (CDC) algorithms, while robust in finding suitable cut-points for chunk-level redundancy elimination, face the key challenges of (1) low chunking throughput that renders the chunking stage the deduplication performance bottleneck and (2) large chunk-size variance that decreases deduplication efficiency. To address these challenges, this paper proposes a new CDC algorithm called the Asymmetric Extremum (AE) algorithm. The main idea behind AE is based on the observation that the extreme value in an asymmetric local range is not likely to be replaced by a new extreme value in dealing with the boundaries-shift problem, which motivates AE's use of asymmetric (rather than symmetric as in MAXP) local range to identify cut-points and simultaneously achieve high chunking throughput and low chunk-size variance. As a result, AE simultaneously addresses the problems of low chunking throughput in MAXP and Rabin and high chunk-size variance in Rabin. The experimental results based on four real-world datasets show that AE improves the throughput performance of the state-of-the-art CDC algorithms by 3x while attaining comparable or higher deduplication efficiency.

With the rapid growth of the amount of information, cloud computing servers need to process and analyze large amounts of high-dimensional and unstructured data timely and accurately, which usually requires many query operations. Due to simplicity and ease of use, cuckoo hashing schemes have been widely used in real-world cloud-related applications. However, due to the potential hash collisions, the cuckoo hashing suffers from endless loops and high insertion latency, even high risks of re-construction of entire hash table. In order to address this problem, we propose a cost-efficient cuckoo hashing scheme, called MinCounter. The idea behind MinCounter is to alleviate the occurrence of endless loops in the data insertion. MinCounter selects the “cold” (infrequently accessed) buckets to handle hash collisions rather than random buckets. MinCounter has the salient features of offering efficient insertion and query services and obtaining performance improvements in cloud servers, as well as enhancing the experiences for cloud users. We have implemented MinCounter in a large-scale cloud testbed and examined the performance by using two real-world traces. Extensive experimental results demonstrate the efficacy and efficiency of MinCounter.

Nowadays, many customers and enterprises backup their data to cloud storage that performs deduplication to save storage space and network bandwidth. Hence, how to perform secure deduplication becomes a critical challenge for cloud storage. According to our analysis, the state-of-the-art secure deduplication methods are not suitable for cross-user finegrained data deduplication. They either suffer brute-force attacks that can recover files falling into a known set, or incur large computation (time) overheads. Moreover, existing approaches of convergent key management incur large space overheads because of the huge number of chunks shared among users. Our observation that cross-user redundant data are mainly from the duplicate files, motivates us to propose an efficient secure deduplication scheme SecDep. SecDep employs User-Aware Convergent Encryption (UACE) and Multi-Level Key management (MLK) approaches. (1) UACE combines cross-user file-level and inside-user chunk-level deduplication, and exploits different secure policies among and inside users to minimize the computation overheads. Specifically, both of file-level and chunk-level deduplication use variants of Convergent Encryption (CE) to resist brute-force attacks. The major difference is that the file-level CE keys are generated by using a server-aided method to ensure security of cross-user deduplication, while the chunk-level keys are generated by using a user-aided method with lower computation overheads. (2) To reduce key space overheads, MLK uses file-level key to encrypt chunk-level keys so that the key space will not increase with the number of sharing users. Furthermore, MLK splits the file-level keys into share-level keys and distributes them to multiple key servers to ensure security and reliability of file-level keys. Our security analysis demonstrates that SecDep ensures data confidentiality and key security. Our experiment results based on several large real-world datasets show that SecDep is more time-efficient and key-space-efficient than the state-of-the-art secure deduplication approaches.

Base station (BS) location privacy has been widely studied and researched in different applications like field monitoring, agriculture, industry and military etc. The purpose is to hide the location of BS from outside/inside attacker in any shape. Hundreds of thousands of sensor nodes are deployed in some area, bring lot of new challenges regarding routing, forwarding, scaling and security. Different approaches are there to provide location privacy in wireless sensor networks (WSN). The work done by researchers is to provide some form of BS location privacy either different from others or boosting them more. In our scheme, we proposed a new algorithm MimiBS “Mimicking Base-Station”. All the deployed aggregator nodes (ANs) in the field/area will be looking like BS, even if the attacker knows about any AN, he will be deceived between the real BS and AN. We approached different schemes for our proposed algorithm mentioning routing without fake packets, with fake packets, without energy consideration and with energy consideration. These different parameters show an improvement over previous work regarding the same problem.

Software delays have become the bottleneck to the overall storage system. Particularly, in iSCSI-based storage system, more disks in the target-end system are configured to make up a target node, but requests initiated by the initiator-end system will go through the traditional single I/O path before they are re-requested by the target-end system, which cannot exploit the serviceability of the target multi-disks adequately. This paper introduces the metadata-request queue to handle metadata I/O requests and scheduling differs from the request queue that exists in the initiator-end system of iSCSI-based storage system, given the different characteristics of metadata and data I/O requests. Instead of dispatching all requests exclusively to the single queue and unified scheduling, a new metadata-request queue is introduced for every logical device and is implemented with immediate response mode to eliminate waiting delays, considering the characteristics of small size, high frequency and scattered accesses that require for real-time response. Data requests in the data-request queue are delivered to the scheduling queue in an I/O waiting state, which is known as plugged-unplugged mechanism. Evaluation of our system using Postmarks and IOMeter shows 9%-19% I/O performance improvement.

Owing to their relatively high random-I/O performance and energy-efficiency, SSDs have been widely deployed in data-intensive computing environments as caching and/or terminal storage devices. The buffer-cache of an SSD plays an essential role in bridging the speed gap between the flash components and the host interface. While the existing SSD buffer management schemes are designed to improve SSD performance, they are often rendered ineffective by the widespread bursts of I/O traffic in data-intensive workloads. To address this problem, this paper proposes a Proactive and Adaptive SSD buffer Scheme (PASS) to judiciously and actively flush dirty buffered data in anticipation of traffic bursts by exploiting the light-traffic intervals, as well as the chip-level and channel-level parallelism inside SSD. The results of our extensive trace-driven simulation show that PASS significantly and consistently outperforms the state-of-the-art buffer management schemes in both response time and endurance measures.

NAND flash memory has attracted wide attention in both academia and industry in recent years. Its high random access performance fills the gap between DRAM and hard disks. While MLC is endorsed for higher density and lower cost per bit, it suffers from poor performance and endurance. Dual-mode flash combines SLC and MLC in a single device and thus provides the opportunity to trade density for performance. In this paper, we propose the Scalable Flash Storage(SFS) abstraction layer to facilitate cache management on dual-mode flash. SFS exposes a virtualized address space to hide the variable density of the medium. A differentiated write interface is introduced, which allows the cache manager to explicitly send write requests to SLC for high performance. SFS dynamically scales the proportions of SLC and MLC to balance between cache capacity and performance. SFS provides partially persistent storage service. It allows the cache manager to manage the data persistence on flash so that critical data can be retained persistently. Non-persistent data are discarded during garbage collection to mitigate write amplification. Based on the SFS, a Dual-mode Flash Cache(DMFC) architecture is designed to utilize the configurable density and performance. Experimental results show that DMFC can significantly improve overall performance for various workloads.

Flash-based Solid State Drive (SSD) has limitations in terms of cost and lifetime. It is used as a second-level cache between main memory and traditional HDD-based storage widely. Adopting traditional cache algorithms, which are designed primarily depending on temporal locality and popular blocks, to SSD-based second-level disk cache can cause unnecessary cache replacements, which not only degrade the cache performance but also shorten the lifetime of SSD. To overcome this problem, this paper proposes a performance-effective Regional Popularity-Aware Cache replacement algorithm (RPAC). Instead of a single block, the popularity of a region which is constituted by many adjacent disk blocks is recorded and used to determine replacing a block or not. In this way, the spatial locality of disk access is completely leveraged and sequential I/O blocks are gathered in SSD cache. Furthermore, it reduces the number of unnecessary cache replacement and erasure operation on SSD, prolonging its lifetime. We have implemented RPAC in real system and evaluated it by many workloads. Compared to traditional cache algorithms, it improve I/O throughput by up to 53% and reduce cache replacements of SSD up to 98.5%.

Erasure codes are widely used in distributed storage systems to prevent data loss. Traditional erasure codes suffer from a typical repair-bandwidth problem in which the amount of data required to reconstruct the lost data, referred to as the repair bandwidth, is often far more than the theoretical minimum. While many novel erasure codes have been proposed in recent years to reduce the repair bandwidth, these codes either require extra storage capacity and computation overhead or are only applicable to some special cases. To address the weaknesses of the existing solutions to the repair-bandwidth problem, we propose Z Codes, a general family of codes capable of achieving the theoretical lower bound of repair bandwidth for a single data node failure. To the best of our knowledge, the Z codes are the first general systematic erasure codes that achieve optimal repair bandwidth under the minimum storage. Our in-memory performance evaluations of a 1-GB file indicate that Z codes have encoding and repairing speeds that are approximately equal to those of the Reed-Solomon (RS) codes, and their speed on the order of GB/s practically removes computation as a performance bottleneck.

Data deduplication has gained increasing attention and popularity as a space-efficient approach in backup storage systems. One of the main challenges for centralized data deduplication is the scalability of fingerprint-index search. In this paper, we propose SiLo, a near-exact and scalable deduplication system that effectively and complementarily exploits similarity and locality of data streams to achieve high duplicate elimination, throughput, and well balanced load at extremely low RAM overhead. The main idea behind SiLo is to expose and exploit more similarity by grouping strongly correlated small files into a segment and segmenting large files, and to leverage the locality in the data stream by grouping contiguous segments into blocks to capture similar and duplicate data missed by the probabilistic similarity detection. SiLo also employs a locality based stateless routing algorithm to parallelize and distribute data blocks to multiple backup nodes. By judiciously enhancing similarity through the exploitation of locality and vice versa, SiLo is able to significantly reduce RAM usage for index-lookup, achieve the near-exact efficiency of duplicate elimination, maintain a high deduplication throughput, and obtain load balance among backup nodes.

It has long been recognized that the Credit scheduler selectively favors CPU-bound applications whereas for I/O-latency sensitive workloads, such as those related to stream-based audio/video services, it only exhibits tolerable, or even worse, unacceptable performance. The reasons behind this phenomenon are the poor understanding (to some degree) of the virtual machine scheduling as well as the network I/O virtualizations. In order to address these problems and make the system more responsive to the I/O-latency sensitive applications, in this paper, we present XCollOpts which performs a collection of novel optimizations to improve the Credit scheduler and the underlying I/O virtualizations in multicore environments, each from two perspectives. To optimize the schedule, in XCollOpts, we first pinpoint the Imbalanced Multi-Boosting problem among the cores thereby minimizing the system response time by load balancing the BOOST VCPUs. Then, we describe the Premature Preemption problem and address it by monitoring the received network packets in the driver domain and deliberately preventing it from being prematurely preempted during the packet delivery. However, these optimizations on the scheduling strategies cannot be fully exploited if the performance issues of the underlying supportive communication mechanisms are not considered. To this end, we make two further optimizations for the network I/O virtualizations, namely, Multi-Tasklet Pairs and Optimized Small Data Packet. Our empirical studies show that with XCollOpts, we can significantly improve the performance of the latency-sensitive applications at a cost of relatively small system overhead.

Cloud computing applications face the challenges of dealing with a huge volume of data that needs the support of accurate and fast approximate queries to enhance system scalability and improve quality of service. Locality-sensitive hashing (LSH) can support the approximate queries that unfortunately suffer from imbalanced load and space inefficiency among distributed data servers, which severely limits the query accuracy and incurs long query latency between users and cloud servers. In this paper, we propose a novel scheme, called NEST, which offers easy-to-use and cost-effective approximate queries for cloud computing. The novelty of NEST is to leverage cuckoo-driven locality-sensitive hashing to find similar items that are further placed closely through cuckoo-driven method to obtain load-balancing buckets in hash tables. NEST hence carries out flat and manageable addressing in adjacent buckets, and obtains constant-scale query complexity even in the worst case. The benefits of NEST include the increments of space utilization and fast query response. Moreover, due to the salient property of flat addressing in NEST, we implement NEST design in a real hybrid storage system, which consists of DRAM, SSD, and hard disk. The flat addressing allows efficient operations in SSD to improve system performance. We argue that a proper “division of labor” among DRAM, SSD, and hard disk in the hybrid and heterogeneous storage hierarchy is desperately needed to strike an optimal balance to remove the indexing bottleneck. Theoretical analysis and extensive experiments (on LANL and Microsoft metadata) in a large-scale cloud testbed demonstrate the salient properties of NEST to meet the needs of approximate query service in cloud computing environments. We have offered open-source codes of NEST for public use.

Data reduction has become increasingly important in storage systems due to the explosive growth of digital data in the world that has ushered in the big data era. In this paper, we present DARE, a Deduplication-Aware Resemblance detection and Elimination scheme for compressing backup datasets that effectively combines data deduplication and delta compression to achieve high data reduction efficiency at low overhead. The main idea behind DARE is to employ a scheme, call Duplicate-Adjacency based Resemblance Detection (DupAdj), by considering any two data chunks to be similar (i.e., candidates for delta compression) if their respective adjacent data chunks are found to be duplicate in a deduplication system, and then further enhance the resemblance detection efficiency by an improved super-feature approach. Our experimental results based on real-world and synthetic backup datasets show that DARE achieves an additional data reduction by a factor of more than 2 (2X) on top of deduplication with very low overhead while nearly doubling the data restore performance of deduplication-only systems by supplementing delta compression to deduplication.

FlexRay is an automotive network communication protocol. It provides support to transmit time-sensitive messages in automobiles. FlexRay transmits periodic messages in a static segment and a periodic messages in a dynamic segment. To improve transmission reliability, FlexRay offers hybrid data management schemes for both static and dynamic segments. However, existing approaches only schedule static segment and dynamic segment separately, leading to poor bandwidth utilization and transmission delay. Moreover, due to the bandwidth limitation, existing best-effort retransmission for all segments fails to achieve high reliability. To address these two concerns, we propose a novel and efficient scheduling scheme, called Coefficient. The idea behind Coefficient is to cooperatively schedule the static and dynamic segments, while judiciously stealing the selective slacks for reliable transmission based on practical fault models. Coefficient schedules both static and dynamic segments in the dual-channel manner based on practical fault models. Extensive experiments based on real-world case studies demonstrate that Coefficient meets the needs of both real-time transmission and reliability requirements, and delivers significant performance improvements.

In order to efficiently achieve fault tolerance in cloud computing, large-scale data centers generally leverage remote backups to improve system reliability. Due to longdistance and expensive network transmission, the backups incur heavy communication overheads and potential errors. To address this important problem, we propose an efficient remote communication service, called Neptune. Neptune efficiently transmits massive data between long-distance data centers via a cost-effective filtration scheme. The filtration in Neptune is interpreted as eliminating redundancy and compressing similarity of files, which are generally studied independently in existing work. In order to bridge the gap between them, Neptune leverages chunk-level deduplication to eliminates duplicate files, and approximate delta compression to compresses similar files. Moreover, in order to reduce the complexity and overheads, Neptune uses a locality-aware hashing to group similar files and proposes shortcut delta chains for fast remote recovery. We have really implemented Neptune. We examine the Neptune performance by using real-world traces of LANL, HP, MSN and Google. Compared with state-of-the-art work, experimental results demonstrate the efficiency and efficacy of Neptune.

With the wide use of video-editing software, more and more similar videos are uploaded to the video-sharing platforms in cloud storage. The large number of near-duplicate videos would not only make the users not satisfactory, but also consume more resources with service providers. Since the near-duplicate videos have redundancy at the content level, the traditional de-duplication technology is not suitable to address the redundancies. In order to decrease the redundancy, we proposed a novel high performance provenance-based video-sharing system, called Provis. Provis records the provenance of the videos and compresses the near-duplicate videos via storing their provenance which replaces storing their video files to improve the space efficiency. When users attempt to upload the edited videos, Provis can rebuild the videos in the cloud via collecting and uploading their provenance to reduce the network overhead of uploading videos. We implemented Provis and compared its performance with other existing video compression techniques. Our evaluation shows that Provis achieves significant space savings.

Real-time file sharing is important to improve the quality of cloud services. Current cloud systems however fail to efficiently offer cost-effective data analytics due to slow response and energy inefficiency. In order to support real-time processing and improve energy efficiency in the cloud, this paper proposes a novel cost-effective data analytics scheme, called Needle. The idea behind Needle is to use flat-addressing and content-aware naming approaches to deliver high performance of network transmission. Needle leverages a suitable division-of-labor model between core and edge network nodes to efficiently support content-aware queries in the cloud data center networks. Experimental results demonstrate that Needle is able to support cloud sharing service in a real-time and efficient manner.

With the rapid growth of data, query performance is an important concern in cloud storage applications. To reduce the query response time, cuckoo hashing via d hash functions has been adopted to achieve O(1) query efficiency. However, in practice the cuckoo hashing consumes a large amount of system resources, since an item insertion may suffer from frequent “kick-out” operations or even endless loops. In order to address this problem, we propose an efficient loop-oblivious scheme, called Necklace, in the cloud. The idea behind Necklace is to minimize data migration among servers and alleviate the collisions in the insertion operation. We identify the shortest path via indexing the auxiliary table, without retrieving the actual storage contents or carrying out the tentative “kick-out” operations. We have implemented Necklace in a real cloud system and examined the performance by using a real-world trace. Extensive experimental results demonstrate the efficiency and efficacy of Necklace.

This paper is motivated by our three key observations: (1) there exists a degradation of performance as the interleaved accesses of heterogeneous streams, (2) for the slow stream, sequential accesses suffer huge misses in the prefetching cache, (3) in concurrence paradigm, providing fairness and QoS to concurrent streams is very important which always ignored by the traditional prefetching algorithms. Therefore, we present Fema, a caching management algorithm that enforces the fairness and efficiency for concurrent heterogeneous streams. Fema focuses on three key designs: (1) An adaptive framework (Fema Ada) for prefetching. In the Fema Ada, we propose a rate-aware adjustment of prefetching degree and analysis the optimal partition size. (2) A novel replacement scheme (Fema Rep) in which the accessed data will be firstly evicted to improve the performance. (3) A round robin allocation scheme (Fema Rou) to achieve fairness while as least performance degradation as possible. Results show that Fema is able to achieve averages 81.4% performance improvement over the LRU algorithm, 53.5% over the default Linux Kernel prefetching (LKP) algorithm and 19.0% over the recently proposed practical AMP (adaptive multi-stream prefetching) algorithm. Fema achieves average 74.2% fairness improvement (metric in fair speedup) over the LKP algorithm and 56.5% over the AMP algorithm.

With the explosive growth in data volume and complexity and the increasing need for highly efficient searchable data analytics, existing cloud storage systems have largely failed to offer an adequate capability for real-time data analytics. Since the true value or worth of data heavily depends on how efficiently data analytics can be carried out on the data in (near-) real-time, large fractions of data end up with their values being lost or significantly reduced due to the data staleness. To address this problem, we propose a near-real-time and cost-effective searchable data analytics methodology, called FAST. The idea behind FAST is to explore and exploit the semantic correlation within and among datasets via correlation-aware hashing and manageable flat-structured addressing to significantly reduce the processing latency, while incurring acceptably small loss of data-search accuracy. The near-real-time property of FAST enables rapid identification of correlated files and the significant narrowing of the scope of data to be processed. FAST supports several types of data analytics, which can be implemented in existing searchable storage systems. We conduct a real-world use case in which children reported missing in an extremely crowded environment (e.g., A highly popular scenic spot on a peak tourist day) are identified in a timely fashion by analyzing 60 million images using FAST. Extensive experimental results demonstrate the efficiency and efficacy of FAST in the performance improvements and energy savings.

Real-time aggregate queries can help obtain interested summary of traffic information on the road. However, due to unreliable connection and limited duration in Vehicular Ad hoc Networks (VANETs), it is difficult to carry out the online computation over all received traffic messages. In order to improve query accuracy and provide quick query response, we propose a novel scheme for real-time aggregate queries, called Road Cube, which essentially makes use of precomputation on interested traffic messages. We utilize Information Retrieval (IR) technique to identify interested information that potentially shows semantic correlation and can be indexed in future with high probability. The Road Cube improves upon conventional data cube by exploiting semantic correlation of multi-dimensional attributes existing in received traffic information so as to obtain partial materialization. The partial materialization usually satisfies real-time and space requirements in VANETs. Extensive performance evaluation based on real-world map and traffic models shows that the Road Cube obtains significant performance improvements, compared with the conventional approaches.

Compared with either block or page-mapping Flash Translation Layer (FTL), hybrid-mapping FTL for flash Solid State Disks (SSDs), such as Fully Associative Section Translation (FAST), has relatively high space efficiency because of its smaller mapping table than the latter and higher flexibility than the former. As a result, hybrid-mapping FTL has become the most commonly used scheme in SSDs. But the hybrid-mapping FTL incurs a large number of costly full-merge operations. Thus, a critical challenge to hybrid-mapping FTL is how to reduce the cost of full-merge operations and improve partial merge operations and switch operations. In this article, we propose a novel FTL scheme, called Virtual Block-based Parallel FAST (VBP-FAST), that divides flash area into Virtual Blocks (VBlocks) and Physical Blocks (PBlocks) where VBlocks are used to fully exploit channel-level, die-level, and plane-level parallelism of flash. Leveraging these three levels of parallelism, the cost of full merge in VBP-FAST is significantly reduced from that of FAST. In the meantime, VBP-FAST uses PBlocks to retain the advantages of partial merge and switch operations. Our extensive trace-driven simulation results show that VBP-FAST speeds up FAST by a factor of 5.3--8.4 for random workloads and of 1.7 for sequential workloads with channel-level, die-level, and plane-level parallelism of 8, 2, and 2 (i.e., eight channels, two dies, and two planes).

The cloud is emerging for scalable and efficient cloud services. To meet the needs of handling massive data and decreasing data migration, the computation infrastructure requires efficient data placement and proper management for cached data. In this paper, we propose an efficient and cost-effective multilevel caching scheme, called MERCURY, as computation infrastructure of the cloud. The idea behind MERCURY is to explore and exploit data similarity and support efficient data placement. To accurately and efficiently capture the data similarity, we leverage a low-complexity locality-sensitive hashing (LSH). In our design, in addition to the problem of space inefficiency, we identify that a conventional LSH scheme also suffers from the problem of homogeneous data placement. To address these two problems, we design a novel multicore-enabled locality-sensitive hashing (MC-LSH) that accurately captures the differentiated similarity across data. The similarity-aware MERCURY, hence, partitions data into the L1 cache, L2 cache, and main memory based on their distinct localities, which help optimize cache utilization and minimize the pollution in the last-level cache. Besides extensive evaluation through simulations, we also implemented MERCURY in a system. Experimental results based on real-world applications and data sets demonstrate the efficiency and efficacy of our proposed schemes.

The explosive growth in data volume and complexity imposes great challenges for file systems. To address these challenges, an innovative namespace management scheme is in desperate need to provide both the ease and efficiency of data access. In almost all today's file systems, the namespace management is based on hierarchical directory trees. This tree-based namespace scheme is prone to severe performance bottlenecks and often fails to provide real-time response to complex data lookups. This paper proposes a Semantic-Aware Namespace scheme, called SANE, which provides dynamic and adaptive namespace management for ultra-large storage systems with billions of files. SANE introduces a new naming methodology based on the notion of semantic-aware per-file namespace, which exploits semantic correlations among files, to dynamically aggregate correlated files into small, flat but readily manageable groups to achieve fast and accurate lookups. SANE is implemented as a middleware in conventional file systems and works orthogonally with hierarchical directory trees. The semantic correlations and file groups identified in SANE can also be used to facilitate file prefetching and data de-duplication, among other system-level optimizations. Extensive trace-driven experiments on our prototype implementation validate the efficacy and efficiency of SANE.

FlexRay is a new industry standard for next-generation communication in automotives. Though there are a few recent researches on performance analysis of FlexRay, two important aspects of the FlexRay design have been overlooked. The first is a holistic integrated scheduling scheme that can handle both static and dynamic segments in a FlexRay network. The second is cost-effective and scalable performance. To address these aspects, we propose a novel holistic scheduling scheme, called HOSA, which can provide scalable performance by using flexible and ease-of-use dual channel communication in FlexRay. HOSA is built upon a novel slot pilfering technique to schedule and optimize the available slots in both static and dynamic segments. Moreover, to achieve efficient implementation, we propose approximate computation, which can efficiently support cost-effective and holistic scheduling by judiciously obtaining the tradeoff between computation complexity and available pilfered slots. HOSA hence offers the salient feature of improving bandwidth utilization. Moreover, to deliver high performance and bridge the gap between real-time communications in networks and storage management in end devices, we implement a deduplication-aware scheme in a real SSD prototype that can reduce space overhead and extend SSD lifespan (by significantly reducing duplicate write operations). The deduplication scheme meets the needs of message updates in FlexRay. Extensive experiments based on synthetic test cases and real-world case studies demonstrate the efficiency and efficacy of HOSA.

Cloud computing has raised new challenges for the membership representation scheme of storage systems that manage very large data sets. This paper proposes DBA, a dynamic Bloom filter array aimed at representing membership for variable large data sets in storage systems in a scalable way. DBA consists of dynamically created groups of space-efficient Bloom filters (BFs) to accommodate changes in set sizes. Within a group, BFs are homogeneous and the data layout is optimized at the bit level to enable parallel access and thus achieve high query performance. DBA can effectively control its query accuracy by partially adjusting the error rate of the constructing BFs, where each BF only represents an independent subset to help locate elements and confirm membership. Further, DBA supports element deletion by introducing a lazy update policy. We prototype and evaluate our DBA scheme as a scalable fast index in the MAD2 deduplication storage system. Experimental results reveal that DBA (with 64 BFs per group) shows significantly higher query performance than the state-of-the-art approach while scaling up to 160 BFs. DBA is also shown to excel in scalability, query accuracy, and space efficiency by theoretical analysis and experimental evaluation.

Existing cloud storage systems have largely failed to offer an adequate capability for real-time data analytics. Since the true value of data heavily depends on how efficiently data analytics can be carried out on the data in (near-) real-time, large fractions of data unfortunately end up with their values being lost or significantly reduced due to the staleness of data. To address this problem, we propose a near real-time and cost-effective data analytics methodology, called FAST, in the cloud. FAST explores and exploits the semantic correlation property within and among datasets via correlation-aware hashing and manageable flat-structured addressing to significantly reduce the processing latency, while incurring acceptably small loss of accuracy. FAST is demonstrated to be a useful tool in supporting near real-time processing of real-world cloud applications.

The increasing popularity of flash memory has changed storage systems. Flash-based solid state drive(SSD) is now widely deployed as cache for magnetic hard disk drives(HDD) to speed up data intensive applications. However, existing cache algorithms focus exclusively on performance improvements and ignore the write endurance of SSD. In this paper, we proposed a novel cache management algorithm for flash-based disk cache, named Lazy Adaptive Replacement Cache(LARC). LARC can filter out seldom accessed blocks and prevent them from entering cache. This avoids cache pollution and keeps popular blocks in cache for a longer period of time, leading to higher hit rate. Meanwhile, LARC reduces the amount of cache replacements thus incurs less write traffics to SSD, especially for read dominant workloads. In this way, LARC improves performance and extends SSD lifetime at the same time. LARC is self-tuning and low overhead. It has been extensively evaluated by both trace-driven simulations and a prototype implementation in flashcache. Our experiments show that LARC outperforms state-of-art algorithms and reduces write traffics to SSD by up to 94.5% for read dominant workloads, 11.2-40.8% for write dominant workloads.

In order to efficiently handle the rapid growth of data and reduce the overhead of network transmission, we propose an in-network deduplication for storage-aware Software Defined Network (SDN), called SMIND. Unlike conventional source or destination deduplication schemes, SMIND implements in-network deduplication via SDN. Moreover, to address the performance bottleneck of accessing and indexing SDN controller, we implement an SDN-enabled Flash Translation Layer (FTL) in a real prototype of Solid State Disk (SSD). Experimental results demonstrate the efficiency and efficacy of SMIND.

Our experimental study and analysis reveal that the bottlenecks of existing hardware transactional memory systems are largely rooted in the extra data movements in version management and in the inefficient scheduling of conflicting transactions in conflict management, particularly in the presence of high-contention and coarse-grained applications. In order to address this problem, we propose an integrated Pseudo-Associativity and Relaxed-Order approach to hardware Transactional Memory, called PARO-TM. It exploits the extra pseudo-associative space in the data cache to hold the new value of each transactional modification, and maintains the mappings between the old and new versions via an implicit pseudo-associative hash algorithm (i.e., by inverting the specific bit of the SET index). PARO-TM can branch out the speculative version from the old version upon each transactional modification on demand without a dedicated hardware component to hold the uncommitted data. This means that it is able to automatically access the proper version upon the transaction's commit or abort. Moreover, PARO-TM augments multi-version support in a chained directory to schedule conflicting transactions in a relaxed-order manner to further reduce their overheads. We compare PARO-TM with the state-of-the-art LogTM-SE, TCC, DynTM, and SUV-TM systems and find that PARO-TM consistently outperforms these four representative HTMs. This performance advantage of PARO-TM is far more pronounced under the high-contention and coarse-grained applications in the STAMP benchmark suite, for which PARO-TM is motivated and designed.

Given the multilevel internal SSD parallelism at the different four levels: channel-level, chip-level, die-level, and plane-level, how to exploit these levels of parallelism will directly and significantly impact the performance and endurance of SSDs, which is in turn primarily determined by three internal factors, namely, advanced commands, allocation schemes, and the priority order of exploiting the four levels of parallelism. In this paper, we analyze these internal factors to characterize their impacts, interplay, and parallelism for the purpose of performance and endurance enhancement of SSDs through an in-depth experimental study. We come to the following key conclusions: 1) Different advanced commands provided by Flash manufacturers exploit different levels of parallelism inside SSDs, where they can either improve or degrade the SSD performance and endurance depending on how they are used; 2) Different physical-page allocation schemes employ different advanced commands and exploit different levels of parallelism inside SSDs, giving rise to different performance and endurance impacts; 3) The priority order of using the four levels of parallelism has the most significant performance and endurance impact among the three internal factors. The optimal priority order of using the four levels of parallelism in SSDs is found to be: 1) the channel-level parallelism; 2) the die-level parallelism; 3) the plane-level parallelism; and 4) the chip-level parallelism.

Provenance is the metadata that describes the history of objects. Provenance provides new functionality in a variety of areas, including experimental documentation, debugging, search, and security. As a result, a number of groups have built systems to capture provenance. Most of these systems focus on provenance collection, a few systems focus on building applications that use the provenance, but all of these systems ignore an important aspect: efficient long-term storage of provenance. In this article, we first analyze the provenance collected from multiple workloads and characterize the properties of provenance with respect to long-term storage. We then propose a hybrid scheme that takes advantage of the graph structure of provenance data and the inherent duplication in provenance data. Our evaluation indicates that our hybrid scheme, a combination of Web graph compression (adapted for provenance) and dictionary encoding, provides the best trade-off in terms of compression ratio, compression time, and query performance when compared to other compression schemes.

Efficient provenance storage is an essential step towards the adoption of provenance. In this paper, we analyze the provenance collected from multiple workloads with a view towards efficient storage. Based on our analysis, we characterize the properties of provenance with respect to long term storage. We then propose a hybrid scheme that takes advantage of the graph structure of provenance data and the inherent duplication in provenance data. Our evaluation indicates that our hybrid scheme, a combination of web graph compression (adapted for provenance) and dictionary encoding, provides the best tradeoff in terms of compression ratio, compression time and query performance when compared to other compression schemes.

Both physical disk failure and logical errors such as software error, user abuse and virus attacks may cause data lose. The risk of logical errors is far greater than physical disk failure. Moreover, existing RAID solution cannot satisfy the reliability requirement in face of the logical errors in data centers. It is therefore becoming increasingly important for RAID-based storage systems to be able to recover data to any point-in-time when logical errors occur. We proposed a novel storage array architecture, Herp Rap, which is able to recover data from both physical disk failure and logical errors. We have implemented a prototype of Herp Rap and carried out extensive performance measurements using DBT-2 and file system benchmarks. Our experiments demonstrated that the proposed Herp Rap is able to track or recover data to any point-in-time quickly by tracing back the history of block logs. Moreover, Herp Rap outperforms existing HDD-based or SSD-based RAID5 with copy-on-write (COW) snapshot in terms of performance, energy efficiency, failure recovery ability and reliability.

In order to maintain the transactional semantics, Transactional Memory (TM) must guarantee isolated read and write operations in each transaction, meaning that it must spend a non-negligible and potentially significant amount of time on keeping track of the transactional modifications in its undo or redo log and switching to the proper version at the end of each transaction. Existing TMs failed to minimize the overheads incurred by these operations that are poised to impose more significant TM overheads in current and future many-core CMPs. A direct consequence of this is that extra and different data movements are needed to manage these modifications depending on commit or abort. To address this problem, we propose a novel Single-Update Version-management (SUV) scheme to redirect each transactional store operation to another memory address, track the mapping information between the original and redirected addresses, and switch to the proper version of data upon the transaction's commit or abort. There is only one data update (movement) in our SUV regardless of commit or abort, thus significantly reducing the TM overheads while allowing it to exploit more thread parallelism. We use SUV to replace version-management schemes in some existing hardware TMs to assess SUV's performance advantages. Our extensive execution-driven experiments show that SUV-TM consistently outperforms the state-of-the-art HTM schemes Log TM-SE, FasTM and DynTM under the STAMP benchmark suite. Moreover, we use CACTI to estimate the hardware overheads of SUV and find it is feasible in hardware implementation.

The management of multi-level caching hierarchy is a critical and challenging task. Although there exist many hardware and OS-based schemes, they are difficult to be adopted in practice since they incur non-trivial overheads and high complexity. In order to efficiently deal with this challenge, we propose MERCURY, a cost-effective and lightweight hardware support to coordinate with OS-based cache management schemes. Its basic idea is to leverage data similarity to reduce data migration costs and deliver high performance. Moreover, in order to accurately and efficiently capture the data similarity, we propose to use low-complexity Locality-Sensitive Hashing (LSH). In our design, in addition to the problem of space inefficiency, we identify that a conventional LSH scheme also suffers from the problem of homogeneous data placement. To address these two problems, we design a novel Multi-Core-enabled LSH (MC-LSH) that accurately captures the differentiated similarity across data. The similarity-aware MERCURY hence efficiently partitions data into L1 cache, L2 cache and main memory based on their distinct localities, which help optimize cache utilization and minimize the pollution in the last level cache. Experiments through real-world benchmarks further corroborate the efficacy and efficiency of MERCURY.

The RAID6 provides high reliability using double-parity-update at cost of high write penalty. In this paper, we propose HRAID6ML, a new logging architecture for RAID6 systems for enhanced energy efficiency, performance and reliability. HRAID6ML explores a group of Solid State Drives (SSDs) and Hard Disk Drives (HDDs): Two HDDs (parity disks) and several SSDs form RAID6. The free space of the two parity disks is used as mirrored log region of the whole system to absorb writes. The mirrored logging policy helps to recover system from parity disk failure. Mirrored logging operation does not introduce noticeable performance overhead to the whole system. HRAID6ML eliminates the additional hardware and energy costs, potential single point of failure and performance bottleneck. Furthermore, HRAID6ML prolongs the lifecycle of the SSDs and improves the systems energy efficiency by reducing the SSDs write frequency. We have implemented proposed HRAID6ML. Extensive trace-driven evaluations demonstrate the advantages of the HRAID6ML system over both traditional SSD-based RAID6 system and HDD-based RAID6 system.

Recent years, the application of solid-state disks (SSDs) increases explosively. All SSDs have to employ error correcting code (ECC) technique to ensure the reliability of flash memory at page level. However, data loss may be caused by bad block or chip failure of flash memory. To solve this problem, the article proposes a flash memory redundant array technique, which is similar to RAID-4. In this scheme, we utilize built-in NVRAM to cache the parity data update for minimal write to flash memory in parity channel.

Traditional analysis on the TCP performance attributes the overhead of TCP mainly on these aspects: many times of memory access and copy, much data transmission on bus and a large number of CPU cycles required by the protocol processing. Compared to TCP, the main advantage of RDMA lies in its single memory access which brings a higher data transmission bandwidth and lower CPU utilization. However, the hardware platforms that these analyses are based on are already out-of-date from today's view. The key point is that TCP and RDMA performance can change with the rapid development of the computer architecture and computer hardware components. In this paper, we re-analysis the performance of TCP and RDMA on the modern server platform (e.g., multi-core), and we find that the upper limit of TCP transmission bandwidth has been greatly upgraded compared to the old platform. Besides, the compatibility and complexity problems in programming brought by RDMA technology are still serious. Based on this conclusion, we constructed a network platform which uses multi-thread technology and TCP protocol to achieve a high speed transmission bandwidth in the InfiniBand network environment.

Data deduplication, an efficient space reduction method, has gained increasing attention and popularity in data-intensive storage systems. Most existing state-of-the-art deduplication methods remove redundant data at either the file level or the chunk level, which incurs unavoidable and significant overheads in time (due to chunking and fingerprinting). These overheads can degrade the write performance to an unacceptable level in a data storage system. In this paper, we propose P-Dedupe, a fast and scalable deduplication system. The main idea behind P-Dedupe is to fully compose pipelined and parallel computations of data deduplication by effectively exploiting the idle resources of modern computer systems with multi-core and many-core processor architectures. Our experimental evaluation of the P-Dedupe prototype based on real-world datasets shows that P-Dedupe speeds up the deduplication write throughput by a factor of 2~4 through pipelining deduplication and parallelizing hash calculation and achieves 80%~250% of the performance of a conventional storage system without data deduplication.

Embedding transparency is one of the most important criteria for steganography. This paper mainly focuses on transparency-orientated encoding strategies for steganography on Voice over IP (VoIP). Our analysis of the existing encoding strategies proposed for steganography on storage media reveals that they have limited applicability in VoIP-based steganography because they enhance embedding transparency at the expense of a decreased embedding rate (ER). In this paper, we propose three encoding strategies based on digital logic for steganography on VoIP. Differing from the existing approaches, our strategies reduce the embedding distortion by improving the similarity between the cover and the covert message using digital logical transformations, instead of reducing the ER. Therefore, in contrast, our strategies improve the embedding transparency without sacrificing the embedding capacity. Of these three strategies, one adopts logical operations, one employs circular shifting operations and the third combines the operations of the first two. All these schemes are evaluated experimentally through their prototype implementations that are compared with some existing methods in a prototypical covert communication system based on VoIP (called StegVoIP). The experimental results show that the proposed strategies can provide better embedding performance than the existing approaches in terms of embedding transparency and ER.

In many network applications, Bloom filters are used to support exact-matching membership query for their randomized space-efficient data structure with a small probability of false answers. In this paper, we extend the standard Bloom filter to Locality-Sensitive Bloom Filter (LSBF) to provide Approximate Membership Query (AMQ) service. We achieve this by replacing uniform and independent hash functions with locality-sensitive hash functions. Such replacement makes the storage in LSBF to be locality sensitive. Meanwhile, LSBF is space efficient and query responsive by employing the Bloom filter design. In the design of the LSBF structure, we propose a bit vector to reduce False Positives (FP). The bit vector can verify multiple attributes belonging to one member. We also use an active overflowed scheme to significantly decrease False Negatives (FN). Rigorous theoretical analysis (e.g., on FP, FN, and space overhead) shows that the design of LSBF is space compact and can provide accurate response to approximate membership queries. We have implemented LSBF in a real distributed system to perform extensive experiments using real-world traces. Experimental results show that LSBF, compared with a baseline approach and other state-of-the-art work in the literature (SmartStore and LSB-tree), takes less time to respond AMQ and consumes much less storage space.

Flash-based Solid State Drive (SSD) has been productively shipped and deployed in large scale storage systems. However, a single flash-based SSD cannot satisfy the capacity, performance and reliability requirements of the modern storage systems that support increasingly demanding data-intensive computing applications. Applying RAID schemes to SSDs to meet these requirements, while a logical and viable solution, faces many challenges. In this article, we propose a Hybrid Parity-based Disk Array architecture (short for HPDA), which combines a group of SSDs and two hard disk drives (HDDs) to improve the performance and reliability of SSD-based storage systems. In HPDA, the SSDs (data disks) and part of one HDD (parity disk) compose a RAID4 disk array. Meanwhile, a second HDD and the free space of the parity disk are mirrored to form a RAID1-style write buffer that temporarily absorbs the small write requests and acts as a surrogate set during recovery when a disk fails. The write data is reclaimed to the data disks during the lightly loaded or idle periods of the system. Reliability analysis shows that the reliability of HPDA, in terms of MTTDL (Mean Time To Data Loss), is better than that of either pure HDD-based or SSD-based disk array. Our prototype implementation of HPDA and the performance evaluations show that HPDA significantly outperforms either HDD-based or SSD-based disk array.

Existing data storage systems based on the hierarchical directory-tree organization do not meet the scalability and functionality requirements for exponentially growing data sets and increasingly complex metadata queries in large-scale, Exabyte-level file systems with billions of files. This paper proposes a novel decentralized semantic-aware metadata organization, called SmartStore, which exploits semantics of files' metadata to judiciously aggregate correlated files into semantic-aware groups by using information retrieval tools. The key idea of SmartStore is to limit the search scope of a complex metadata query to a single or a minimal number of semantically correlated groups and avoid or alleviate brute-force search in the entire system. The decentralized design of SmartStore can improve system scalability and reduce query latency for complex queries (including range and top-k queries). Moreover, it is also conducive to constructing semantic-aware caching, and conventional filename-based point query. We have implemented a prototype of SmartStore and extensive experiments based on real-world traces show that SmartStore significantly improves system scalability and reduces query latency over database approaches. To the best of our knowledge, this is the first study on the implementation of complex queries in large-scale file systems.

With the development of the NAND-Flash technology, NAND-Flash based Solid-State Disk (SSD) has been attracting a great deal of attention from both industry and academia. While a range of SSD research topics, from interface techniques to buffer management and Flash Translation Layer (FTL), from performance to endurance and energy efficiency, have been extensively studied in the literature, the SSD being studied was by and large treated as a grey or black box in that many of the internal features such as advanced commands, physical-page allocation schemes and data granularity are hidden or assumed away. We argue that, based on our experimental study, it is these internal features and their interplay that will help provide the missing but significant insights to designing high-performance and high-endurance SSDs.

Due to the relatively low bandwidth of WAN (Wide Area Network) that supports cloud backup services, both the backup time and restore time in the cloud backup environment are in desperate need for reduction to make cloud backup a practical and affordable service for small businesses and telecommuters alike. Existing solutions that employ the deduplication technology for cloud backup services only focus on removing redundant data from transmission during backup operations to reduce the backup time, while paying little attention to the restore time that we argue is an important aspect and affects the overall quality of service of the cloud backup services. In this paper, we propose a CAusality Based deduplication performance booster for both cloud backup and restore operations, called CABdedupe, which captures the causal relationship among chronological versions of datasets that are processed in multiple backups/restores, to remove the unmodified data from transmission during not only backup operations but also restore operations, thus to improve both the backup and restore performances. CABdedupe is a middleware that is orthogonal to and can be integrated into any existing backup system. Our extensive experiments, where we integrate CABdedupe into two existing backup systems and feed real world datasets, show that both the backup time and restore time are significantly reduced, with a reduction ratio of up to 103 : 1.

This paper proposes a Dynamic Bloom filter Array (DBA) to represent membership for variable large data sets in storage systems in a scalable way. DBA consists of dynamically created groups of space-efficient Bloom Filters (BFs) to accommodate changes in set sizes. In each group, BFs are homogeneous and the data layout is optimized at the bit level, so that they can be accessed in parallel to achieve high query performance. DBA can effectively control its query accuracy by partially adjusting the error rate of constructing BFs, where each BF corresponds to an independent subset of the data set to facilitate element location and membership confirmation. Further, DBA supports element deletion by introducing a lazy update policy. We prototype and evaluate our DBA scheme as a scalable fast index in the MAD2 deduplication storage system. Experimental results show that DBA (with 64 BFs per group) is capable of maintaining 90% of the peek query performance while scaling up to 160 BFs. DBA is also shown to excel in performance and space efficiency by theoretical analysis and other experiments based on real-world data sets.

The RAID6 architecture is playing an increasingly important role in modern storage systems due to its provision of very high reliability. However, its high write penalty, because of the double-parity-update overheads upon each write operation, has been a persistent performance bottleneck of the RAID6 systems. In this paper, we propose a log-assisted RAID6 architecture, called RAID6L, to boost the write performance of the RAID6 systems. RAID6L integrates a log disk into the traditional RAID6 architecture, and alleviates its write penalty by simplifying the processing steps to service a write request. On the other hand, RAID6L also guarantees that the accelerated RAID6 systems can still recover from double disk failures. The Parity Logging scheme was originally proposed to accelerate the XOR based RAID5, and we propose a method to make it also applicable to the Reed-Solomon based RAID6. We present a detailed comparison between RAID6L and Parity Logging, and show that RAID6L has several advantages over Parity Logging. Experimental results show that RAID6L significantly increases the data transfer rate and decreases the request response time when compared with the traditional RAID6 and Parity Logging systems.

In this paper, we present the design and performance evaluation of Oasis, an active storage framework for object-based storage systems that complies with the current T10 OSD standard. In contrast with previous work, Oasis has the following advantages. First, Oasis enables users to transparently process the OSD object and supports different processing granularity (from the single object to all the objects in the OSD) by extending the OSD object attribute page defined in the T10 OSD standard. Second, Oasis provides an easy and efficient way for users to manage the application functions in the OSD by using the existing OSD commands. Third, Oasis can authorize the execution of the application function in the OSD by enhancing the T10 OSD security protocol, allowing only authorized users to use the system. We evaluate the performance and scalability of our system implementation on Oasis by running three typical applications. The results indicate that active storage far outperforms the traditional object-based storage system in applications that filter data on the OSD. We also experiment with Java based applications and C based applications. Our experiments indicate that Java based applications may be bottlenecked for I/O-intensive applications, while for applications that do not heavily rely on the I/O operations, both Java based applications and C based applications achieve comparable performance. Our microbenchmarks indicate that Oasis implementation overhead is minimal compared to the Intel OSD reference implementation, between 1.2% to 5.9% for Read commands and 0.6% to 9.9% for Write commands.

The RAID-6 architecture is playing an increasingly important role in modern storage systems. There are generally two kinds of RAID-6 codes, horizontal codes and vertical codes. Horizontal codes have been extensively studied and widely implemented, while vertical codes have not gained the equal attention. In this paper, we investigate the state-of-the-art horizontal and vertical RAID-6 codes and select two representative ones, RDP for horizontal codes and P-Code for vertical codes, to compare their performance. Since the code lengths of vertical codes are usually restricted, we first provide two efficient code shortening algorithms for vertical codes, by which the length of a vertical code can be extended to an arbitrary given one. In the context of our code shortening algorithms for vertical codes, we compare the theoretical performance of RDP and P-Code at consecutive lengths, and examine their practical behaviors in the real environment. Theoretical analysis and experimental evaluation results have demonstrated that vertical codes can provide comparable, and sometimes even better, performance than horizontal codes.

Detecting duplicates over sliding windows is an important technique for monitoring and analysing data streams. Since recording the exact information of elements in a sliding window can be RAM-resource-intensive and introduce an unacceptable search complexity, several approximate membership representation schemes have been proposed to build in-memory fast indices. However, various challenges facing RAM utilization and scalability remain. This paper proposes a Detached Counting Bloom filter Array (DCBA) to flexibly and efficiently detect duplicates over sliding windows. A DCBA consists of an array of detached counting Bloom filters (DCBFs), where each DCBF is essentially a Bloom filter that is associated with a detached timer (counter) array. The DCBA scheme functions as a circular FIFO queue and keeps a filling DCBF for accommodating fresh elements and a decaying DCBF for evicting stale elements. DCBA allows the timer arrays belonging to fully filled DCBFs to be offloaded to disks to greatly improve the memory space efficiency. The fully filled DCBFs will remain stable until their elements become stale, which allows a DCBA to be efficiently replicated for the purpose of data reliability or information sharing. Further, DCBA can be cooperatively maintained by clustered nodes, which provides scalable solution for mining massive data streams. Mathematical analysis and experimental results show that a DCBA (containing 64 DCBFs) requires less than 10% of its components to be kept in RAM while maintaining more than 95% of its query performance, which significantly outperforms existing schemes in memory efficiency and scalability.

As more threads added to execute the multi-threaded applications in the many-core era, memory contentions among different threads impose a severe challenge to both the programmability and performance. Existing studies show that Transactional Memory (TM) is able to solve the programmability problem and scale well on the fine-grained applications in the SPLASH-2 benchmark suite. As more investigations on the coarse-grained applications in the STAMP benchmark suite, the long-running transactions block the parallelism among the concurrent transactions and failed to obtain the performance returns when the number of threads is beyond 4. In order to address this problem, we propose TMTLS, which combines TM with Thread-Level Speculation (TLS) to limit the number of concurrent executing transactions due to the memory contention in the runtime, divides the coarse-grained transactions into several epochs and assigns them to the available threads to speculatively exploit the parallelism in the coarse-grained transactions. This proposal not only alleviates the memory contention among the threads but also shortens the execution period of the coarse-grained transactions. Moreover, it further reduces the serializing overheads due to the transactional conflicts among the transactions. Our evaluation show this method achieves an average speedup of 2.27 over the baseline TM system under the 4 high-contention and coarse-grained applications selected from the STAMP benchmark suite on a 16-core CMP.

Performance and power consumption are two important design objectives for data centers consisting of thousands or tens of thousands of disks (or disk arrays). To leverage the two objectives, in this study we propose a multi-version disk array (MDA). The main idea of MDA is to exploit the I/O workload characteristics to guide the replication strategy by replicating multiple versions of the popular data blocks and simply offloading the write data to the free space of the reserved version region, thus achieving high performance in the burst period and low power consumption in the idle period. Our prototype implementation of MDA and the performance evaluations show that the performance of MDA outperforms that of traditional RAID10 by up to 34.4% and 42.3% in terms of the average response time for the online transaction processing (OLTP) application I/O and search engine I/O, respectively. Moreover, the energy efficiency of MDA outperforms that of RAID10 by up to 48.7% and 36.4%, respective to the aforementioned measures.

Due to the contention for the shared disk bandwidth, the user I/O intensity can significantly impact the performance of the online low-priority background tasks, thus reducing the reliability and availability of RAID-structured storage systems. In this paper, we propose a novel and practical scheme, called WorkOut (I/O Workload Outsourcing), to significantly boost the performance of those low-priority background tasks. WorkOut effectively outsources all write requests and popular read requests originally targeted at the degraded RAID set that is performing the low-priority background tasks to a surrogate RAID set. The lightweight prototype implementation of WorkOut and extensive trace-driven and benchmark-driven experiments on two case studies demonstrate that, compared with existing approaches, WorkOut effectively improves the performance of the low-priority background tasks, such as RAID reconstruction and RAID resynchronization. Importantly, WorkOut is portable and can be easily incorporated into any existing optimizing algorithms for RAID-structured storage systems.

In this article, we propose a simple but powerful online availability upgrade mechanism, S upplementary P arity A ugmentations( SPA ), to address the availability issue in parity-based RAID systems. The basic idea of SPA is to store and update the supplementary parity units on one or a few newly augmented spare disks for online RAID systems in the operational mode, thus achieving the goals of improving the reconstruction performance while tolerating multiple disk failures and latent sector errors simultaneously. By applying the exclusive OR operations appropriately among supplementary parity, full parity, and data units, SPA can reconstruct the data on the failed disks with a fraction of the original overhead that is proportional to the supplementary parity coverage, thus significantly reducing the overhead of data regeneration and decreasing recovery time in parity-based RAID systems. Our extensive trace-driven simulation study shows that SPA can significantly improve the reconstruction performance of the RAID5 and RAID5+0 systems, at an acceptable performance overhead imposed in the operational mode. Moreover, our reliability analytical modeling and sequential Monte-Carlo simulation demonstrate that SPA is consistently more than double the MTTDL of the RAID5 system and improves the reliability of the RAID5+0 system noticeably.

This paper presents a scalable and adaptive decentralized metadata lookup scheme for ultralarge-scale file systems (more than Petabytes or even Exabytes). Our scheme logically organizes metadata servers (MDSs) into a multilayered query hierarchy and exploits grouped Bloom filters to efficiently route metadata requests to desired MDSs through the hierarchy. This metadata lookup scheme can be executed at the network or memory speed, without being bounded by the performance of slow disks. An effective workload balance method is also developed in this paper for server reconfigurations. This scheme is evaluated through extensive trace-driven simulations and a prototype implementation in Linux. Experimental results show that this scheme can significantly improve metadata management scalability and query efficiency in ultralarge-scale storage systems.

In the background of cloud, self-destructing data mainly aims at protecting the data privacy. All the data and its copies will become destructed or unreadable after a user-specified period, without any user intervention. Besides, anyone cannot get the decryption key after timeout, neither the sender nor the receiver. The Washington's Vanish system is a system for self-destructing data under cloud computing, and it is vulnerable to “hopping attack” and “sniffer attack”. We propose a new scheme in this paper, called Safe Vanish, to prevent hopping attacks by way of extending the length range of the key shares to increase the attack cost substantially, and do some improvement on the Shamir Secret Sharing algorithm implemented in the Original Vanish system. We present an improved approach against sniffing attacks by using the public key cryptosystem to protectt from sniffing operations. In addition, we evaluate analytically the functionality of the proposed Safe Vanish system.

Data replication has been widely used as a mean of increasing the data availability of large-scale cloud storage systems where failures are normal. Aiming to provide cost-effective availability, and improve performance and load-balancing of cloud storage, this paper presents a cost-effective dynamic replication management scheme referred to as CDRM. A novel model is proposed to capture the relationship between availability and replica number. CDRM leverages this model to calculate and maintain minimal replica number for a given availability requirement. Replica placement is based on capacity and blocking probability of data nodes. By adjusting replica number and location according to workload changing and node capacity, CDRM can dynamically redistribute workloads among data nodes in the heterogeneous cloud. We implemented CDRM in Hadoop Distributed File System (HDFS) and experiment results conclusively demonstrate that our CDRM is cost effective and outperforms default replication management of HDFS in terms of performance and load balancing for large-scale cloud storage.

We propose RoLo (Rotated Logging), a new logging architecture for RAID10 systems for enhanced energy efficiency, performance and reliability. By spreading destaging I/O activities among short idle time slots and proactively reclaiming the stale logging space, RoLo rotates loggers among a logical logging space pool formed collectively from the free storage space available among mirrored disks. Therefore, without the extra dedicated log disks and the corresponding centralized logging, RoLo eliminates the additional hardware and energy costs, potential single point of failure and performance bottleneck. Furthermore, RoLo prolongs the lifecycle of the disks and improves the system's energy efficiency by reducing the disk spin up/down frequency. We develop three flavors of RoLo, that is, RoLo-E/R/P, to emphasize energy efficiency, reliability, and performance respectively. Extensive trace-driven evaluations demonstrate the advantages of the three RoLo schemes over both a RAID10 system with centralized logging architecture and a typical RAID10 system.

Parity redundancy is widely employed in RAID-structured storage systems to protect against disk failures. However, the small-write problem has been a persistent root cause of the performance bottleneck of such parity-based RAID systems, due to the additional parity update overhead upon each write operation. In this paper, we propose a novel RAID architecture, TRIP, based on the conventional parity-based RAID systems. TRIP alleviates the small-write problem by integrating and exploiting the temporal redundancy (i.e., snapshots and logs) that commonly exists in storage systems to protect data from soft errors while boosting write performance. During the write-intensive periods, TRIP can reduce the penalty of each small-write request to as few as one device IO operation, at a minimal cost of maintaining the temporal redundant information. Reliability analysis, in terms of Mean Time to Data Loss (MTTDL), shows that the reliability of TRIP is only marginally affected. On the other hand, our prototype implementation and performance evaluation demonstrate that TRIP significantly outperforms the conventional parity-based RAID systems in data transfer rate and user response time, especially in write-intensive environments.

Existing de-duplication solutions in cloud backup environment either obtain high compression ratios at the cost of heavy de-duplication overheads in terms of increased latency and reduced throughput, or maintain small de-duplication overheads at the cost of low compression ratios causing high data transmission costs, which results in a large backup window. In this paper, we present SAM, a Semantic-Aware Multitiered source de-duplication framework that first combines the global file-level de-duplication and local chunk-level deduplication, and further exploits file semantics in each stage in the framework, to obtain an optimal tradeoff between the deduplication efficiency and de-duplication overhead and finally achieve a shorter backup window than existing approaches. Our experimental results with real world datasets show that SAM not only has a higher de-duplication efficiency/overhead ratio than existing solutions, but also shortens the backup window by an average of 38.7%.

A single flash-based Solid State Drive (SSD) can not satisfy the capacity, performance and reliability requirements of a modern storage system supporting increasingly demanding data-intensive computing applications. Applying RAID schemes to SSDs to meet these requirements, while a logical and viable solution, faces many challenges. In this paper, we propose a Hybrid Parity-based Disk Array architecture, HPDA, which combines a group of SSDs and two hard disk drives (HDDs) to improve the performance and reliability of SSD-based storage systems. In HPDA, the SSDs (data disks) and part of one HDD (parity disk) compose a RAID4 disk array. Meanwhile, a second HDD and the free space of the parity disk are mirrored to form a RAID1-style write buffer that temporarily absorbs the small write requests and acts as a surrogate set during recovery when a disk fails. The write data is reclaimed back to the data disks during the lightly loaded or idle periods of the system. Reliability analysis shows that the reliability of HPDA, in terms of MTTDL (Mean Time To Data Loss), is better than that of either pure HDD-based or SSD-based disk array. Our prototype implementation of HPDA and performance evaluations show that HPDA significantly outperforms either HDD-based or SSD-based disk array.

Driven by the increasing demand for large-scale and high-performance data protection, disk-based de-duplication storage has become a new research focus of the storage industry and research community where several new schemes have emerged recently. So far these systems are mainly inline de-duplication approaches, which are centralized and do not lend themselves easily to be extended to handle global de-duplication in a distributed environment. We present DEBAR, a de-duplication storage system designed to improve capacity, performance and scalability for de-duplication backup/archiving. DEBAR performs post-processing de-duplication, where backup streams are de-duplicated and cached on server-disks through an in-memory preliminary filter in phase I, and then completely de-duplicated in-batch in phase II. By decentralizing fingerprint lookup and update, DEBAR supports a cluster of servers to perform de-duplication backup in parallel, and is shown to scale linearly in both write throughput and physical capacity, achieving an aggregate throughput of 1.7GB/s and supporting a physical capacity of 2PB with 16 backup servers.

Flash Translation Layer (FTL) is one of the most important components of SSD, whose main purpose is to perform logical to physical address translation in a way that is suitable to the unique physical characteristics of the Flash memory technology. The pure page-mapping FTL scheme, arguably the best FTL scheme due to its ability to map any logical page number (LPN) to any physical page number (PPN) to minimize erase operations, cannot be practically deployed since it consumes a prohibitively large RAM (SRAM or DRAM) space to store the page-mapping table for an SSD of moderate to large size. Alternatives to the pure page-mapping FTL, such as block-mapping FTLs, hybrid FTLs (e.g., FAST) and the latest demand-based page-mapping FTLs (e.g., DFTL), require significantly less RAM space but suffer from a few performance issues. Block-mapping FTLs perform poorly with higher erasure counts, particularly under random write workloads. Hybrid FTL schemes incur costly merge operations that hurt performance and increase the erasure counts. Performances of demand-based FTLs heavily depend on workload characteristics such as access locality, read/write ratio and request arrival interval time. This paper proposes a new FTL scheme, called HAT, to achieve the performance of a pure page-mapping FTL at the RAM cost of a block-mapping FTL while consuming lower energy, by hiding the address translation (HAT). The basic idea behind our scheme is to create a separate access path to read/write the address mapping information to significantly Hide the Address-Translation latency by incorporating a low energy-consuming solid-state memory device that stores the entire page mapping table. We implement an SSD simulator, SSDsim, to validate our HAT design and evaluate its performance. The extensive trace-driven simulation results show that the performance of HAT is within 0.8% of the pure page-mapping FTL, while consuming about 50% of the energy.

Deduplication has been widely used in disk-based secondary storage systems to improve space efficiency. However, there are two challenges facing scalable high-throughput deduplication storage. The first is the duplicate-lookup disk bottleneck due to the large size of data index that usually exceeds the available RAM space, which limits the deduplication throughput. The second is the storage node island effect resulting from duplicate data among multiple storage nodes that are difficult to eliminate. Existing approaches fail to completely eliminate the duplicates while simultaneously addressing the challenges. This paper proposes MAD2, a scalable high-throughput exact deduplication approach for network backup services. MAD2 eliminates duplicate data both at the file level and at the chunk level by employing four techniques to accelerate the deduplication process and evenly distribute data. First, MAD2 organizes fingerprints into a Hash Bucket Matrix (HBM), whose rows can be used to preserve the data locality in backups. Second, MAD2 uses Bloom Filter Array (BFA) as a quick index to quickly identify non-duplicate incoming data objects or indicate where to find a possible duplicate. Third, Dual Cache is integrated in MAD2 to effectively capture and exploit data locality. Finally, MAD2 employs a DHT-based Load-Balance technique to evenly distribute data objects among multiple storage nodes in their backup sequences to further enhance performance with a well-balanced load. We evaluate our MAD2 approach on the backend storage of B-Cloud, a research-oriented distributed system that provides network backup services. Experimental results show that MAD2 significantly outperforms the state-of-the-art approximate deduplication approaches in terms of deduplication efficiency, supporting a deduplication throughput of at least 100MB/s for each storage component.

A challenging issue in performance evaluation of parallel storage systems through trace-driven simulation is to accurately characterize and emulate I/O behaviors in real applications. The correlation study of inter-arrival times between I/O requests, with an emphasis on I/O-intensive scientific applications, shows the necessity to further study the self-similarity of parallel I/O arrivals. This paper analyzes several I/O traces collected in large-scale supercomputers and concludes that parallel I/Os exhibit statistically self-similar like behavior. Instead of Markov model, a new stochastic model is proposed and validated in this paper to accurately model parallel I/O burstiness. This model can be used to predicting I/O workloads in real systems and generate reliable synthetic I/O sequences in simulation studies.

With data amount growing at an ever increasing rate, file system, as the basis of data management, its functionality and content are becoming more and more complex. Thus the file system is forced to be upgraded constantly, in order to satisfy the growing demands. However, the reinvention of new file systems is always a high-cost task. Although there are frameworks such as User-space File System that may help speeding up the development process, ease the pain of deploying new functions, they also bring in significant overhead, especially when file operation frequency is fairly high. We designed and implemented a lightweight file system management framework based on LD_PRELOAD in user-space referred to as User-Space Preload (USP), and tested several management tasks with it. According to the comparison with FUSE, USP-based file system management achieved an obvious performance improvement.

Beyond the storage savings brought by chunk-level de-duplication in backup and archiving systems, a prominent challenge facing this technology is how to efficiently and effectively identify the duplicate chunks. Most of the chunk fingerprints used to identify individual chunks are stored on disks due to the limited main memory capacity. Checking for chunk fingerprint match on disk for every input chunk is known to be a severe performance bottleneck for the backup process. On the other hand, our intuitions and analyses of real backup data both indicate that duplicate chunks tend to strongly concentrate according to the data ownership. Motivated by this observation and to avoid or alleviate the aforementioned backup performance bottleneck, we propose DAM, a dataownership-aware multi-layered de-duplication scheme that exploits the data chunks' ownership and uses a tri-layered de-duplication approach to narrow the search space for duplicate chunks to reduce the total disk accesses. Our experimental results with real world datasets on DAM show it reduces the disk accesses by an average of 60.8% and shortens the de-duplication time by an average of 46.3%.

Storage management server, compatible with decoupled data and meta data fashion, is being employed frantically to build large-scale distributed storage system for performance and capacity. To design this hot commodity on flexibly managing the extracted data with little meta data but extended attributes has become a big challenge. This paper breaks a new way to object orient store and implement the dedicated prototype, called EDOS. We reexamine several new requirements and prior works, and employ Mini-DB as the back-end (\emph{like DBFS}) to guarantee the scalability and durability for EDOS. We design three kinds of object locators and multi-indices to improve retrieval performance and absorb random I/O, utilize a swap mechanism between internal and external objects for tunable throughput, which nested beneath the generic key-value database schema and benefited from memory pool technique. The replication component in Mini-DB helps to build the multi nodes in the distributed environment. It is easy to build up the object-based distributed file system by EDOS with ACID transaction semantics and high reliability. The experimental results show that our kernel-level implementation of EDOS performed better than the other existences in practice.