Trustworthy multi-modal learning reliably integrates multiple data sources. However, current methods often face a challenge, which is the inherent non-convex nature of deep neural networks. It leads to their susceptibility to local minima, ultimately resulting in a reduced capacity for generalization. To address this issue, we first create a theoretical framework, which extends the application of curriculum learning in multi-modal scenarios. Secondly, we propose a novel curriculum termed the Dynamic SRM Curriculum (DSRMC). It consists of two modules: a scoring function and a training schedule. The scoring function sorts samples from simple to complex. The training scheduler aims to manage the quantity of samples supplied at each round during training. DSRMC facilitates positioning the learned model in a flatter region of the loss landscape, thereby enhancing its overall generalization ability. Building on DSRMC, we eventually propose an innovative method termed as Curriculum Trustworthy Multi-modal Learning (CTML). It applies DSRMC in multi-modal learning application scenarios. Extensive experiments conducted on three open datasets show that the proposed CTML outperforms state-of-the-art methods, with a maximum improvement of 6.7% in macro F1 score. Our code and dataset are publicly available on https://github.com/HackerHyper/DSRMC.git .

Multimodal learning has celebrated remarkable success across diverse areas, yet faces the challenge of prohibitively expensive data collection and annotation when adapting models to new environments.In this context, domain adaptation has gained growing popularity as a technique for knowledge transfer, which, however, remains underexplored in multimodal settings compared with unimodal ones.This paper investigates multimodal domain adaptation, focusing on a practical partially shifting scenario where some modalities (referred to as anchors) remain domain-stable, while others (referred to as drifts) undergo a domain shift.We propose a bi-alignment scheme to simultaneously perform drift-drift and anchordrift matching.The former is achieved through adversarial learning, aligning the representations of the drifts across source and target domains; the latter corresponds to an "anchor dragging drift" strategy, which matches the distributions of the drifts and anchors within the target domain using the optimal transport (OT) method.The overall design principle features Adversarial Alignment with Anchor Dragging Drift, abbreviated as A 3 D 2 , for multimodal domain adaptation with partially shifted modalities.Comprehensive empirical results verify the effectiveness of the proposed approach, and demonstrate that A 3 D 2 achieves superior performance compared with state-of-the-art approaches.The code is available at: https: //github.com/sunjunaimer/A3D2.git.

Large-scale visual-language representation learning has demonstrated its potential for constructing multimodal models. Current methods employ encoders to learn special classification tokens ([CLS]) for cross-modal representation and alignment. However, current methods fail to effectively capture fine-grained attributes in images and structural information in text, limiting their performance on tasks requiring detailed semantics. To address these issues, we propose 3DASEM (3D Constrasive Anchors and Structured-Enhanced Module), which leverages attribute-structure injection and a novel three-dimensional contrastive learning strategy for precise multimodal alignment. Additionally, we constructed an attribute dataset designed to compensate for the lack of attribute and structural information. Experimental results demonstrate that 3DASEM consistently outperforms state-of-the-art models on benchmark datasets, achieving a 4.1% improvement in recall@1 on MSCOCO with the same size training data and closing or even surpassing the performance of models trained on more than three times the amount of data.

Because of the increasing speed gap between speed of compute and storage, caching is critical for improving the throughput of distributed file systems. It has been shown that prefetching can hide the latency resulted by network communication or disk operations. However, conventional client-based prefetching schemes are not efficient in distributed file systems as the limited computing and memory power of client nodes. In this paper, we present an effective and load-aware server-side prefetching scheme for distributed file systems, name SSPF. As an orthogonal approach, SSPF can be coupled with any existing caching scheme. First, SSPF exploits spatial locality to improve the efficiency of the prefetching cache and minimize memory requirement. Then, for maximizing the efficiency of cache, a multi-queue based cache manager is designed to coordinate between the prefetching blocks and other caching blocks. Furthermore, a heuristic-based request distribution strategy is proposed to optimize the balance between data server nodes and improve the overall performance. Finally, we have implemented and evaluated SSPF on the real distributed file system. Experimental results show that SSPF can significantly improve the read performance with negligible memory over-head.

Abstract The ‘out-of-place’ update nature of Solid State Drives (SSDs) introduces a security risk. Scrubbing, a secure deletion method, mitigates this issue but negatively affects SSD endurance and requires redundant data for recovery due to page errors. Previously, the RAID-5 based scheme was used to manage redundant data. While effective for reading, it causes significant write latency due to the channel blocking issue. In response, we propose FlashFox, which integrates secret sharing and Reed-Solomon coding into SSDs, enabling the application of scrubbing within an encrypted storage environment. This innovation ensures secure deletion by cleaning up sensitive data, thereby reducing wear on SSD endurance. Furthermore, we have developed a RAID-4-based scheme for implementing FlashFox on SSDs. This scheme, by assigning specific channels to manage redundant data, successfully avoids the channel blocking issue prevalent in RAID-5. Experimental results show FlashFox reduces endurance wear by at least 15% compared to traditional scrubbing methods and writing response delay by at least 8$\times $ compared to RAID-5 based scheme.

The massive computational and memory requirements of deep convolutional neural networks (DCNNs) have led to the development of neural network (NN) accelerators. However, as DCNN models grow in size, the demands on NN accelerators in terms of performance, memory bandwidth, and power efficiency continue to increase. We, therefore, present 9Ring , a flexible and efficient DCNN accelerator that takes full advantage of 3D-stacked memory, focusing on its hardware architecture, software scheduling, and optimization strategy. In particular, we first show that the mismatch between DCNN accelerators and DCNN models can lead to increased energy consumption and performance bottlenecks. We then present three flexible dataflow scheduling strategies to mitigate this mismatch. Afterward, we introduce an energy efficiency analysis tool that can automatically search for the optimal scheduling scheme with respect to different DCNN models for energy efficiency. Finally, we conduct an empirical study showing that 9Ring can reduce energy consumption by 31.4% and 43.9% on average, and improve performance by 12% and 10% on average, compared with Tetris and the NN accelerators on conventional low-power DRAM memory systems, respectively.

Federated learning (FL), a distributed learning paradigm focused on preserving data privacy, faces challenges due to varying data distributions among clients, impacting global model performance. To mitigate data heterogeneity, we propose FedUB-a personalized FL framework leveraging uniform feature representation and balancing personalization and collaboration in the classifier. Specifically, the uniform representation (UR) in FedUB provides all clients with a shared feature extractor and a common representation centroid (RC). Achieving this uniformity involves incorporating a regularization term to reduce the gap between global and local RCs. Additionally, an importance estimation of the parameters in the classifier is provided to partition the parameters into two parts: the personalized component and the collaborated component. Specifically, the personalized component adapts to local data, while the collaborated component prevents the classifier from overfitting local data. Theoretically, we establish the existence of the UR, demonstrating its effectiveness in reducing the average generalization bound. Experiments on benchmark datasets consistently demonstrate the performance gains and improved generalization behavior of FedUB.

Over the past four decades, Learning Classifier Systems (LCSs) have faced challenges in producing accurate and interpretable models for domains with continuous features, mainly due to the irrelevance issue caused by genotypic methods. These methods directly modify genotypes (conditions), leading to the creation of irrelevant rules. Phenotypic LCSs, which first modify a rule's phenotype (covered instance set) before altering its genotype, can avoid this issue. However, previous phenotypic LCSs struggle with overfitting, resulting in lower testing performance. In response, we propose a novel phenotypic LCS featuring innovations: 1) a heterogeneous phenotype approach in the rule discovery mechanism to alleviate overfitting, and 2) Informed Mutation leverages the inherent neighbouring of similar instances to enhance rule generalization, thereby improving model interpretability. The proposed LCS demonstrates its success with superior testing performance and more interpretable models in all experiments compared to other LCSs. Notably, in a problem with 2048 features, the proposed LCS model outperformed the genotypic UCS by achieving a 97.4% testing accuracy with just 13 rules, compared to the UCS's 9961 rules but only 49.9% accuracy.

The multi-view hash method converts heterogeneous data from multiple views into binary hash codes, which is one of the critical technologies in multimedia retrieval. However, the current methods mainly explore the complementarity among multiple views while lacking confidence in learning and fusion. Moreover, in practical application scenarios, the single-view data contains redundant noise. To conduct confidence learning and eliminate unnecessary noise, we propose a novel Adaptive Confidence Multi-View Hashing (ACMVH) method. First, a confidence network is developed to extract useful information from various single-view features and remove noise information. Furthermore, an adaptive confidence multi-view network is employed to measure the confidence of each view and then fuse multi-view features through a weighted summation. Lastly, a dilation network is designed to further enhance the feature representation of the fused features. To the best of our knowledge, we pioneer the application of confidence learning into the field of multimedia retrieval. Extensive experiments on two public datasets show that the proposed ACMVH performs better than state-of-the-art methods (maximum increase of 3.24%). The source code is available at https://github.com/HackerHyper/ACMVH.

Data I/O poses a significant bottleneck for distributed deep learning applications. Utilizing computing-node attached storage as a cache has become a prevalent solution to this problem. Given the large size of training datasets and the limited capacity of a single-node local storage, training samples are loaded in shards across computing nodes, requiring a deep learning job on one node to access samples from neighboring nodes. Compared to local node access, these neighboring accesses are highly inefficient for small samples. To mitigate this issue, we propose a node-aware prefetch algorithm that greedily fetches samples from neighboring nodes. Evaluation results show that our approach improves neighboring access performance by 60x for small samples of 64B size. For large samples of 1MB size, our approach still exhibits a 26% improvement.

When running deep learning training jobs, in order to prevent training loss due to softwarelhardware failures, a checkpointing mechanism is usually used to periodically store snapshots of the training model in non-volatile memory to facilitate recovery from failures. However, the recovery effect of this method is closely related to the checkpointing period and the failure time, and in the worst case, even nearly one period of training accuracy will be lost. With increasing model sizes, the need to develop more fine-grained model fault recovery methods to reduce model fault loss is more urgent for deep learning training tasks that use checkpointing mechanisms as fault recovery guarantees. We present TranLogs, a checkpoint failure recovery method combined with training logs for lossless failure recovery when training deep learning models. TranLogs proposes to use logging to supplement the checkpoint file and track the iterative process of model training parameters. For the situation where the checkpoint file is not updated in time and encounters model failures, resulting in a large loss during recovery, TranLogs can handle it effectively. The best-case scenario achieves about 24 % and 6% improvement in the loss and accuracy of the recovered training state, respectively, which is closer to the state already achieved by the model training at the moment of failure.

Existing graph systems focus mainly on the execution efficiency of the graph analysis tasks, often ignoring the importance and efficiency of time-evolving graph storage. However, to effectively mine the potential application values, an efficient storage system is important for time-evolving graphs whose storage requirement scales with the increasing number of snapshots. Storage cost and snapshot access speed are the two most important performance indicators for a time-evolving graph storage system, which are challenging for designers of such systems because they are conflicting goals. In this article, we address these challenges by proposing an efficient storage scheme for the large time-evolving graphs. We first design aSnapshot-level Data Deduplication (SLDD)strategy to eliminate the large number of repeated vertices and edges among the snapshots, and then aStructure-Changing Graph Representation (SCGR)to significantly improve the snapshot access speed. We implement an efficient time-evolving graph storage system, TgStore, based on this scheme to effectively store large-scale time-evolving graphs, aiming to efficiently support the time-evolving graph analysis tasks. Experimental results show that TgStore can obtain a high compression ratio of 43.03:1 when storing 100 snapshots of Twitter, while with an average snapshot access speedup of 16×. Efficient storage scheme enables TgStore to efficiently support time-evolving graph algorithms. For example, when executing the Pagerank algorithm on the time-evolving graph of Twitter, TgStore outperforms Graphone, a state-of-the-art time-evolving graph storage system, by 15.9× in algorithm execution speed and 1.45× in memory usage.

The sparse representation of graphs has shown great potential for accelerating the computation of graph applications (e.g., social networks and knowledge graphs) on traditional computing architectures (CPU, GPU, or TPU). But, the exploration of large-scale sparse graph computing on processing-in-memory (PIM) platforms (typically with memristive crossbars) is still in its infancy. To implement the computation or storage of large-scale or batch graphs on memristive crossbars, a natural assumption is that a large-scale crossbar is demanded, but with low utilization. Some recent works question this assumption; to avoid the waste of storage and computational resource, the fixed-size or progressively scheduled “block partition” schemes are proposed. However, these methods are coarse-grained or static and are not effectively sparsity-aware. This work proposes the dynamic sparsity-aware mapping scheme generating method that models the problem with a sequential decision-making model, and optimizes it by reinforcement learning (RL) algorithm (REINFORCE). Our generating model [long short-term memory (LSTM), combined with the dynamic-fill scheme] generates remarkable mapping performance on the small-scale graph/matrix data (complete mapping costs 43% area of the original matrix) and two large-scale matrix data (costing 22.5% area on qh882 and 17.1% area on qh1484). Our method may be extended to sparse graph computing on other PIM architectures, not limited to the memristive device-based platforms.

Learning the hash representation of multi-view heterogeneous data is an important task in multimedia retrieval. However, existing methods fail to effectively fuse the multi-view features and utilize the metric information provided by the dissimilar samples, leading to limited retrieval precision. Current methods utilize weighted sum or concatenation to fuse the multi-view features. We argue that these fusion methods cannot capture the interaction among different views. Furthermore, these methods ignored the information provided by the dissimilar samples. We propose a novel deep metric multi-view hashing (DMMVH) method to address the mentioned problems. Extensive empirical evidence is presented to show that gate-based fusion is better than typical methods. We introduce deep metric learning to the multi-view hashing problems, which can utilize metric information of dissimilar samples. On the MIR-Flickr25K, MS COCO, and NUS-WIDE, our method outperforms the current state-of-the-art methods by a large margin (up to 15.28 mean Average Precision (mAP) improvement).

Directory tree walks on parallel file systems are costly operations frequently required by many storage management tasks. Even listing the contents of a single directory can take minutes to hours for huge directories, as the tree walk performance of parallel file systems in Linux is severely throttled by sequentially accessing distributed metadata for each file through the syscall interface.

Data deduplication has become a commodity feature in flash storage products to effectively reduce redundant write data and improve space efficiency. However, it also introduces computing and memory overhead to generate and store the cryptographic hash (fingerprint) in face of the moderate data redundancy in primary storage. With the advent of 3D XPoint and Z-NAND technologies, and the stronger cryptographic hash functions in use, such as SHA-256, both the computing and memory overheads are increasingly serious performance bottlenecks for inline data deduplication in these ultra-low latency flash storage. To address these problems, we propose an ECC-assisted Deduplication approach, called EaD, which exploits the ECC property and the asymmetric read-write performance characteristics of modern flash storage. EaD first identifies data similarity by leveraging the device-generated ECC values of data chunks as their fingerprints, significantly reducing the costly MD5/SHA-based cryptographic hash computing and alleviating the memory space overhead. Based on the identification results, similar data chunks and their ECCs are read from the flash to perform a byte-by-byte comparison in memory to definitively identify and remove redundant data chunks. Our experiments show that the EaD approach significantly increases I/O performance by up to 4.2${\times }$, with an average of 2.5${\times }$, compared with the existing MD5/SHA- and sampling-based deduplication approaches.

Oblivious Random Access Machines (ORAMs) allow cloud users to access remote data without leaking access patterns. Current ORAM solutions achieve this goal at expense of either increasing bandwidth consumption by a factor of$O(\log N)$, where$N$is the number of data blocks, or relying on homomorphic encryption for bandwidth amplification reduction to$O(1)$. Furthermore, most ORAMs are only effective for a single user, while the solutions for multi-user scenarios often induce security or performance problems. This article introducesTianji— an asynchronous multi-user Shamir-based ORAM system — which supports asynchronous network access scenarios for multiple users with improved security and performance.Tianjiis implemented on top ofS$^{3}$3ORAM$^+$+—an extension of the state-of-the-art Shamir-based S$^{3}$ORAM with a new non-eviction data write-back scheme to achieve$O(1)$consumption in both bandwidth amplification and storage capacity. Our experimental results show that the proposedTianjiwithS$^{3}$3ORAM$^+$+can significantly outperform the state-of-the-art multi-userTaoStorein terms of access latency and client scalability. Additionally, its average response time is relatively stable when client loads increase.

Statistical analytics on encrypted data requires a fully-homomorphic encryption (FHE) scheme. However, heavy computation overheads make FHE impractical. In this paper we propose a novel approach to achieve privacy-preserving statistical analysis on an encrypted database. The main idea of this work is to construct a privacy-preserving calculator to calculate attributes’ count values for later statistical analysis. To authenticate these encrypted count values, we adopt an authenticable additive homomorphic encryption scheme to construct the calculator. We formalize the notion of an authenticable privacy-preserving calculator that has properties of broadcasting and additive homomorphism. Further, we propose a cryptosystem based on binary vectors to achieve complex logic expressions for statistical analysis on encrypted data. With the aid of the proposed cryptographic calculator, we design several protocols for statistical analysis including conjunctive, disjunctive and complex logic expressions to achieve more complicated statistical functionalities. Experimental results show that the proposed scheme is feasible and practical.

With the rapid advancement of information technology, a large amount of high-value data have been generated. To exploit the potential value of big data and at the same time to protect individuals' sensitive information, a global combination and clustering based differential privacy (DP) mixed data publishing method is proposed in this paper. The main idea of the proposed method is to improve the truthfulness of the published data as well as to enhance the utility by shifting the sensitivity of query function from a single record to a group of records using$k$-median clustering algorithm. Specifically, to improve the accuracy and utility of categorical attributes, a global combination method is proposed to take the correlation among categorical attributes into account. The proposed combination method takes all categorical attributes as a unit and then applies the exponential mechanism to improve the data utility. Then we combine it with the$k$-median clustering with differential privacy to publish the mixed data. Theoretical analysis shows that the proposed method satisfies$\varepsilon$-differential privacy. Experimental results on real datasets illustrate that the proposed method has a much lower information loss and time overhead than the state-of-the-art approach for the same parameters.

Traditional searchable symmetric encryption (SSE) schemes rarely support context-aware semantic extension, and then lead to the searched results being incomplete or deviating from the user’s query intention. To address this problem, a new context-aware semantically extensible searchable symmetric encryption based on Word2vec model (CASE-SSE) is proposed to achieve context-aware semantic extension in this article. The proposed scheme utilizes outsourced datasets as corpora to extract all keywords for training the Word2vec model, and the trained results is the ontology knowledge base that can be used to extend the semantics of query keywords directly. Further, to facilitate multi-keyword search using the extended query vector, we use the$k$-means clustering algorithm to classify outsourced datasets. We then construct an AVL-tree index and an inverted index based on the classified results, thereby achieving efficient context-aware semantically extensible SSE. The security analysis indicates it is secure and effective. The experimental results show that our scheme is superior in both efficiency and accuracy.

Dual-mode phase change memory (PCM) allows each PCM cell to operate concurrently in two modes: multi-level cell (MLC) and single-level cell (SLC), each with its own distinct features in transfer speeds and power consumption. This paper describes the design and implementation of a log-structured file system, called DPLFS, which is proposed to provide a storage system with high throughput and low latencies. DPLFS achieves this merit by exploiting the dual-mode PCM. In particular, file data in DPLFS is stored in MLC, while its metadata is maintained in SLC, and will be written back to MLC when the space in SLC is not enough. DPLFS adapts log-structured file system techniques to exploit fast random accesses in SLC and large capacity in MLC and maintains a file directory in SLC to speed up the data retrieval speed. Our experimental results show a 21% in write-intensive workloads and a 11% in read-intensive workloads, compared to the state-of-the-art file systems.

In parallel and distributed file systems, caching can improve data performance and metadata operations. Currently, most distributed file systems adopt a write-back data cache for performance and a write-through metadata cache for simplifying consistency. However, with modern file systems scales and workloads, write-through metadata caching can impact overall file system performance, e.g., through lock contention and heavy RPC loads required for namespace synchronization and transaction serialization. This paper proposes a novel metadata write-back caching (MetaWBC) mechanism to improve the performance of metadata operations in distributed environments. To achieve extreme metadata performance, we developed a fast, lightweight, and POSIXcompatible memory file system as a metadata cache. Further, we designed a file caching state machine and included other performance optimizations. We coupled MetaWbc with Lustre and evaluated that MetaWbc can outperform the native parallel file system by up to 8x for metadata-intensive benchmarks, and up to 7x for realistic workloads in throughput.

File system metadata is the data in charge of maintaining namespace, permission semantics and location of file data blocks. Operations on the metadata can account for up to 80% of total file system operations. As such, the performance of metadata services significantly impacts the overall performance of file systems. A large-scale distributed file system (DFS) is a storage system that is composed of multiple storage devices spreading across different sites to accommodate data files, and in most cases, to provide users with location independent access interfaces. Large-scale DFSs have been widely deployed as a substrate to a plethora of computing systems, and thus their metadata management efficiency is crucial to a massive number of applications, especially with the advent of the Big Data age, which poses tremendous pressure on underlying storage systems. This paper reports the state-of-the-art research on metadata services in large-scale distributed file systems, which is conducted from three indicative perspectives that are always used to characterize DFSs: high-scalability, high-performance, and high-availability, with special focus on their respective major challenges as well as their developed mainstream technologies. Additionally, the paper also identifies and analyzes several existing problems in the research, which could be used as a reference for related studies.

Abstract The write performance of multi-level cell (MLC) is several times slower than single-level cell (SLC); however, the cost per bit of MLC is much lower than SLC. Dual-mode flash (the medium can be partially switched to SLC mode by programming only 1 bit in some cells) can combine SLC and MLC to provide trading density opportunity for performance. In this paper, we present Hercules—a hybrid storage system that couples dual-mode flash memory and hard drive disk (HDD)—based on the content locality principle for high storage performance. The data are divided into two types: the reference data for read operation and the delta data for write operation. The reference data are stored in SLC and the delta data in MLC or HDD in sequential orders. Hercules organizes the metadata for the mapping of the physical locations of the reference blocks and the delta data of the original blocks, intelligently identifies hot/cold data and performs the data migration between MLC and disk for performance improvements. To validate our findings, we implemented Hercules and made evaluation to show that Hercules can effectively improve the data access speed and reduce the response time, compared with the Flashcache storage structure, and in particular, with Hercules, we can achieve 10% performance improvement over the system in absence of hot delta data caching.

In high-performance computing (HPC), data and metadata are stored on special server nodes and client applications access the servers’ data and metadata through a network, which induces network latencies and resource contention. These server nodes are typically equipped with (slow) magnetic disks, while the client nodes store temporary data on fast SSDs or even on non-volatile main memory (NVMM). Therefore, the full potential of parallel file systems can only be reached if fast client side storage devices are included into the overall storage architecture. In this article, we propose an NVMM-based hierarchical persistent client cache for the Lustre file system (NVMM-LPCC for short). NVMM-LPCC implements two caching modes: a read and write mode (RW-NVMM-LPCC for short) and a read only mode (RO-NVMM-LPCC for short). NVMM-LPCC integrates with the Lustre Hierarchical Storage Management (HSM) solution and the Lustre layout lock mechanism to provide consistent persistent caching services for I/O applications running on client nodes, meanwhile maintaining a global unified namespace of the entire Lustre file system. The evaluation results presented in this article show that NVMM-LPCC can increase the average read throughput by up to 35.80 times and the average write throughput by up to 9.83 times compared with the native Lustre system, while providing excellent scalability.

Summary LSM‐trie‐based key‐value (KV) store is often used to manage an ultralarge dataset in reality by introducing a number of sublevels at each level, its linear growth pattern can fairly reduce the write amplification in store operations. Although this design is effective for the write operation, the last level holds a large proportion of KV items, leading to the extreme imbalance of data distribution. Therefore, to support efficient read, we need to carefully consider this imbalance. On the other hand, to ensure that acquired data is latest, the LSM‐trie needs to search the dataset at different levels one by one, and this search method may take a lot of unnecessary time. When the number of items is ultralarge, the random lookup performance may be poor due to the imbalance data distribution. To address this issue, we improve the read performance of the LSM‐trie by changing its serial search to parallel search, using two threads to simultaneously search at the last level and other levels, respectively. Our experiment results show that the read performance of the LSM‐trie can be improved up to 98.35% and on average 71.55%.

In this paper, we design an efficient deduplication algorithm based on the distributed storage architecture of Ceph. The algorithm uses on-line block-level data deduplication technology to complete data slicing, which neither affects the data storage process in Ceph nor alter other interfaces and functions in Ceph. Without relying on any central node, the algorithm maintains the characteristics of Ceph by designing a special hash object to store the data fingerprint, and uses the CRUSH algorithm to judge the data duplication based on calculation, instead of global search. The algorithm replaces the duplicate data with the deduplicated objects, which storage their fingerprints with less storage space. We compare the effects of different block sizes with respect to the performance and deduplication rates through experimental studies, and select the most appropriate block size in our prototype implementation. The experimental results show that the algorithm can not only effectively save the storage space but also improve the bandwidth utilization when reading and writing the duplicate data.

Most high-performance computing (HPC) clusters use a global parallel file system to enable high data throughput. The parallel file system is typically centralized and its storage media are physically separated from the compute cluster. Compute nodes as clients of the parallel file system are often additionally equipped with SSDs. The node internal storage media are rarely well-integrated into the I/O and compute workflows. How to make full and flexible use of these storage media is therefore a valuable research question.

Indexes are essential in data management systems to increase the speed of data retrievals. Widespread data structures to provide fast and memory-efficient indexes are prefix tries. Implementations like Judy, ART, or HOT optimize their internal alignments for cache and vector unit efficiency. While these measures usually improve the performance substantially, they can have a negative impact on memory efficiency. In this paper we present Hyperion, a trie-based main-memory key-value store achieving extreme space efficiency. In contrast to other data structures, Hyperion does not depend on CPU vector units, but scans the data structure linearly. Combined with a custom memory allocator, Hyperion accomplishes a remarkable data density while achieving a competitive point query and an exceptional range query performance. Hyperion can significantly reduce the index memory footprint and its performance-to-memory ratio is more than two times better than the best implemented alternative strategy for randomized string data sets.

Summary Performance isolation is highly desirable in cloud platforms where the virtual disks of virtual machines are simply large files on the shared and networked storage servers. However, existing isolation techniques cannot deal with the implications of the file system used by the networked storage servers, such that underlying resource usage is unpredictable (eg, the delayed write‐back mechanism could postpone writes, and the journaling mechanism could amplify writes). The lack of visibility on underlying resource usage leads to the predicament of being unable to meet isolation goals. In this paper, we present a software‐defined file system (SDFS) that exploits the underlying file system to allocate resources at per‐image‐file granularity and provide tenants with guaranteed throughput. The SDFS comprises two components: control plane and data plane. At the control plane, we provide a set of system calls to document tenant performance requirements into the metadata of image files. At the data plane, we construct a file‐based scheduler to manage memory and disk resources according to the tenant performance requirements. The SDFS design does not require a modification to guest operating systems, hypervisors, or file server protocols. Through a prototype implementation, we demonstrate that the SDFS can meet isolation goals and increase resource utilization with negligible overhead.

With the development of technology and price decline, flash-based Solid state drives (SSDs) are rapidly used to construct RAIDs by storage vendors. SSD does not need to seek and rotate, therefore, its read performance is much better than that of HDD. However, the small write performance of SSD is limited by its inherent characteristics such as out- of-place updates and garbage collection. The traditional parity-based RAID also has small write problem because of parity updating. SSD-based RAID, which is called RAIS, is generally based on the traditional RAID design and implementation. Consequently, handling small write requests is a serious challenge when SSD is used to construct parity-based RAID. In RAIS storage system, small write requests not only result in poor performance, but also shorten the lifetime of each SSD. In this paper, we propose a novel write through cache method, called CRAIS5, which uses a RAM as the write cache of RAIS5, and adopts the write-through mode to delay the parity update. The write-through cache method makes full use of the flash characteristics, and removes the pre-read operation. CRAIS5 improves the small write performance and reduces the erase time. We have implemented the CRAIS5 prototype in Disksim simulator, and used the real traces to evaluate the performance. The evaluations demonstrate that our CRAIS5 outperforms RAIS5, and PPC, on average, by 42.82%, and 34.49% respectively.

HPC file systems today work in a best-effort manner where individual applications can flood the file system with requests, effectively leading to a denial of service for all other tasks. This paper presents a classful Token Bucket Filter (TBF) policy for the Lustre file system. The TBF enforces Remote Procedure Call (RPC) rate limitations based on (potentially complex) Quality of Service (QoS) rules. The QoS rules are enforced in Lustre's Object Storage Servers, where each request is assigned to an automatically created QoS class.

Summary Virtual clusters (VCs) have exhibited various advantages over traditional cluster computing platforms by virtue of their extensibility, reconfigurability, and maintainability. As such, they have become a major execution environment for cloud‐based cluster applications. However, compared with traditional clusters, their distributed‐memory programming paradigm still remains largely unchanged, which implies that cluster applications cannot be efficiently deployed in VCs, especially when virtual machines (VMs) are running in different physical hosts. Recently, some efforts have been made to improve inter‐VM communication, resulting in many studies on how cluster applications could take advantages of VCs. However, most of them mainly focus on the situation that the VMs are all coresident on the same physical machine where the message passing mechanism is usually optimized away by exploiting the host's shared memory. In this paper, we present a design and implementation of Naplus, a kernel‐based virtual machine approach to the inter‐VM communications that are across different physical hosts. Naplus is based on Nahanni, a mechanism for shared‐memory communication in virtual environments. As such, it not only inherits the major merits of Nahanni with respect to flexible data structures and efficient synchronization but also achieves a shared‐memory paradigm among VMs. With Naplus, we enable the size of shared space to be maximized as large as the sum of each machine's local memory to accommodate cluster applications with large memory footprints. We prototype Naplus in a dual‐host system where an empirical study is conducted to show the effectiveness of the Naplus approach in achieving distributed shared memory for VCs in data centers. Copyright © 2017 John Wiley & Sons, Ltd.

Summary In this paper, we propose a simulation model to study real‐world replication workflows for cloud storage systems. With this model, we present three new methods to maximize the storage space usage during replica creation, and two novel QoS aware greedy algorithms for replica placement optimization. By using a simulation method, our algorithms are evaluated, through a comparison with the existing placement algorithms, to show that (i) a more evenly distributed replicas for a data set can be achieved by using round‐robin methods in replica creation phase and (ii) the two proposed greedy algorithms, namedGS_QoSandGS_QoS_C1, not only have more economical results than those from Chenet al., but also guarantee the QoS for clients. Copyright © 2016 John Wiley & Sons, Ltd.

In this article, we design and implement a cooperative shingle-aware file system, called CosaFS , on heterogeneous storage devices that mix solid-state drives (SSDs) and shingled magnetic recording (SMR) technology to improve the overall performance of storage systems. The basic idea of CosaFS is to classify objects as hot or cold objects based on a proposed Lookahead with Recency Weight scheme. If an object is identified as a hot (small) object, then it will be served by SSD. Otherwise, cold (large) objects are stored on SMR. For an SMR, large objects can be accessed in large sequential blocks, rendering the performance of their accesses comparable with that of accessing the same large sequential blocks as if they were stored on a hard drive. Small objects, such as inodes and directories, are stored on the SSD where “seeks” for such objects are nearly free. With thorough empirical studies, we demonstrate that CosaFS, as a cooperative shingle-aware file system, with metadata separation and cache-assistance, is a very effective way to handle the disk-based data demanded by the shingled writes and outperforms the device- and host-side shingle-aware file systems in terms of throughput, IOPS, and access latency as well.

We present a network I/O allocation framework, called Raccoon, for workload-aware VM scheduling algorithm to facilitate hybrid I/O workloads in virtual environments. Raccoon combines the strengths of paravirtual I/O and SR-IOV techniques to not only minimize the network latency, but also optimize the bandwidth utilization for workload-aware VM scheduling. In Raccoon, a limited number of VFs in SR-IOV are granted to I/O-intensive VMs while the paravirtual Network Interface Cards (vNICs) are allocated to other non-I/O-intensive VMs as the default resources. With this design, Raccoon provides latency reduction and bandwidth guarantee under the premise that I/O-intensive VMs will always be granted the VFs to facilitate their I/O operations. The types of workloads in each VM are identified at runtime by modified XenMon. By leveraging the ACPI Hotplug technique, Raccoon can adaptively plugin and plugout the SR-IOV VFs upon the changes of VM requirements so that an efficient I/O workload-aware VM scheduling algorithm can be implemented based on the bonding driver technique. The experimental results reveal that Raccoon can combine the benefits of para-virtual I/O and SR-IOV techniques to improve the overall performance of virtualized platforms with VMs that have diverse I/O workloads.

In a WSN, sink-node gathers data from surrounding nodes and sends it to outside world via a gateway. Therefore, its location information is important to both attacker and network operator. The former can launch attacks on a sink-node to steal information or damage it, while the latter must hide its location to ensure data's safety, and physical protection. Providing sink-node location information anonymity in WSN against a local and global adversary with minimal energy penalties, we propose a novel technique Location Privacy via Software Defined Networking (LPSDN), inspired by the concept of Software Defined Networking. LPSDN uses three kinds of nodes: a centralized controller, a special buffering and forwarding node, and a slave node. These nodes work together to hide sink-node location information against traffic analysis attack. LPSDN conserves more energy, because it does not generate fake packets to hide sink-node location information. A GUI/command-line can be used to monitor (energy status, traffic density) and control the whole network. Comparing with traditional WSNs, LPSDN efficiently reduces processing burden on slave nodes, increases overall network life, and creates on-demand traffic density zones to deceive an adversary. It also provides better maintainability, manageability, and control over all nodes and their behavior.

Shingled write disk (SWD) is a new kind of HDD, which increases areal density by adopting SMR technology. In the hybrid storage system consisting of SSDs and SWDs, there are some special characteristics such as large performance gap between SSDs and SWDs. In this paper we explore how to efficiently combine the advantages of SSDs and SWDs together. We propose Apas, an application aware hybrid storage system combining SSDs and SWDs, where SSDs are used as nonvolatile cache and SWDs work as lower level storage. In Apas we propose a cache partitioning policy to divide the cache space according to the application's characteristics. To mitigate the write amplification problem in SWDs, we first classify data bands in the SWDs into five types according to the distribution of valid data in the bands, and then design a new garbage collection policy combining the aggressive data cleaning and lazy data cleaning together according to the detected idle time and the five types of bands. Experiments are carried out and the results show that Apas has good performance.

In modern RAID-structured storage systems, reliability is guaranteed by the use of parity blocks. But the parity-update overheads upon each write request have become a performance bottleneck of RAID systems. In some ways, an attached log disk is used to improve the write performance by delaying the parity blocks update. However, these methods are data-block-oriented and they need more time to rebuild or synchronize the RAID system when a data disk or the log disk fails. In this paper, we propose a novel optimization method, called SWO, which can improve RAID write performance and reconstruction performance. Moreover, when handling a write request, the SWO chooses reconstruction- write or read-modify-write combining with the log information to further minimize the number of pre- read data blocks. We have implemented the proposed SWO prototype and carried out some performance measurements using IOmeter and RAIDmeter. We have implemented the main idea of RAID6L in RAID5 and call it RAID5L. At the same time, we have evaluated the reconstruction time and the synchronization time of the SWO. Our experiments demonstrate that the SWO significantly improves write performance and saves more time than Data Logging and RAID5L when rebuilding and synchronizing.

Summary Although deduplication can reduce data volume for backup, it pauses the running system for the purpose of data consistency. This problem becomes severe when the target data are Virtual Machine Image (VMI), the volume of which can scale up to several gigabytes. In this paper, we propose an online framework for VM image backup and recovery, called VMBackup, which comprises three major components: (1) Similarity Retrieval that indexes chunks' fingerprints by its segment id for fast identification, (2) one‐level File‐Index that efficiently tracks file id to its content chunks in a correct order, and (3) Adjacent Storage model that places adjacent chunks of an image in the same disk partition to maximize chunk locality. The experimental results show that (1) the images of one OS serial and the same custom can share high percentage of duplicated contents, (2) variable‐length chunk partitioning is superior to fixed‐length chunk partitioning for deduplication, and (3) VMBackup, in our environment, can provide 8M/s backup throughput and 9.5M/s recovery throughput, which are only 15% and 4% less than storage systems without deduplication. Copyright © 2015 John Wiley & Sons, Ltd.

In cloud services, users may frequently be required to reveal their personal private information which could be stored in the cloud to used by different parts for different purposes. However, in a cloud-wide storage network, the servers are easily under strong attacks and also commonly experience software/hardware faults. As such, the private information could be under great risk in such an untrusted environment. Given that the presented personal sensitive information is usually out of user's controlin most cloud-based services, ensuring data security and privacy protection with respect to untrusted storage network has become a formidable challenge in research. To address these challenges, in this paper we propose a self-destruction system, named CloudSky, which is able to enforce the security of user privacy over the untrusted cloud in a controllable way. CloudSky exploits a key control mechanism based on the attribute-based encryption (ABE) and takes advantage of active storage networks to allow the user to control the subjective life-cycle and the access control polices of the private data whose integrity is ensured by using HMAC to cope with untrusted environments. %and thereby adapting it to the cloud in terms of both performance and security requirements. The feasibility of the system in terms of its performance and scalability is demonstrated by experiments on a real large-scale storage network.

Base station (BS) location privacy has been widely studied and researched in different applications like field monitoring, agriculture, industry and military etc. The purpose is to hide the location of BS from outside/inside attacker in any shape. Hundreds of thousands of sensor nodes are deployed in some area, bring lot of new challenges regarding routing, forwarding, scaling and security. Different approaches are there to provide location privacy in wireless sensor networks (WSN). The work done by researchers is to provide some form of BS location privacy either different from others or boosting them more. In our scheme, we proposed a new algorithm MimiBS “Mimicking Base-Station”. All the deployed aggregator nodes (ANs) in the field/area will be looking like BS, even if the attacker knows about any AN, he will be deceived between the real BS and AN. We approached different schemes for our proposed algorithm mentioning routing without fake packets, with fake packets, without energy consideration and with energy consideration. These different parameters show an improvement over previous work regarding the same problem.

It has long been recognized that the Credit scheduler selectively favors CPU-bound applications whereas for I/O-latency sensitive workloads, such as those related to stream-based audio/video services, it only exhibits tolerable, or even worse, unacceptable performance. The reasons behind this phenomenon are the poor understanding (to some degree) of the virtual machine scheduling as well as the network I/O virtualizations. In order to address these problems and make the system more responsive to the I/O-latency sensitive applications, in this paper, we present XCollOpts which performs a collection of novel optimizations to improve the Credit scheduler and the underlying I/O virtualizations in multicore environments, each from two perspectives. To optimize the schedule, in XCollOpts, we first pinpoint the Imbalanced Multi-Boosting problem among the cores thereby minimizing the system response time by load balancing the BOOST VCPUs. Then, we describe the Premature Preemption problem and address it by monitoring the received network packets in the driver domain and deliberately preventing it from being prematurely preempted during the packet delivery. However, these optimizations on the scheduling strategies cannot be fully exploited if the performance issues of the underlying supportive communication mechanisms are not considered. To this end, we make two further optimizations for the network I/O virtualizations, namely, Multi-Tasklet Pairs and Optimized Small Data Packet. Our empirical studies show that with XCollOpts, we can significantly improve the performance of the latency-sensitive applications at a cost of relatively small system overhead.

This paper proposes a replication cost model and two greedy algorithms, named GS QoS and GS QoS C1, for replication placements in cloud-based storage systems. The model aims to minimize replication cost with full consideration of quality of user access to storage nodes. Our two algorithms employ a utility measurement to guide placement procedures. Our final experimental results show that 1) GS QoS outperforms GS QoS C1, 2) both algorithms have more economical results than those from existing greedy replica placement algorithm.

This paper is motivated by our three key observations: (1) there exists a degradation of performance as the interleaved accesses of heterogeneous streams, (2) for the slow stream, sequential accesses suffer huge misses in the prefetching cache, (3) in concurrence paradigm, providing fairness and QoS to concurrent streams is very important which always ignored by the traditional prefetching algorithms. Therefore, we present Fema, a caching management algorithm that enforces the fairness and efficiency for concurrent heterogeneous streams. Fema focuses on three key designs: (1) An adaptive framework (Fema Ada) for prefetching. In the Fema Ada, we propose a rate-aware adjustment of prefetching degree and analysis the optimal partition size. (2) A novel replacement scheme (Fema Rep) in which the accessed data will be firstly evicted to improve the performance. (3) A round robin allocation scheme (Fema Rou) to achieve fairness while as least performance degradation as possible. Results show that Fema is able to achieve averages 81.4% performance improvement over the LRU algorithm, 53.5% over the default Linux Kernel prefetching (LKP) algorithm and 19.0% over the recently proposed practical AMP (adaptive multi-stream prefetching) algorithm. Fema achieves average 74.2% fairness improvement (metric in fair speedup) over the LKP algorithm and 56.5% over the AMP algorithm.

Traditionally, the "best effort, cost free" model of Supercomputers/Grids does not consider pricing. Clouds have progressed towards a service-oriented paradigm that enables a new way of service provisioning based on "pay-as-you-go" model. Large scale many-task workflow (MTW) may be suited for execution on Clouds due to its scale-* requirement (scale up, scale out, and scale down). In the context of scheduling, MTW execution cost must be considered based on users' budget constraints. In this paper, we address the problem of scheduling MTW on Clouds and present a budget-conscious scheduling algorithm, referred to as ScaleStar (or Scale-*). ScaleStar assigns the selected task to a virtual machine with higher comparative advantage which effectively balances the execution time-and-monetary cost goals. In addition, according to the actual charging model, an adjustment policy, refer to as DeSlack, is proposed to remove part of slack without adversely affecting the overall makespan and the total monetary cost. We evaluate ScaleStar with an extensive set of simulations and compare with the most popular HEFT-based LOSS3 algorithm and demonstrate the superior performance of ScaleStar.

Both physical disk failure and logical errors such as software error, user abuse and virus attacks may cause data lose. The risk of logical errors is far greater than physical disk failure. Moreover, existing RAID solution cannot satisfy the reliability requirement in face of the logical errors in data centers. It is therefore becoming increasingly important for RAID-based storage systems to be able to recover data to any point-in-time when logical errors occur. We proposed a novel storage array architecture, Herp Rap, which is able to recover data from both physical disk failure and logical errors. We have implemented a prototype of Herp Rap and carried out extensive performance measurements using DBT-2 and file system benchmarks. Our experiments demonstrated that the proposed Herp Rap is able to track or recover data to any point-in-time quickly by tracing back the history of block logs. Moreover, Herp Rap outperforms existing HDD-based or SSD-based RAID5 with copy-on-write (COW) snapshot in terms of performance, energy efficiency, failure recovery ability and reliability.

The RAID6 provides high reliability using double-parity-update at cost of high write penalty. In this paper, we propose HRAID6ML, a new logging architecture for RAID6 systems for enhanced energy efficiency, performance and reliability. HRAID6ML explores a group of Solid State Drives (SSDs) and Hard Disk Drives (HDDs): Two HDDs (parity disks) and several SSDs form RAID6. The free space of the two parity disks is used as mirrored log region of the whole system to absorb writes. The mirrored logging policy helps to recover system from parity disk failure. Mirrored logging operation does not introduce noticeable performance overhead to the whole system. HRAID6ML eliminates the additional hardware and energy costs, potential single point of failure and performance bottleneck. Furthermore, HRAID6ML prolongs the lifecycle of the SSDs and improves the systems energy efficiency by reducing the SSDs write frequency. We have implemented proposed HRAID6ML. Extensive trace-driven evaluations demonstrate the advantages of the HRAID6ML system over both traditional SSD-based RAID6 system and HDD-based RAID6 system.

Flash-based Solid State Drive (SSD) has been productively shipped and deployed in large scale storage systems. However, a single flash-based SSD cannot satisfy the capacity, performance and reliability requirements of the modern storage systems that support increasingly demanding data-intensive computing applications. Applying RAID schemes to SSDs to meet these requirements, while a logical and viable solution, faces many challenges. In this article, we propose a Hybrid Parity-based Disk Array architecture (short for HPDA), which combines a group of SSDs and two hard disk drives (HDDs) to improve the performance and reliability of SSD-based storage systems. In HPDA, the SSDs (data disks) and part of one HDD (parity disk) compose a RAID4 disk array. Meanwhile, a second HDD and the free space of the parity disk are mirrored to form a RAID1-style write buffer that temporarily absorbs the small write requests and acts as a surrogate set during recovery when a disk fails. The write data is reclaimed to the data disks during the lightly loaded or idle periods of the system. Reliability analysis shows that the reliability of HPDA, in terms of MTTDL (Mean Time To Data Loss), is better than that of either pure HDD-based or SSD-based disk array. Our prototype implementation of HPDA and the performance evaluations show that HPDA significantly outperforms either HDD-based or SSD-based disk array.

Current FTL schemes have inevitable limitations in terms of memory requirement, performance, garbage collection overhead, and scalability. To overcome these limitations, we propose a workload adaptive flash translation layer referred to as WAFTL. WAFTL explores either page-level or block-level address mapping for normal data block based on access patterns. Page Mapping Block (PMB) is used to store random data and handle large number of partial updates. Block Mapping Block (BMB) is utilized to store sequential data and lower overall mapping table. PMB or BMB is allocated on demand and the number of PMB or BMB eventually depends on workload. An efficient address mapping is designed to reduce overall mapping table and quickly conduct address translation. WAFTL explores a small part of flash space as Buffer Zone to log writes sequentially and migrate data into BMB or PMB based on threshold. Static and dynamic threshold setting are proposed to balance performance and mapping table size. WAFTL has been extensively evaluated under various enterprise workloads. Benchmark results conclusively demonstrate that proposed WAFTL is workload adaptive and achieves up to 80% performance improvement, 83% garbage collection overhead reduction and 50% mapping table reduction compared to existing FTL schemes.

In the background of cloud, self-destructing data mainly aims at protecting the data privacy. All the data and its copies will become destructed or unreadable after a user-specified period, without any user intervention. Besides, anyone cannot get the decryption key after timeout, neither the sender nor the receiver. The Washington's Vanish system is a system for self-destructing data under cloud computing, and it is vulnerable to “hopping attack” and “sniffer attack”. We propose a new scheme in this paper, called Safe Vanish, to prevent hopping attacks by way of extending the length range of the key shares to increase the attack cost substantially, and do some improvement on the Shamir Secret Sharing algorithm implemented in the Original Vanish system. We present an improved approach against sniffing attacks by using the public key cryptosystem to protectt from sniffing operations. In addition, we evaluate analytically the functionality of the proposed Safe Vanish system.

Data replication has been widely used as a mean of increasing the data availability of large-scale cloud storage systems where failures are normal. Aiming to provide cost-effective availability, and improve performance and load-balancing of cloud storage, this paper presents a cost-effective dynamic replication management scheme referred to as CDRM. A novel model is proposed to capture the relationship between availability and replica number. CDRM leverages this model to calculate and maintain minimal replica number for a given availability requirement. Replica placement is based on capacity and blocking probability of data nodes. By adjusting replica number and location according to workload changing and node capacity, CDRM can dynamically redistribute workloads among data nodes in the heterogeneous cloud. We implemented CDRM in Hadoop Distributed File System (HDFS) and experiment results conclusively demonstrate that our CDRM is cost effective and outperforms default replication management of HDFS in terms of performance and load balancing for large-scale cloud storage.

A single flash-based Solid State Drive (SSD) can not satisfy the capacity, performance and reliability requirements of a modern storage system supporting increasingly demanding data-intensive computing applications. Applying RAID schemes to SSDs to meet these requirements, while a logical and viable solution, faces many challenges. In this paper, we propose a Hybrid Parity-based Disk Array architecture, HPDA, which combines a group of SSDs and two hard disk drives (HDDs) to improve the performance and reliability of SSD-based storage systems. In HPDA, the SSDs (data disks) and part of one HDD (parity disk) compose a RAID4 disk array. Meanwhile, a second HDD and the free space of the parity disk are mirrored to form a RAID1-style write buffer that temporarily absorbs the small write requests and acts as a surrogate set during recovery when a disk fails. The write data is reclaimed back to the data disks during the lightly loaded or idle periods of the system. Reliability analysis shows that the reliability of HPDA, in terms of MTTDL (Mean Time To Data Loss), is better than that of either pure HDD-based or SSD-based disk array. Our prototype implementation of HPDA and performance evaluations show that HPDA significantly outperforms either HDD-based or SSD-based disk array.

With data amount growing at an ever increasing rate, file system, as the basis of data management, its functionality and content are becoming more and more complex. Thus the file system is forced to be upgraded constantly, in order to satisfy the growing demands. However, the reinvention of new file systems is always a high-cost task. Although there are frameworks such as User-space File System that may help speeding up the development process, ease the pain of deploying new functions, they also bring in significant overhead, especially when file operation frequency is fairly high. We designed and implemented a lightweight file system management framework based on LD_PRELOAD in user-space referred to as User-Space Preload (USP), and tested several management tasks with it. According to the comparison with FUSE, USP-based file system management achieved an obvious performance improvement.

This paper proposes a simple and practical RAID reconstruction optimization scheme, called JOurnal-guided Reconstruction (JOR). JOR exploits the fact that significant portions of data blocks in typical disk arrays are unused. JOR monitors the storage space utilization status at the block level to guide the reconstruction process so that only failed data on the used stripes is recovered to the spare disk. In JOR, data consistency is ensured by the requirement that all blocks in a disk array be initialized to zero (written with value zero) during synchronization while all blocks in the spare disk also be initialized to zero in the background. JOR can be easily incorporated into any existing reconstruction approach to optimize it, because the former is independent of and orthogonal to the latter. Experimental results obtained from our JOR prototype implementation demonstrate that JOR reduces reconstruction times of two state-of-the-art reconstruction schemes by an amount that is approximately proportional to the percentage of unused storage space while ensuring data consistency.

With our society more information-driven, we have begun to distribute data in wide-area storage systems. At the same time, both physical failure and logic error have made it difficult to bring the necessary recovery to bear on remote data disaster, and understanding this proceeding. We describe ARRAY, a system architecture for data disaster recovery that combines reliability, storage space, and security to improve performance for data recovery applications. The paper presents an exhaustive analysis of the design space of ARRAY systems, focusing on the trade-offs between reliability, storage space, security, and performance that ARRAY must make. We present RSRAII (Replication-based Snapshot Redundant Array of Independent Imagefiles) which is a configurable RAID-like data erasure-coding, and also others benefits come from consolidation both erasure-coding and replication strategies. A novel algorithm is proposed to improve snapshot performance referred to as SMPDP (Snapshot based on Multi-Parallel Degree Pipeline).

ISCSI has been paid much attention since it allows the storage data to be transported over the popular TCP/IP networks and takes advantage of the networks. Carrying data over the TCP/IP networks introduces the security problem. The IP security mechanisms, such as IPSec, always degrade the performance greatly. In this paper we introduced a parallel encryption method, combined with the IPSec A,H to provide the security and high performance for the iSCSI storage system. We have implemented parallel encryption with IPSec AH in the UNH iSCSI software. Numerical results using popular benchmark program and OLTP traces have shown dramatic performance gain and the average response time reduced greatly.

Object-based storage is emerging as a next generation of distributed storage technology. Aiming at improving the performance and load-balancing of large-scale object-based storage system, we present a dynamic weight-based cooperative caching scheme referred to as DWC2, which allows an object-based storage device (OSD) to use the available free cache of the neighbouring OSD. Our proposed DWC2replaces objects based on their weights which is a function of object size, popularity and replica number, and dynamically partitions the memory of OSD into local cache and remote cache according to activity workload. An object data is cached in local cache or remote cache of the cooperative OSDs, thus increasing cache hit ratio, reducing expensive disk access time as well as improving load balance. We benchmarked our proposed DWC2with existing cooperative caching schemes under various OSD environments. Our rigorous experiment results conclusively demonstrate that our DWC2is scalable and can achieve a significant cache hit ratio, average response time, and load balancing for large-scale OSD cluster.

avoid re-encryption in cryptographic storage system when revoking users, Field Programmable Gate Array (FPGA) and Application Specific Integrated Circuit (ASIC) hardware module have been introduced to a cryptographic object store system, let private key never leave the hardware module and symmetric key only exist in hardware module in plaintext. Anyone doesn 7 know private or symmetric key, so when revoking users, it just needs to modify access control list (A CL) to delete the privileges of the users. To facilitate file sharing and key management, group is adopted. In our system, almost all computationally expensive cryptographic operations are through FPGA/ASIC hardware module. Once creator revokes some users, objects don't need re- encryption. How to use ACL and FPGA/ASIC hardware module to authenticate and authorize are described. And the procedure of object store and the distribution of meta-data are detailed. Finally, a cryptographic object store prototype system is implemented with tested and effective performance.

Geographic Information System Grid integrates geographic information systems and Grid technology for data gathering, accessing, transmitting and service, in different I/O patterns, built upon massive storage systems. Existing non-standardized multi-source and multi-scale data lack spatial information sharing either internally or externally between organizations or departments, especially in national or global applications. HUSt is a massive storage system that was built at Wuhan National Laboratory for Optoelectronics, in China. There are heterogeneous storage areas in the system, including Object-based Storage System for the main data storing especially for the data searched frequently, Virtual Interface based Storage System for the data required at high transfer speed, and InfiniBand based SAN for high performance. HUSt is primarily meant for research on the organization and key technologies of storage systems for the next generation Internet. The goal is to unify network storage and construct a peta-byte storage system, which supports GIS Grid and applications.

Traditional storage systems, such as NAS, SAN, are largely unaware of the users and applications actually using the storage, because block-based storage devices manage opaque data blocks. But, with OBSS (object-based storage system), the attributes and methods among the storage devices can be adopted in the storage system, the data can be distributed on some of the storage devices and organized better to anticipate users demand. In this paper, the scalability of object storage (including object attributes, object methods and OBSS) is studied. And a self-managing approach, denoted adaptive policy trigger mechanism (APTM), is presented. APTM borrows proven machine learning techniques and takes the perspective scalable object storage. The implementation reveals that APTM is the embodiment of the idea about smart storage device and facilitates to self-manage mass storage system.