As services become more complex with multiple interactions, and storage servers are shared by multiple services, the different I/O streams arising from these multiple services compete for disk attention. Aggressive Pipelined Prefetching (APP) enabled storage clients are designed to manage the buffer cache and I/O streams to minimize the disk I/O-interference arising from competing streams. Due to the large number of streams serviced by a storage server, most of the disk time is spent seeking, leading to degradation in response times. The goal of APP is to decrease application execution time by increasing the throughput of individual I/O streams and utilizing idle capacity on remote nodes along with idle network times thus effectively avoiding alternating bursts of activity followed by periods of inactivity. APP significantly increases overall I/O throughput and decreases overall messaging overhead between servers. In APP, the intelligence is embedded in the clients and they automatically infer parameters in order to achieve the maximum throughput. APP clients make use of aggressive prefetching and data offloading to remote buffer caches in multi-level buffer cache hierarchies in an effort to minimize disk interference and tranquilize the effects of aggressive prefetching. We used an extremely I/O-intensive Radix-k application employed in studies on the scalability of parallel image composition and particle tracing developed at the Argonne National Laboratory with data sets of up to 128 GB and implemented our scheme on a 16-node Linux cluster. We observed that the execution time of the application decreased by 68% on average when using our scheme.

While network bandwidth is steadily increasing, it is doing so at a much slower rate than the corresponding increase in CPU performance. This trend has widened the gap between CPU and network speed. In this paper, we investigate improvements to I/O performance by exploiting this gap. We harness idle CPU resources to compress network traffic, reducing the amount of data transferred over the network and increasing effective network bandwidth. We created a set of compression services within the I/O Forwarding Scalability Layer. These services transparently compress and decompress data as it is transferred over the network. We studied the effect of the compression services on a variety of data sets and conducted experiments on a high-performance computing cluster.

In this paper, we study the impact of cache sharing on co-mapped applications in multi-level buffer cache hierarchies. When the number of applications exceeds the number of resources, resource sharing is inevitable. However, unless applications are co-mapped carefully, destructive interference may cause applications to thrash and spend most of their time paging data to and from disks. We propose two novel models which predict the performance of an application in the presence of other applications and an algorithm which uses the output of these models to perform application-to-node mapping in a multi-level buffer cache hierarchy. Our models use the reuse distances of the application reference streams and their respective I/O rates. This information can be obtained either online or offline. Our main advantage is that we do not require profile information of all application pairs to predict their interferences. The goal of this mapping is to minimize destructive interference during execution. We validate the effectiveness of our models and mapping scheme using several I/O-intensive applications, and found that the error in prediction of our two models is only 3.9% and 2.7% respectively, on average. Further, using our approach, we were effectively able to co-map applications to maximize the performance of the buffer cache hierarchy by 43.6% and 56.8% on average over the median and worst mappings respectively in the entire I/O stack.

In this work, we propose, implement and test a novel approach to the management of parallel I/O in high-performance computing. Our proposed approach is built upon three complementary ideas: (i) allowing users to place hints into the application code indicating high-level data access patterns, (ii) enabling an optimizing compiler to process these hints and develop I/O optimization strategies, and (iii) enhancing the I/O stack to accept these optimizations and process them across the different layers in the stack. We describe a general hint processing framework that accommodates this approach and demonstrate its potential by applying it to two sample problems: (i) shared storage cache management and (ii) I/O prefetching. In the former, our approach decides, at each program point of interest, the ideal set of data blocks to keep in shared storage caches in the I/O stack, and in the latter, the high-level data access pattern is propagated from application layer to the parallel file system layer for prefetching data from the storage subsystem. Our approach is designed to complement and work synergistically with the MPI-IO and PVFS frameworks and exploits the characteristics of applications written using these software. We tested our approach using both synthetic data access patterns and disk I/O intensive application programs. The results collected indicate that the proposed approach improves over existing storage caching and I/O prefetching schemes by 28% and 66%, respectively.

Increasing complexity of large-scale applications and continuous increases in data set sizes of such applications combined with slow improvements in disk access latencies has resulted in I/O becoming a performance bottleneck. While there are several ways of improving I/O access latencies of dataintensive applications, one of the promising approaches has been using different layers of the I/O subsystem to cache recently and/or frequently used data so that the number of I/O requests accessing the disk is reduced. These different layers of caches across the storage hierarchy introduce the need for efficient cache management schemes to derive maximum performance benefits. Several state-of-the-art multi-level storage cache management schemes focus on optimizing aggregate hit rate or overall I/O latency, while being agnostic to Service Level Objectives (SLOs). Also, most of the existing works focus on different cache replacement algorithms for managing storage caches and discuss different exclusive caching techniques in the context of multilevel cache hierarchy. However, the orthogonal problem of storage cache space allocation to multiple, simultaneously-running applications in a multi-level hierarchy of storage caches with multiple storage servers has remained an open research problem. In this work, using a combination of per-application latency model and a linear programming model, we proportion storage caches dynamically among multiple concurrently-executing applications across the different levels of the storage hierarchy and across multiple servers to provide isolation to applications while satisfying the application level SLOs. Further, our algorithm improves the overall system performance significantly.

Applications performing scientific computations or processing streaming media benefit from parallel I/O significantly, as they operate on large data sets that require large I/O. MPI-I/O is a commonly used library interface in parallel applications to perform I/O efficiently. Optimizations like collective-I/O embedded in MPI-I/O allow multiple processes executing in parallel to perform I/O by merging requests of other processes and sharing them later. In such a scenario, preserving confidentiality of disk-resident data from unauthorized accesses by processes without significantly impacting performance of the application is a challenging task. In this paper, we evaluate the impact of ensuring data-confidentiality in MPI-I/O on the performance of parallel applications and provide an enhanced interface, called MPISec I/O, which brings an average overhead of only 5.77% over MPI-I/O in the best case, and about 7.82% in the average case.

Resources are often shared to improve resource utilization and reduce costs. However, not all resources exhibit good performance when shared among multiple applications. The work presented here focuses on effectively managing a shared storage cache. To provide differentiated services to applications exercising a storage cache, we propose a novel scheme that uses curve fitting to dynamically partition the storage cache. Our scheme quickly adapts to application execution, showing increasing accuracy over time. It satisfies application QoS if it is possible to do so, maximizes the individual hit rates of the applications utilizing the cache, and consequently increases the overall storage cache hit rate. Through extensive trace-driven simulation, we show that our storage cache partitioning strategy not only effectively insulates multiple applications from one another but also provides QoS guarantees to applications over a long period of execution time. Using our partitioning strategy, we were able to increase the individual storage cache hit rates of the applications by 67% and 53% over the no-partitioning and equal-partitioning schemes, respectively. Additionally, we improved the overall cache hit rates of the entire storage system by 11% and 12.9% over the no-partitioning and equal-partitioning schemes, respectively, while meeting the QoS goals all the time.

We introduce a dynamic and efficient shared cache management scheme, called Maxperf, that manages the aggregate cache space in multi-server storage architectures such that the service level objectives (SLOs) of concurrently executing applications are satisfied and any spare cache capacity is proportionately allocated according to the marginal gains of the applications to maximize performance. We use a combination of Neville's algorithm and linear-programming-model to discover the required storage cache partition size, on each server, for every application accessing that server. Experimental results show that our algorithm enforces partitions to provide stronger isolation to applications, meets application level SLOs even in the presence of dynamically changing storage cache requirements, and improves I/O latency of individual applications as well as the overall I/O latency significantly compared to two alternate storage cache management schemes, and a state-of-the-art single server storage cache management scheme extended to multi-server architecture.

No abstract available.