Data storage is an integral part of IT infrastructures, where Quality of Service (QoS) differentiation amongst customers and their applications is essential for many. Achieving this objective in a production environment is nontrivial, because these environments are complex and dynamic. Numerous practical and engineering constraints render the task even more challenging. This paper presents SLED-2, a QoS differentiation solution that meets these challenges in offering effective protection to the performance of important workloads at the expense of less important workloads when needed. SLED-2 uses a customized feedback heuristic that rate-limits selected I/O streams. This approach is unique in that it accounts for a number of important practical considerations, including fine-grained controls, errors in storage systems models, and inexpensive and safe QoS management. SLED-2 has been implemented for the IBM DS8000 series storage servers and shown to be highly effective in a set of hostile and practical scenarios using test facilities for IBM storage products.

MapReduce has emerged as a model of choice for supporting modern data-intensive applications. The model is easy-to-use and promising in reducing time-to-solution. It is also a key enabler for cloud computing, which provides transparent and flexible access to a large number of compute, storage and networking resources. Setting up and operating a large MapReduce cluster entails careful evaluation of various design choices and run-time parameters to achieve high efficiency. However, this design space has not been explored in detail. In this paper, we adopt a simulation approach to systematically understanding the performance of MapReduce setups. The resulting simulator, MRPerf, captures such aspects of these setups as node, rack and network configurations, disk parameters and performance, data layout and application I/O characteristics, among others, and uses this information to predict expected application performance. Specifically, we use MRPerf to explore the effect of several component inter-connect topologies, data locality, and software and hardware failures on overall application performance. MRPerf allows us to quantify the effect of these factors, and thus can serve as a tool for optimizing existing MapReduce setups as well as designing new ones.

Storage architecture includes more and more processing power for increasing requirement of reliability, managibility and scalability. For example, an IBM storage server is equipped with 4 or 8 state-of-the-art processors and gigabytes of memories. This trend enables analyzing data locally inside a storage server. Processing data locally is appealing under the following circumstances: (1) huge reduction of data flowing to the host, (2) reduction of CPU consumption on host. Accordingly, the benefits are (1) less data traffic through IO channel to the host, (2) better utilization of host bufferpool, and (3) enabling more workload on the host. One crucial task is to understand how DBMS can benefit from such hardware. That is to identify which database operations are beneficial to be offloaded given a query workload in a particular setting. For certain operations, we establish value proposition via various approaches and show the analytical and experimental results. In particular, starjoin queries are commonly used in business warehouses. We propose to offload a portion of a starjoin query from host to the POWER5 P processors on a storage server, which dramatically reduces the amount of channel IO and host CPU consumption. Moreover, the query elapsed time is improved via the exploitation of the state-of-the-art P processors on a storage server.

Abstract There are many group communication systems (GCSs) that provide consistent group membership and reliable, ordered multicast properties in the presence of crash faults. However, relatively few GCS implementations are able to provide these properties in the presence of malicious faults resulting from intrusions. We describe the systematic transformation of a crash‐tolerant GCS, namely C‐Ensemble, into an intrusion‐tolerant GCS, the ITUA GCS. To perform the transformation, we devised intrusion‐tolerant versions of key group communication protocols. We then inserted implementations of the protocols into C‐Ensemble and made significant changes to the rest of the C‐Ensemble protocol stack to make the stack intrusion tolerant. We quantify the cost of providing intrusion‐tolerant group communication in two ways. First, we quantify the implementation effort by presenting a detailed analysis of the amount of change required to the original C‐Ensemble system. In doing so, we provide insight into the choice of building an intrusion‐tolerant GCS from scratch versus building one by leveraging a crash‐tolerant implementation. Second, we quantify the run‐time performance cost of tolerating intrusions by presenting results from an experimental evaluation of the main intrusion‐tolerant microprotocols. The results are analyzed to identify the parts that contribute the most overhead while providing intrusion tolerance during both normal operation and recovery from intrusions. Copyright © 2007 John Wiley & Sons, Ltd.

Current data centers require storage capacities of hundreds of terabytes to petabytes. Time-critical applications such as online transactions processing depend on getting adequate performance from the storage subsystem: otherwise, they fail. It is difficult to provide predictable quality of service at this level of complexity, because I/O workloads are extremely variable and device behavior is poorly understood. Ensuring that unrelated but competing workloads do not affect each other's performance is still more difficult, and equally necessary. We present SLEDS (Service Level Enforcement Discipline for Storage), a distributed controller that provides statistical performance guarantees on a storage system built from commodity components. SLEDS can adaptively handle unpredictable workload variations so that each client continues to get the performance it needs even in the presence of misbehaving, competing peers. After evaluating the SLEDS on a heterogeneous mid-range storage system, we found that it is vastly superior to the raw system in its ability to provide performance guarantees, while only introducing a negligible overhead.

Group communication systems that provide consistent group membership and reliable, ordered multicast properties in the presence of faults resulting from malicious intrusions have not been analyzed extensively to quantify the cost of tolerating these intrusions. This paper attempts to quantify this cost by presenting results from an experimental evaluation of three new intrusion-tolerant microprotocols that have been added to an existing crash-fault-tolerant group communication system. The results are analyzed to identify the parts that contribute the most overhead during provision of intrusion tolerance at the group communication system level.