Public and private cloud computing environments employ virtualization methods to consolidate application workloads onto shared servers. Modern servers typically have one or more sockets each with one or more computing cores, a multi-level caching hierarchy, a memory subsystem, and an interconnect to the memory of other sockets. While resource management methods may manage application performance by controlling the sharing of processing time and input-output rates, there is generally no management of contention for virtualization kernel resources or for the memory hierarchy and subsystems. Yet such contention can have a significant impact on application performance. Hardware platform specific counters have been proposed for detecting such contention. We show that such counters alone are not always sufficient for detecting contention. We propose a software probe based approach for detecting contention for shared platform resources and demonstrate its effectiveness. We show that the probe imposes low overhead and is remarkably effective at detecting performance degradations due to inter-VM interference over a wide variety of workload scenarios and on two different server architectures. The probe successfully detected virtualization-induced software bottleneck and memory contention on both server architectures. Our approach supports the management of workload placement on shared servers and pools of shared servers.

Cloud computing has emerged as a new and alternative approach for providing computing services. Customers acquire and release resources by requesting and returning virtual machines to the cloud. Different service models and pricing schemes are offered by cloud service providers. This can make it difficult for customers to compare cloud services and select an appropriate solution. Cloud Infrastructure-as-a-Service vendors offer a t-shirt approach for Virtual Machines (VMs) on demand. Customers can select from a set of fixed size VMs and vary the number of VMs as their demands change. Private clouds often offer another alternative, called time-sharing, where the capacity of each VM is permitted to change dynamically. With this approach each virtual machine is allocated a dynamic amount of CPU and memory resources over time to better utilize available resources. We present a tool that can help customers make informed decisions about which approach works most efficiently for their workloads in aggregate and for each workload separately. A case study using data from an enterprise customer with 312 workloads demonstrates the use of the tool. It shows that for the given set of workloads the t-shirt model requires almost twice the number of physical servers as the time share model. The costs for such infrastructure must ultimately be passed on to the customer in terms of monetary costs or performance risks. We conclude that private and public clouds should consider offering both resource sharing models to meet the needs of customers.

Engaging patients in the management of their health care can improve the quality of their care and enhance their experience while making more efficient use of care provider resources, especially for chronic diseases. However, health care system complexity and the challenge of consumer health literac

The World Wide Web and the services it provides are continually evolving. Even for a single time instant, it is a complex task to methodologically determine the infrastructure over which these services are provided and the corresponding effect on user perceived performance. For such tasks, researchers typically rely on active measurements or large numbers of volunteer users. In this paper, we consider an alternative approach, which we refer to as passive crowd-based monitoring. More specifically, we use passively collected proxy logs from a global enterprise to observe differences in the quality of service (QoS) experienced by users on different continents. We also show how this technique can measure properties of the underlying infrastructures of different Web content providers. While some of these properties have been observed using active measurements, we are the first to show that many of these properties (such as location of servers) can be obtained using passive measurements of actual user activity. Passive crowd-based monitoring has the advantages that it does not add any overhead on Web infrastructure, it does not require any specific software on the clients, but still captures the performance and infrastructure observed by actual Web usage.

Public and private clouds are being adopted as a cost-effective approach for sharing IT resources. Customers acquire and release resources by requesting and returning virtual machines to the cloud. Different service models are proposed for virtual machine resource management. Some public cloud providers follow a t-shirt model for VM resource sizing. A second approach for resource management is based on a time share model. This paper compares the two approaches from the perspective of resource usage for both the service provider and workload owner. Using data from 312 customer applications, we show that the t-shirt model requires 40% more infrastructure than when a finer degree of resource sharing based on time varying resource shares is permitted.

We present an automatic skew mitigation approach for user-defined MapReduce programs and present SkewTune, a system that implements this approach as a drop-in replacement for an existing MapReduce implementation. There are three key challenges: (a) require no extra input from the user yet work for all MapReduce applications, (b) be completely transparent, and (c) impose minimal overhead if there is no skew. The SkewTune approach addresses these challenges and works as follows: When a node in the cluster becomes idle, SkewTune identifies the task with the greatest expected remaining processing time. The unprocessed input data of this straggling task is then proactively repartitioned in a way that fully utilizes the nodes in the cluster and preserves the ordering of the input data so that the original output can be reconstructed by concatenation. We implement SkewTune as an extension to Hadoop and evaluate its effectiveness using several real applications. The results show that SkewTune can significantly reduce job runtime in the presence of skew and adds little to no overhead in the absence of skew.

We demonstrate SkewTune, a system that automatically mitigates skew in user-defined MapReduce programs and is a drop-in replacement for Hadoop. The demonstration has two parts. First, we demonstrate how SkewTune mitigates skew in real MapReduce applications at runtime by running a real application in a public cloud. Second, through an interactive graphical interface, we demonstrate the details of the skew mitigation process using both real and synthetic workloads that represent various skew configurations.

We introduce BURN, a methodology to create customized benchmarks for testing multitier applications under time-varying resource usage conditions. Starting from a set of preexisting test workloads, BURN finds a policy that interleaves their execution to stress the multitier application and generate controlled burstiness in resource consumption. This is useful to study, in a controlled way, the robustness of software services to sudden changes in the workload characteristics and in the usage levels of the resources. The problem is tackled by a model-based technique which first generates Markov models to describe resource consumption patterns of each test workload. Then, a policy is generated using an optimization program which sets as constraints a target request mix and user-specified levels of burstiness at the different resources in the system. Burstiness is quantified using a novel metric called overdemand, which describes in a natural way the tendency of a workload to keep a resource congested for long periods of time and across multiple requests. A case study based on a three-tier application testbed shows that our method is able to control and predict burstiness for session service demands at a fine-grained scale. Furthermore, experiments demonstrate that for any given request mix our approach can expose latency and throughput degradations not found with nonbursty workloads having the same request mix.

We present a new technique for predicting the resource demand requirements of services implemented by multitier systems. Accurate demand estimates are essential to ensure the efficient provisioning of services in an increasingly service-oriented world. The demand estimation technique proposed in this paper has several advantages compared with regression-based demand estimation techniques, which many practitioners employ today. In contrast to regression, it does not suffer from the problem of multicollinearity, it provides more reliable aggregate resource demand and confidence interval predictions, and it offers a measurement-based validation test. The technique can be used to support system sizing and capacity planning exercises, costing and pricing exercises, and to predict the impact of changes to a service upon different service customers.

Virtualization offers the potential for cost-effective service provisioning. For service providers who make significant investments in new virtualized data centers in support of private or public clouds, one of the serious challenges is the problem of recovering costs for new server hardware, software, network, storage, management, etc. Gaining visibility and accurately determining the cost of shared resources used by collocated services is essential for implementing a proper chargeback approach in cloud environments. We introduce and compare three different models for apportioning cost and champion the one that is least sensitive to workload placement decisions and provides the most robust and repeatable cost estimates. A detailed study involving 312 workloads from an HP customer environment demonstrates the result. Finally, we employ the cost model in a case study that evaluates the impact on the cost of exploiting different virtualization platform alternatives for the 312 workloads. For example, some workloads may cost more to host using certain virtualization platforms than on others or on standalone hosts. We demonstrate different decision points with potential cost savings of nearly 20% by “right-virtualizing” the workloads.

This paper presents three methods for apportioning server costs among workloads in shared resource environments such as computing clouds. We consider a fine sharing of resources, the impact of time varying resource usage, large ratios for peak to mean workload demands, and the influence of random choices for the co-placement of workloads on shared servers. These features can affect the quantity of servers needed to support workloads as well as the robustness of the cost values assigned to each workload. We compare the three methods for apportioning costs and recommend the method that assigns costs in the most repeatable manner. A detailed study involving 312 workloads from an HP customer environment demonstrates the result.

Predictive performance models are important tools that support system sizing, capacity planning, and systems management exercises. We introduce the Weighted Average Method (WAM) to improve the accuracy of analytic predictive performance models for systems with bursts of concurrent customers. WAM considers the customer population distribution at a system to reflect the impact of bursts. The WAM approach is robust with respect to distribution functions, including heavy-tail-like distributions, for workload parameters. We demonstrate the effectiveness of WAM using a case study involving a multitier TPC-W benchmark system. To demonstrate the utility of WAM with multiple performance modeling approaches, we developed both Queuing Network Models and Layered Queuing Models for the system. Results indicate that WAM improves prediction accuracy for bursty workloads for QNMs and LQMs by 10 and 12 percent, respectively, with respect to a Markov Chain approach reported in the literature.

Scientists today have the ability to generate data at an unprecedented scale and rate and, as a result, they must increasingly turn to parallel data processing engines to perform their analyses. However, the simple execution model of these engines can make it difficult to implement efficient algorithms for scientific analytics. In particular, many scientific analytics require the extraction of features from data represented as either a multidimensional array or points in a multidimensional space. These applications exhibit significant computational skew, where the runtime of different partitions depends on more than just input size and can therefore vary dramatically and unpredictably. In this paper, we present SkewReduce, a new system implemented on top of Hadoop that enables users to easily express feature extraction analyses and execute them efficiently. At the heart of the SkewReduce system is an optimizer, parameterized by user-defined cost functions, that determines how best to partition the input data to minimize computational skew. Experiments on real data from two different science domains demonstrate that our approach can improve execution times by a factor of up to 8 compared to a naive implementation.

This paper describes an approach for designing a power management plan that matches the supply of power with the demand for power in data centers. Power may come from the grid, from local renewable sources, and possibly from energy storage subsystems. The supply of renewable power is often time-varying in a manner that depends on the source that provides the power, the location of power generators, and the weather conditions. The demand for power is mainly determined by the time-varying workloads hosted in the data center and the power management policies implemented by the data center. A case study demonstrates how our approach can be used to design a plan for realistic and complex data center workloads. The study considers a data center's deployment in two geographic locations with different supplies of power. Our approach offers greater precision than other planning methods that do not take into account time-varying power supply and demand and data center power management policies.

The consolidation of multiple servers and their workloads aims to minimize the number of servers needed thereby enabling the efficient use of server and power resources. At the same time, applications participating in consolidation scenarios often have specific quality of service requirements that need to be supported. To evaluate which workloads can be consolidated to which servers we employ a trace-based approach that determines a near optimal workload placement that provides specific qualities of service. However, the chosen workload placement is based on past demands that may not perfectly predict future demands. To further improve efficiency and application quality of service we apply the trace-based technique repeatedly, as a workload placement controller. We integrate the workload placement controller with a reactive controller that observes current behavior to i) migrate workloads off of overloaded servers and ii) free and shut down lightly-loaded servers. To evaluate the effectiveness of the approach, we developed a new host load emulation environment that simulates different management policies in a time effective manner. A case study involving three months of data for 138 SAP applications compares our integrated controller approach with the use of each controller separately. The study considers trade-offs between i) required capacity and power usage, ii) resource access quality of service for CPU and memory resources, and iii) the number of migrations. We consider two typical enterprise environments: blade and server based resource pool infrastructures. The results show that the integrated controller approach outperforms the use of either controller separately for the enterprise application workloads in our study. We show the influence of the blade and server pool infrastructures on the effectiveness of the management policies.

Advances in server, network, and storage virtualization are enabling the creation of resource pools of servers that permit multiple application workloads to share each server in the pool. This paper proposes and evaluates aspects of a capacity management process for automating the efficient use of such pools when hosting large numbers of services. We use a trace based approach to capacity management that relies on i) a definition for required capacity, ii) the characterization of workload demand patterns, iii) the generation of synthetic workloads that predict future demands based on the patterns, and iv) a workload placement recommendation service. A case study with 6 months of data representing the resource usage of 139 workloads in an enterprise data center demonstrates the effectiveness of the proposed capacity management process. Our results show that when consolidating to 8 processor systems, we predicted future per-server required capacity to within one processor 95% of the time. The approach enabled a 35% reduction in processor usage as compared to today's current best practice for workload placement.

The Internet has evolved into a multi-service infrastructure for the telecom and computer industries. Internet services are affected by congestions caused by the operations of Internet routing protocols. We propose a novel information service that guides the operations of Internet routing protocols to avoid such congestion. The information service maintains information about utilization of inter-domain links. The proposed service minimizes the maximum of utilization of Internet links by selecting potential network domains to be traversed by Internet services. Simulation results show better balanced distribution of traffic workloads among network links using the proposed service. We show that the proposed service scales well as network size grows. This comes at the cost of greater control messaging overhead which suggests using the proposed service with long-lived and higher bandwidth services

Grid computing environments typically federate heterogeneous resource clusters belonging to several organizations. To fully realize the promise of a grid environment, it is necessary to support tools that help obtain insights into the behaviour of individual clusters. This paper describes a cluster service called Replay that simplifies the development and maintenance of such tools. The service provides a common model-based interface for obtaining current and historical information about a cluster. Replay can manage multiple views which allows tools to obtain information about an existing cluster as well as information that shows how a cluster might have behaved under alternate configurations and workloads. The model-based interface allows tools to be ported to different clusters with little effort. Furthermore, Replay uses different mechanisms to manage information that typically changes infrequently and information that can change in a more dynamic, continuous manner allowing it to handle information more efficiently than existing services that provide similar functionality. The paper presents a job analysis tool to illustrate the utility of the service

We consider shared resource pool management taking into account per-application quality of service (QoS) requirements and server failures. Application QoS requirements are defined by complementary specifications for acceptable and time-limited degraded performance. Furthermore, a requirement specification is provided for both the normal case and for the case where an application server fails in the pool. Independently, the resource pool operator provides a resource access QoS commitment for two classes of service (CoS). These govern statistical multiplexing within the pool. A QoS translation automatically maps application demands onto the resource pool's CoS to best enable sharing. A workload placement service consolidates applications to a small number of servers while satisfying application QoS requirements. The service reports whether a spare server is needed or how applications affected by a single failure can operate according to failure QoS constraints using remaining servers until the failure can be repaired. A case study demonstrates the approach

This paper describes a scalable and hierarchical virtual network creation (HVNC) protocol. The protocol encapsulates network control operations such as signaling and routing. It relies on a hierarchical organization of network resources and their managers. The network hierarchy of HVNC provides network-wide views of network resource status. This enables network-wide decisions such as balancing traffic loads among network domains and traffic isolation in accordance to policies. HVNC enables flexible and dynamic virtual network management including reconfiguration and failure recovery. These advantages come at the cost of higher setup and information storage costs when compared to standard service creation protocols. We characterize the advantages and costs of HVNC by simulation. The relatively high costs of HVNC make it more suitable to create long-lived high-bandwidth virtual networks. This type of virtual networks supports emerging technologies and applications such as grid and peer-to-peer computing and data-intensive e-science applications

Resource pools are computing environments that offer virtualized access to shared resources. When used effectively they can align the use of capacity with business needs (flexibility), lower infrastructure costs (via resource sharing), and lower operating costs (via automation). Using resources effectively can rely on a combination of workload placement and workload management technologies. Workload placement decides which workloads will share resources. Workload management governs short term access to resource capacity. It provides performance isolation within resource pools to ensure resource sharing even under high loads. A workload manager can have a direct impact both on an application's overall resource access quality of service and on the number of workloads that can be assigned to a pool. In this paper we take a detailed look at an application workload's demands. We explore tradeoffs in resource access quality of service received by the application and the minimum allocation of resources for the workload. We show that by careful selection of workload scheduling parameters along with a proposed fast allocation policy we can sometimes more than triple the number of workloads that can be assigned to a pool without sacrificing application workload quality of service or the efficiency of the resource pool.

Enterprise applications are often business critical but lack effective synthetic workload generation techniques to evaluate performance. These workloads are characterized by sessions of interdependent requests that often cause and exploit dynamically generated responses. Interrequest dependencies must be reflected in synthetic workloads for these systems to exercise application functions correctly. This poses significant challenges for automating the construction of representative synthetic workloads and manipulating workload characteristics for sensitivity analyses. This paper presents a technique to overcome these problems. Given request logs for a system under study, the technique automatically creates a synthetic workload that has specified characteristics and maintains the correct interrequest dependencies. The technique is demonstrated through a case study involving a TPC-W e-commerce system. Results show that incorrect performance results can be obtained by neglecting interrequest dependencies, thereby highlighting the value of our technique. The study also exploits our technique to investigate the impact of several workload characteristics on system performance. Results establish that high variability in the distributions of session length, session idle times, and request service times can cause increased contention among sessions, leading to poor system responsiveness. To the best of our knowledge, these are the first results of this kind for a session-based system. We believe our technique is of value for studies where fine control over workload is essential

Utility computing is envisioned as the future of enterprise IT environments. Achieving utility computing is a daunting task, because enterprise users have diverse and complex needs. In this paper we describe quartermaster, an integrated set of tools that addresses some of these needs. Quartermaster supports the entire lifecycle of computing tasks - including design, deployment, operation, and decommissioning of each task. Although individual components of this lifecycle have been addressed in earlier work, quartermaster integrates them in a unified framework using model-based automation. All tools within quartermaster are integrated using models based on the common information model (CIM), an industry-standard model from the distributed management task force (DMTF). The paper discusses the quartermaster implementation, and describes two case studies using quartermaster.

The paper presents a novel scalable end-to-end routing protocol, HDRA, which combines the characteristics of both hierarchical and distributed routing protocols. HDRA selects a number of network domains to be traversed by a requested route. Only the selected domains are flooded with probing messages. This enables the deployment of flooding-based routing algorithms in large networks. The protocol has smaller setup time because it performs no central routing computation. HDRA does not require global state maintenance. However, it has higher message overheads when compared to PNNI (private network-network interface).

Network service provisioning involves the control of network resources through signaling, routing and management protocols that achieve quality of service and traffic engineering objectives. Network service providers are typically faced with scalability problems due to the explosion in network size and demand. In this paper we propose a new hierarchical distributed protocol for the provisioning of MPLS-based tunnels. The protocol encapsulates the required signaling and routing. The protocol alleviates the scalability problem through extending the current two-tier architecture of the Internet to a multi-level hierarchical one. It enables mechanisms for quality of service, and intra- and inter-domain traffic engineering. Analytical analysis and simulation results show that the protocol exploits parallelism in routing computation to reduce the setup time of an MPLS path. This comes at the expense of an increased message complexity relative to other hierarchical protocols.

One of the goals of network service providers (SP) is to provision efficiently value-added networking services to their end users. The services may have different duration, connectivity, and quality of service (QoS) requirements. An SP meets such requirements by exploiting various routing and management mechanisms for edge and core networks. We focus on network services that require the establishment of virtual networks (VNs). This paper considers a VN creation architecture based on multi-protocol label switching (MPLS). MPLS is one approach for implementing VNs that enables programmability for network infrastructure and hence the creation of value-added services. An analytic performance model is developed to assess the scalability of the architecture as the underlying physical network infrastructure evolves. The model we present describes only coarse features of the architecture, yet offers insights regarding its performance and scalability. In particular we assess the impact of a flat versus hierarchical MPLS mechanism on the response time of service creation.

This article presents an analysis of five days of workload data from a large Web-based shopping system. The multitier environment of this Web-based shopping system includes Web servers, application servers, database servers, and an assortment of load-balancing and firewall appliances. We characterize user requests and sessions and determine their impact on system performance scalability. The purpose of our study is to assess scalability and support capacity planning exercises for the multitier system. We find that horizontal scalability is not always an adequate mechanism for supporting increased workloads and that personalization and robots can have a significant impact on system scalability.

Distributed application systems are composed of classes of objects with instances that interact to accomplish common goals. Such systems can have many classes of users with many types of requests. Furthermore, the relative load of these classes can shift throughout the day, causing changes to system behavior and bottlenecks. When designing and deploying such systems, it is necessary to determine a process replication and threading policy for the server processes that contain the objects, as well as process activation policies. To avoid bottlenecks, the policy must support all possible workload conditions. Licensing, implementation or resource constraints can limit the number of permitted replicas or threads of a server process. Process activation policies determine whether a server is persistent or should be created and terminated with each call. This paper describes quantitative techniques for choosing process replication or threading levels and process activation policies. Inappropriate policies can lead to unnecessary queuing delays for callers or unnecessarily high consumption of memory resources. The algorithms presented consider all workload conditions, are iterative in nature and are hybrid mathematical programming and analytic performance evaluation methods. An example is given to demonstrate the technique and describe how the results can be applied during software design and deployment.

Performance models of software designs can give early warnings of problems such as resource saturation or excessive delays. However models are seldom used because of the considerable effort needed to construct them. The ANGIOTRACE/sup TM/ was developed to gather the necessary information from an executable design and develop a model in an automated fashion. It applies to distributed and concurrent software with synchronous (send-reply or RPC) communications, developing a layered queuing network model. The trace-based load characterization (TLC) technique presented here extends the ANGIOTRACE/sup TM/ to handle software with both synchronous and asynchronous interactions. TLC also detects interactions which are effectively synchronous or partly-synchronous (forwarding) but are built up from asynchronous messages. These patterns occur in telephony software and in other systems. The TLC technique can be applied throughout the software life-cycle, even after deployment.

Distributed applications are being developed that contain one or more layers of software servers. Software processes within such systems suffer contention delays both for shared hardware and at the software servers. The responsiveness of these systems is affected by the software design, the threading level and number of instances of software processes, and the allocation of processes to processors. The Method of Layers (MOL) is proposed to provide performance estimates for such systems. The MOL uses the mean value analysis (MVA) linearizer algorithm as a subprogram to assist in predicting model performance measures.>

Distributed computing applications are collections of processes allocated across a network that cooperate to accomplish common goals. The applications require the support of a distributed computing runtime environment that provides services to help manage process concurrency and interprocess communication. This support helps to hide much of the inherent complexity of distributed environments via industry standard interfaces and permits developers to create more portable applications. The resource requirements of the runtime services can be significant and may impact application performance and system throughput. This paper describes work done to study the potential benefits of redesigning some aspects of the DCE RPC and its current implementation on a specific platform.