MapReduce is a distributed computing paradigm widely used for building large-scale data processing applications. When used in cloud environments, MapReduce clusters are dynamically created using virtual machines (VMs) and managed by the cloud provider. In this paper, we study the energy efficiency problem for such MapReduce clouds. We describe a unique spatio-temporal tradeoff that includes efficient spatial fitting of VMs on servers to achieve high utilization of machine resources, as well as balanced temporal fitting of servers with VMs having similar runtimes to ensure a server runs at a high utilization throughout its uptime. We propose VM placement algorithms that explicitly incorporate these tradeoffs. Further, we propose techniques that dynamically scale MapReduce clusters to further improve energy consumption while ensuring that jobs meet or improve their expected runtimes. Our algorithms achieve energy savings over existing placement techniques, and an additional optimization technique further achieves savings while simultaneously improving job performance.

MapReduce is a distributed computing paradigm widely used for building large-scale data processing applications. When used in cloud environments, MapReduce clusters are dynamically created using virtual machines (VMs) and managed by the cloud provider. In this paper, we study the energy efficiency problem for such MapReduce clusters in private cloud environments, that are characterized by repeated, batch execution of jobs. We describe a unique spatio-temporal tradeoff that includes efficient spatial fitting of VMs on servers to achieve high utilization of machine resources, as well as balanced temporal fitting of servers with VMs having similar runtimes to ensure a server runs at a high utilization throughout its uptime. We propose VM placement algorithms that explicitly incorporate these tradeoffs. Our algorithms achieve energy savings over existing placement techniques, and an additional optimization technique further achieves savings while simultaneously improving job performance.

MapReduce has gained in popularity as a distributed data analysis paradigm, particularly in the cloud, where MapReduce jobs are run on virtual clusters. The provisioning of MapReduce jobs in the cloud is an important problem for optimizing several user as well as provider-side metrics, such as runtime, cost, throughput, energy, and load. In this paper, we present an intelligent provisioning framework called STEAMEngine that consists of provisioning algorithms to optimize these metrics through a set of common building blocks. These building blocks enable spatio-temporal tradeoffs unique to MapReduce provisioning: along with their resource requirements (spatial component), a MapReduce job runtime (temporal component) is a critical element for any provisioning algorithm. We also describe tw o novel provisioning algorithms - a user-driven performance optimization and a provider-driven energy optimization - that leverage these building blocks. Our experimental results based on an Amazon EC2 cluster and a local Xen/Hadoop cluster show the benefits of STEAMEngine through improvements in performance and energy via the use of these algorithms and building blocks.

Many modern systems exploit data redundancy to improve efficiency. These systems split data into chunks, generate identifiers for each of them, and compare the identifiers among other data items to identify duplicate chunks. As a result, chunk size becomes a critical parameter for the efficiency of these systems: it trades potentially improved similarity detection (smaller chunks) with increased overhead to represent more chunks. Unfortunately, the similarity between files increases unpredictably with smaller chunk sizes, even for data of the same type. Existing systems often pick one chunk size that is "good enough'' for many cases because they lack efficient techniques to determine the benefits at other chunk sizes. This paper addresses this deficiency via two contributions: (1) we present multi-resolution (MR) handprinting, an application-independent technique that efficiently estimates similarity between data items at different chunk sizes using a compact, multi-size representation of the data; (2) we then evaluate the application of MR handprints to workloads from peer-to-peer, file transfer, and storage systems, demonstrating that the chunk size selection enabled by MR handprints can lead to real improvements over using a fixed chunk size in these systems.

This paper presents the design, implementation, and evaluation of Ditto, a system that opportunistically caches overheard data to improve subsequent transfer throughput in wireless mesh networks. While mesh networks have been proposed as a way to provide cheap, easily deployable Internet access, they must maintain high transfer throughput to be able to compete with other last-mile technologies. Unfortunately, doing so is difficult because multi-hop wireless transmissions interfere with each other, reducing the available capacity on the network. This problem is particularly severe in common gateway-based scenarios in which nearly all transmissions go through one or a few gateways from the mesh network to the Internet.Ditto exploits on-path as well as opportunistic caching based on overhearing to improve the throughput of data transfers and to reduce load on the gateways. It uses content-based naming to provide application independent caching at the granularity of small chunks, a feature that is key to being able to cache partially overheard data transfers. Our evaluation of Ditto shows that it can achieve significant performance gains for cached data, increasing throughput by up to 7x over simpler on-path caching schemes, and by up to an order of magnitude over no caching.

Several multicast protocols for mobile ad hoc networks have been proposed, which build multicast trees by using location information that is available from the Global Positioning System (GPS) or localization algorithms and use geographic forwarding to forward packets down the multicast trees. These stateless multicast protocols carry encoded membership, location, and tree information in each packet and are more efficient and robust than stateful protocols (for example, ADMR and ODMRP), as they avoid the difficulty of maintaining distributed state in the presence of frequent topology changes. However, current stateless multicast protocols are not scalable to large groups because of the per-packet encoding overhead, and the centralized group membership and location management. We present the hierarchical rendezvous point multicast (HRPM) protocol, which significantly improves the scalability of stateless multicast with respect to the group size. HRPM consists of two key design ideas: 1) hierarchical decomposition of a large group into a hierarchy of recursively organized manageable-sized subgroups and 2) the use of distributed geographic hashing to construct and maintain such a hierarchy at virtually no cost. Our detailed simulations demonstrates that HRPM achieves significantly enhanced scalability and performance due to hierarchical organization and distributed hashing.

Overlay routing has emerged as a promising approach to improving performance and reliability of Internet paths. To fully realize the potential of overlay routing under the constraints of deployment costs in terms of hardware, network connectivity and human effort, it is critical to carefully place infrastructure overlay nodes to balance the trade-off between performance and resource constraints. In this paper, we investigate approaches to perform intelligent placement of overlay nodes to facilitate (i) resilient routing and (ii) TCP performance improvement. We formulate objective functions to accurately capture application behavior: reliability and TCP performance, and develop several placement algorithms, which offer a wide range of trade-offs in complexity and required knowledge of the client- server location and traffic load. Using simulations on synthetic and real Internet topologies, and PlanetLab experiments, we demonstrate the effectiveness of the placement algorithms and objective functions developed, respectively. We conclude that an approach, hybrid of random and greedy approaches, provides the best tradeoff between computational efficiency and accuracy. We also uncover the fundamental challenge in simultaneously optimizing for reliability and TCP performance, and propose a simple unified algorithm to achieve the same.

Multi-hop wireless mesh networks are increasingly being deployed for last-mile Internet access. Typically, network algorithms such as routing, channel assignment and topology control for such networks rely heavily on metrics that intend to capture link "quality" across the network. However, the underlying dynamics of the proposed link metrics themselves have not yet been studied in detail. In this paper, we study the dynamics of the most popular link metrics in real network deployments. Using two wireless mesh testbeds, we measure a number of link metrics across different hardware platforms and network environments. The collected measurements allow us to study the stability and sensitivity of the different metrics to various conditions. Our study provides several insights and future research directions on how network algorithms need to adapt to link dynamics as well as how popular and widely used link metrics can be improved.

Network delays and delay variations are two of the most important network performance metrics directly impacting real-time applications such as voice over IP and time-critical financial transactions. This importance is illustrated by past work on understanding the delay constancy of Internet paths and recent work on predicting network delays using virtual coordinate systems. Merely understanding currently observed delays is insufficient, as network performance can degrade not only due to traffic variability but also as a result of routing changes. Unfortunately this latter effect so far has been ignored in understanding and predicting delay related performance metrics of Internet paths. Our work is the first to address this short coming by systematically analyzing changes in network delays and jitter of a diverse and comprehensive set of Internet paths. Using empirical measurements, we illustrate that routing changes can result in roundtrip delay increase of converged paths by more than 1 second. Surprisingly, intradomain routing changes can also cause such large delay increase.

An important stage of wide-area systems and networking research is to prototype a system to understand its performance when deployed in the real Internet. A key requirement of prototyping is that results obtained from the prototype experiments be representative of the behavior if the system were deployed over nodes connected to commercial ISPs. Recently, distributed testbeds such as PlanetLab and RON have become increasingly popular for performing wide-area experimentation. However, such testbeds typically consist of a significant fraction of nodes with connectivity to research and education networks which potentially hinder their usability in prototyping systems.In this paper, we investigate the impact of testbeds with connectivity to research and education networks on the applications and network services so that such testbeds can be leveraged for evaluation and prototyping. Specifically, we investigate when the representativeness of wide-area experiments deployed on such testbeds is affected by studying the routing paths that applications use over such testbeds. We then investigate how the representativeness of wide-area experiments is affected by studying the performance properties of such paths. We further measure the impact of using such testbeds on application performance via application case studies. Finally, we propose a technique that uses the currently available testbeds but reduces their bias by exposing applications evaluated to network conditions more reflective of the conditions in the commercial Internet.

The nature of digital content is undergoing a radical transformation due to the growing infusion of user-generated content. Users that generate content have a strong motivation to actively promote their content in order to achieve publicity, recognition or to simply spread their viewpoints, knowledge and creations. Existing content sharing mechanisms available to such users, such as in P2P file sharing, are pull-based, relegating the user to a passive role. To enable active participation from users, we propose Promoter, a framework that enhances P2P file sharing systems to support content promotion in addition to the available pull-based content search. Promotion is enabled by publicizing a user's content to peers and disseminating the content by incentivizing peers to download and further propagate it.

Wireless mesh networks (WMNs) have been proposed as an effective solution for ubiquitous last-mile broadband access. Three key factors that affect the usability of WMNs are high throughput, cost-effectiveness, and ease of deployability. In this paper, we propose DMesh, a WMN architecture that combines spatial separation from directional antennas with frequency separation from orthogonal channels to improve the throughput of WMNs. DMesh achieves this improvement without inhibiting cost-effectiveness and ease of deployability by utilizing practical directional antennas that are widely and cheaply available (e.g., patch and yagi) in contrast to costly and bulky smart beamforming directional antennas. Thus, the key challenge in DMesh is to exploit spatial separation from such practical directional antennas despite their lack of electronic steerability and interference nulling, as well as the presence of significant sidelobes and backlobes. In this paper, we study how such practical directional antennas can improve the throughput of a WMN. Central to our architecture is a distributed, directional channel assignment algorithm for mesh routers that effectively exploits the spatial and frequency separation opportunities in a DMesh network. Simulation results show that DMesh improves the throughput of WMNs by up to 231% and reduces packet delay drastically compared to a multiradio multichannel omni antenna network. A DMesh implementation in our 16-node 802.11b WMN testbed using commercially available practical directional antennas provides transmission control protocol throughput gains ranging from 31% to 57%

The use of wireless in local loop (WiLL) has generated considerable interest due to the advantages it offers such as ease and low cost of deployment and maintenance. With an increase in the number of subscribers in the network, it becomes expedient to employ spectrum reusability techniques such as the use of multihop relaying in order to improve the capacity of the wireless systems. Throughput enhanced wireless in local loop (TWiLL) is one such architecture that employs multihop relaying and shortcut relaying to reuse bandwidth in WiLL systems. Compared to other multihop wireless network architectures, TWiLL architecture assumes significance due to its potential use in fixed wireless broadband services such as LMDS (local multipoint distribution service) and MMDS (multichannel multipoint distribution system). Analysis of the call acceptance ratio (CAR) in multihop wireless architectures including TWiLL is nontrivial as the Erlang B formula no longer holds. In this paper, we build multidimensional Markov chains to analyze the performance of multihop wireless systems such as TWiLL that has multiple types of channels. We also compare the results of our analysis with results from simulations. We observe that multihop relaying and shortcut relaying lead to a significant increase in the CAR of WiLL systems. Also, the free space propagation model that is normally used to model the radio channel is a very unrealistic model and does not consider reflection, diffraction, scattering, and multipath propagation that hinder transmissions in WiLL systems. In this paper, we studied the effect of several realistic radio channel propagation models on the performance of the TWiLL system through analysis and simulations

In this paper, we study the problem of providing fairness in multi-hop wireless infrastructure access. In such networks, it is well known that the use of current media access and transport protocols can result in severe unfairness and even starvation for flows originated from different numbers of hops away from a wired infrastructure point or gateway. In this paper, we study a different type of fairness that exists in such networks - flows initiated by nodes that are similar numbers of hops away from the gateway can experience significant unfairness, and such unfairness exists even for perfectly symmetrical node distribution and channel conditions. We denote such fairness as symmetrical fairness. We first provide a framework to characterize and measure symmetrical fairness. We then perform an extensive set of simulation experiments to quantify the causes of symmetrical unfairness. Finally, we develop and study a distributed routing algorithm that significantly improves the symmetrical fairness

Geographic routing protocols allow stateless routing in mobile ad hoc networks (MANETs) by taking advantage of the location information of mobile nodes and thus are highly scalable. A central challenge in geographic routing protocols is the design of scalable distributed location services that track mobile node locations. A number of location services have been proposed, but little is known about the relative performance of these location services. In this paper, we perform a detailed performance comparison of three rendezvous-based location services that cover a range of design choices: a quorum-based protocol (XYLS) which disseminates each node's location to O(/spl radic/N) nodes, a hierarchical protocol (GLS) which disseminates each node's location to O(logN) nodes, and a geographic hashing based protocol (GHLS) which disseminates each node's location to O(1) nodes. We present a quantitative model of protocol overheads for predicting the performance tradeoffs of the protocols for static networks. We then analyze the performance impact of mobility on these location services. Finally, we compare the performance of routing protocols equipped with the three location services with two topology-based routing protocols, AODV and DSR, for a wide range of network sizes. Our study demonstrates that when practical MANET sizes are considered, robustness to mobility and the constant factors matter more than the asymptotic costs of location service protocols. In particular, while GLS scales better asymptotically, GHLS is far simpler, transmits fewer control packets, and delivers more data packets than GLS when used with geographic routing in MANETs of sizes considered practical today and in the near future. Similarly, although XYLS scales worse asymptotically than GLS, it transmits fewer control packets and delivers more data packets than GLS in large mobile networks.

In this paper, we examine the communication model widely used in simulation studies of mobile ad hoc networks (MANETs). We find that the communication model uses an overly simplistic traffic pattern which restricts the number of connections that originate from each source node to be one or two, and thus the communication model may not represent traffic patterns in many potential applications in MANETs. We then propose a new communication model which extends the previous communication model to include a more general traffic pattern that varies the number of connections per source node. We study the performance impact of traffic patterns on various routing protocols via detailed simulations of a MANET of 112 mobile nodes. Our simulation results show that many of the conclusions drawn in previous protocol comparison studies no longer hold under the new communication model. These results motivate the need for performance evaluation of MANETs to not only include rich and diverse mobility models as has been done in the past but also include diverse traffic patterns that stress a wide set of protocol design issues.