Online advertising is progressively moving towards a programmatic model in which ads are matched to actual interests of individuals collected as they browse the web. Letting the huge debate around privacy aside, a very important question in this area, for which little is known, is: How much do advertisers pay to reach an individual?

We have developed a novel methodology for deriving bandwidth prices for premium direct peering between Access ISPs (A-ISPs) and Content and Service Providers (CSPs) that want to deliver content and services in premium quality. Our methodology establishes a direct link between service profitability, for example, from advertising, user and subscriber loyalty, interconnection costs, and finally bandwidth price for peering. Unlike existing work in both the networking and economics literature, our resulting computational model, built around Nash bargaining, can be used for deriving quantitative results comparable to actual market prices. We analyze the U.S. market and derive prices for video, that compare favorably with existing prices for transit and paid peering. We also observe that the fair prices returned by the model for high-profit/low-volume services such as search, are orders of magnitude higher than current bandwidth prices. This implies that resolving existing (fierce) interconnection tussles may require per service, instead of wholesale, peering between A-ISPs and CSPs. Our model can be used for deriving initial benchmark prices for such negotiations.

The concepts of Big Data have became intertwined with those of the Internet of Things, creating mental pictures of a fully connected, all-encompassing, cyber-physical world, where each and every object will contribute with information to a "fully aware" society. Academic works are presenting this as the natural evolution for our current technologies. The panel looks at these promises from the hard perspective of reality: what is being done, how much it cost, what needs to be developed, and what can be expected in the near and mid-term.

A significant fraction of Internet traffic is now encrypted and HTTPS will likely be the default in HTTP/2. However, Transport Layer Security (TLS), the standard protocol for encryption in the Internet, assumes that all functionality resides at the endpoints, making it impossible to use in-network services that optimize network resource usage, improve user experience, and protect clients and servers from security threats. Re-introducing in-network functionality into TLS sessions today is done through hacks, often weakening overall security.

A substantial amount of work has recently gone into localizing BitTorrent traffic within an ISP in order to avoid excessive and often times unnecessary transit costs. Several architectures and systems have been proposed and the initial results from specific ISPs and a few torrents have been encouraging. In this work we attempt to deepen and scale our understanding of locality and its potential. Looking at specific ISPs, we consider tens of thousands of concurrent torrents, and thus capture ISP-wide implications that cannot be appreciated by looking at only a handful of torrents. Second, we go beyond individual case studies and present results for few thousands ISPs represented in our data set of up to 40K torrents involving more than 3.9M concurrent peers and more than 20M in the course of a day spread in 11K ASes. Finally, we develop scalable methodologies that allow us to process this huge data set and derive accurate traffic matrices of torrents. Using the previous methods we obtain the following main findings: i) Although there are a large number of very small ISPs without enough resources for localizing traffic, by analyzing the 100 largest ISPs we show that Locality policies are expected to significantly reduce the transit traffic with respect to the default random overlay construction method in these ISPs; ii) contrary to the popular belief, increasing the access speed of the clients of an ISP does not necessarily help to localize more traffic; iii) by studying several real ISPs, we have shown that soft speed-aware locality policies guarantee win-win situations for ISPs and end users. Furthermore, the maximum transit traffic savings that an ISP can achieve without limiting the number of inter-ISP overlay links is bounded by “unlocalizable” torrents with few local clients. The application of restrictions in the number of inter-ISP links leads to a higher transit traffic reduction but the QoS of clients downloading “unlocalizable” torrents would be severely harmed.

The co-existence of cellular and wired networks has been exploited almost exclusively in the direction of OffLoading traffic from the former onto the latter. In this paper we claim that there exist cases that call for the exact opposite, i.e, use the cellular network to assist a fixed wired network. In particular, we show that by "OnLoading'' traffic from the wired broadband network onto the cellular network we can usefully speedup wired connections, on the downlink or the uplink. We consider the technological challenges pertaining to this idea and implement a prototype 3G OnLoading service that we call 3GOL, that can be deployed by an operator providing both the wired and cellular network services. By strategically OnLoading a fraction of the data transfers to the 3G network, one can significantly enhance the performance of particular applications. In particular we demonstrate non-trivial performance benefits of 3GOL to two widely used applications: video-on-demand and multimedia upload. We also consider the case when the operator that provides wired and cellular services is different, adding the analysis on economic constraints and volume cap on cellular data plans that need to be respected. Simulating 3GOL}over a DSLAM trace we show that 3GOL can reduce video pre-buffering time by at least 20% for 50% of the users while respecting data caps and we design a simple estimator to compute the daily allowance that can be used towards 3GOL while respecting caps. Our prototype is currently being piloted in 30 households in a large European city by a large network provider.

The large-scale collection and exploitation of personal information to drive targeted online advertisements has raised privacy concerns. As a step towards understanding these concerns, we study the relationship between how much information is collected and how valuable it is for advertising. We use HTTP traces consisting of millions of users to aid our study and also present the first comparative study between aggregators. We develop a simple model that captures the various parameters of today's advertising revenues, whose values are estimated via the traces. Our results show that per aggregator revenue is skewed (5% accounting for 90% of revenues), while the contribution of users to advertising revenue is much less skewed (20% accounting for 80% of revenue). Google is dominant in terms of revenue and reach (presence on 80% of publishers). We also show that if all 5% of the top users in terms of revenue were to install privacy protection, with no corresponding reaction from the publishers, then the revenue can drop by 30%.

Many emerging scientific and industrial applications require transferring multiple terabytes of data on a daily basis. Examples include pushing scientific data from particle accelerators/colliders to laboratories around the world, synchronizing datacenters across continents, and replicating collections of high-definition videos from events taking place at different time-zones. A key property of all above applications is their ability to tolerate delivery delays ranging from a few hours to a few days. Such delay-tolerant bulk (DTB) data are currently being serviced mostly by the postal system using hard drives and DVDs, or by expensive dedicated networks. In this paper, we propose transmitting such data through commercial ISPs by taking advantage of already-paid-for off-peak bandwidth resulting from diurnal traffic patterns and percentile pricing. We show that between sender-receiver pairs with small time-zone difference, simple source scheduling policies are able to take advantage of most of the existing off-peak capacity. When the time-zone difference increases, taking advantage of the full capacity requires performing store-and-forward through intermediate storage nodes. We present an extensive evaluation of the two options based on traffic data from 200+ links of a large transit provider with points of presence (PoPs) at three continents. Our results indicate that there exists huge potential for performing multiterabyte transfers on a daily basis at little or no additional cost.

The difficulty of partitioning social graphs has introduced new system design challenges for scaling of online social networks (OSNs). Vertical scaling by resorting to full replication can be a costly proposition. Scaling horizontally by partitioning and distributing data among multiple servers using, for e.g., distributed hash tables (DHTs), can suffer from expensive interserver communication. Such challenges have often caused costly rearchitecting efforts for popular OSNs like Twitter and Facebook. We design, implement, and evaluate SPAR, a Social Partitioning and Replication middleware that mediates transparently between the application and the database layer of an OSN. SPAR leverages the underlying social graph structure in order to minimize the required replication overhead for ensuring that users have their neighbors' data colocated in the same machine. The gains from this are multifold: Application developers can assume local semantics, i.e., develop as they would for a single machine; scalability is achieved by adding commodity machines with low memory and network I/O requirements; and N+K redundancy is achieved at a fraction of the cost. We provide a complete system design, extensive evaluation based on datasets from Twitter, Orkut, and Facebook, and a working implementation. We show that SPAR incurs minimum overhead, can help a well-known Twitter clone reach Twitter's scale without changing a line of its application logic, and achieves higher throughput than Cassandra, a popular key-value store database.

Monetizing personal information is a key economic driver of online industry. End-users are becoming more concerned about their privacy, as evidenced by increased media attention. This paper proposes a mechanism called 'transactional' privacy that can be applied to personal information of users. Users decide what personal information about themselves is released and put on sale while receiving compensation for it. Aggregators purchase access to exploit this information when serving ads to a user. Truthfulness and efficiency, attained through an unlimited supply auction, ensure that the interests of all parties in this transaction are aligned. We demonstrate the effectiveness of transactional privacy for web-browsing using a large mobile trace from a major European capital. We integrate transactional privacy in a privacy-preserving system that curbs leakage of information. These mechanisms combine to form a market of personal information that can be managed by a trusted third party.

A substantial amount of work has recently gone into localizing BitTorrent traffic within an ISP in order to avoid excessive and often times unnecessary transit costs. Several architectures and systems have been proposed and the initial results from specific ISPs and a few torrents have been encouraging. In this work we attempt to deepen and scale our understanding of locality and its potential. Looking at specific ISPs, we consider tens of thousands of concurrent torrents, and thus capture ISP-wide implications that cannot be appreciated by looking at only a handful of torrents. Secondly, we go beyond individual case studies and present results for the top 100 ISPs in terms of number of users represented in our dataset of up to 40K torrents involving more than 3.9M concurrent peers and more than 20M in the course of a day spread in 11K ASes. We develop scalable methodologies that allow us to process this huge dataset and get concrete quantitative answers rather than qualitative speculations to questions like: “what is the minimum and the maximum transit traffic reduction across hundreds of ISPs?”, “what are the win-win boundaries for ISPs and their users?”, “what is the maximum amount of transit traffic that can be localized without requiring fine-grained control of inter-AS overlay connections?”.

Large datacenter operators with sites at multiple locations dimension their key resources according to the peak demand of the geographic area that each site covers. The demand of specific areas follows strong diurnal patterns with high peak to valley ratios that result in poor average utilization across a day. In this paper, we show how to rescue unutilized bandwidth across multiple datacenters and backbone networks and use it for non-real-time applications, such as backups, propagation of bulky updates, and migration of data. Achieving the above is non-trivial since leftover bandwidth appears at different times, for different durations, and at different places in the world.

Access networks include modems, home gateways, and DSL Access Multiplexers (DSLAMs), and are responsible for 70-80% of total network-based energy consumption. In this paper, we take an in-depth look at the problem of greening access networks, identify root problems, and propose practical solutions for their user- and ISP-parts. On the user side, the combination of continuous light traffic and lack of alternative paths condemns gateways to being powered most of the time despite having Sleep-on-Idle (SoI) capabilities. To address this, we introduce Broadband Hitch-Hiking (BH2), that takes advantage of the overlap of wireless networks to aggregate user traffic in as few gateways as possible. In current urban settings BH2 can power off 65-90% of gateways. Powering off gateways permits the remaining ones to synchronize at higher speeds due to reduced crosstalk from having fewer active lines. Our tests reveal speedup up to 25%. On the ISP side, we propose introducing simple inexpensive switches at the distribution frame for batching active lines to a subset of cards letting the remaining ones sleep. Overall, our results show an 80% energy savings margin in access networks. The combination of B2 and switching gets close to this margin, saving 66% on average.

The Internet is witnessing explosive growth in traffic due to bulk content transfers, such as multimedia and software downloads, and online sharing of personal, commercial, and scientific data. Yet bulk data transfers remain very expensive and inefficient. As a result, huge amounts of digital data continue to be delivered outside of the Internet using hard drives, optical media or tapes. Meanwhile, large reserves of spare bandwidth lie unutilized in today's networks, where links are overprovisioned for peak load. We designed NetEx, a bulk transfer system that opportunistically exploits the excess capacities of network links to deliver bulk content cheaply and efficiently. Our results based on data from both a commercial tier-1 ISP and the Abilene network suggest that NetEx can considerably increase the capacity of the network, and at the same time it can provide good average performance to bulk transfers.

In recent years there has been renewed interest in the problem of transferring bulk data (terabytes) utilizing commercial ISPs. The need to transfer bulk data arises in various scientific and industrial applications. Today, this data is moved using postal service in conjunction with hard drives and DVDs or special high performance dedicated networks. The key insight underlying the recent work was that many of the applications are delay- tolerant and hence the bulk data can be transferred at minimal cost, utilizing already paid-for off-peak bandwidth resulting from diurnal traffic patterns, using store-and-forward through intermediate storage nodes. In this paper we expand on this theme and consider the computational complexity of transferring data over a network whose links have time-varying capacities. We show that the general problem of finding a cost-optimal transfer of the bulk data can be solved in polynomial-time using minimum cost flow algorithms on a time-expanded version of the underlying network. Our solution involves graph transformations. We present additional transformations that enable the handling of half-duplex links (e.g. fiber-optic links) as well as node processing constraints (e.g. limitations on the processing power available for filtering or archiving). An important characteristic of our solution is the ability to handle nodes with storage. We consider nodes with storage that varies over time in terms of both capacity and cost. We show that our approach provably extends to cover the case of linear costs, providing polynomial- time algorithms. However, the flat-fee storage model is NP- complete and hence unlikely to be tractable in polynomial-time. Interestingly, with constrained storage, the optimal solutions may involve loops, i.e. the data may pass through the same node more than once on its way from the destination to the source along the optimal route. We use data from one of the world's leading ISPs and perform a comprehensive evaluation of our algorithm. We show that there exists a huge potential for cost savings in real-world networks with time-varying costs for both link capacities and node storage.

In wireless multihop networks, techniques such as multipath, local retransmissions and network coding have been successfully used to increase throughput and reduce losses. However, while these techniques improve forwarding performance for UDP, they often introduce side effects such as packet reordering and delay that heavily affect TCP traffic. In this paper we introduce CoMP, a network coding multipath forwarding scheme that improves the reliability and the performance of TCP sessions in wireless mesh networks. CoMP exploits the wireless mesh path diversity using network coding, performs congestion control and uses a credit-based method to control the rate at which linear combinations are transmitted. CoMP uses a simple algorithm to estimate losses and to send redundant linear combinations in order to maintain the decoding delay at a minimum and to prevent TCP timeouts and retransmissions. We evaluate CoMP through extensive simulations and compare it to state-of-the-art protocols. We show that CoMP not only achieves a higher throughput, but also is more efficient than existing protocols, making TCP sessions feasible for wireless mesh networks even under heavy losses.

One of the major concerns of IPTV network deployment is channel switching (or zapping) delay. This delay can add up to two seconds or more, and its main culprits are synchronisation and buffering. By analysing an extensive dataset - comprising 255 thousand users, 150 TV channels, and covering a 6-month period - we have observed that most channel switching events are linear: it is very common the user switching up or down to the next TV channel. This fact led us to the proposal, in this paper, of a simple mechanism to reduce channel switching delay. Our proposal is to send the neighbouring channels (i.e., channels adjacent to the requested one) to the Set Top Box (STB) during zapping periods. If the user switches to any of these channels the switching latency is virtually eliminated, not affecting therefore user's experience. Notwithstanding the simplicity of this scheme, trace-driven simulations show that the zapping delay can be virtually eliminated for a significant percentage of channel switching requests. As an example, by sending the previous and the next channel concurrently with the requested one, for only one minute after a zapping event, switching delay is eliminated for around 45% of all channel switching requests. Furthermore, this simple scheme has a performance close to that of an ideal predictor, while the increase of bandwidth utilisation in the access link is negligible.

Cost control for the Internet access providers (AP) influences not only the nominal speeds offered to the customers, but also other, more controversial, policies related to traffic shaping and discrimination. Given that the cost for the AP is determined by the peak-hour traffic (e.g. through the 95th-percentile), the individual user contribution towards the aggregate cost is not a linear function of its byte usage. In this paper we propose a metric for evaluating the contribution each individual user has on the peak demand, that is based on Shapley value, a well known game-theoretic concept. Given the computational complexity of calculating the Shapley value, we use a Monte Carlo method for approximating it with reasonable accuracy. We employ our methodology to study a dataset that logs per-subscriber temporal usage patterns over one month period for 10K broadband subscribers of a European AP and report observed results.

Aggregating multiple 802.11 Access Point (AP) backhauls using a single-radio WLAN card has been considered as a way of bypassing the backhaul capacity limit. However, current AP aggregation solutions greedily maximize the individual station throughput without taking fairness into account. This can lead to grossly unfair throughput distributions, which can discourage user participation and severely limit commercial deployability.

The difficulty of scaling Online Social Networks (OSNs) has introduced new system design challenges that has often caused costly re-architecting for services like Twitter and Facebook. The complexity of interconnection of users in social networks has introduced new scalability challenges. Conventional vertical scaling by resorting to full replication can be a costly proposition. Horizontal scaling by partitioning and distributing data among multiples servers - e.g. using DHTs - can lead to costly inter-server communication.

A substantial amount of work has recently gone into localizing BitTorrent traffic within an ISP in order to avoid excessive and often times unnecessary transit costs. In this work we aim to answer yet unanswered questions such as: what is the minimum and the maximum transit traffic reduction across hundreds of ISPs?, what are the win-win boundaries for ISPs and their users?, what is the maximum amount of transit traffic that can be localized without requiring fine-grained control of inter-AS overlay connections?, what is the impact to transit traffic from upgrades of residential broadband speeds?.

Motivated by increased concern over energy consumption in moderndatacenters,wepropose anew,distributedcomputingplatform calledNanoDataCenters(NaDa). NaDausesISP-controlledhome gateways to provide computing and storage services and adopts a managed peer-to-peer model to form a distributed data center infrastructure. To evaluate the potential for energy savings in NaDa platform we pick Video-on-Demand (VoD) services. We develop an energy consumption model for VoD in traditional and in NaDa data centers and evaluate this model using a large set of empirical VoD access data. We find that even under the most pessimistic scenarios, NaDa saves at least 20 % to 30 % of the energy compared to traditional data centers. These savings stem from energypreserving properties inherent to NaDa such as the reuse of already committed baseline power on underutilized gateways, the avoidance of cooling costs, and the reduction of network energy consumption as a result of demand and service co-localization in NaDa. Categories andSubject Descriptors

The typical state-of-the-art routing algorithms for delay tolerant networks are based on best next hop hill-climbing heuristics in order to achieve throughput and efficiency. The combination of these heuristics and the social network structure leads the routing to direct most of the traffic through a small subset of good users. For instance, in the SimBet algorithm, the top 10% of users carry out 54% of all the forwards and 85% of all the handovers. This unfair load distribution is not sustainable as it can quickly deplete constraint resources in heavily utilized mobile devices (e.g. storage, battery, budget, etc.). Moreover, because a small number of users carry a significant amount of the traffic, the system is not robust to random failures and attacks. To overcome these inefficiencies, this paper introduces Fair-Route, a routing algorithm for delay tolerant networks inspired by the social processes of perceived interaction strength, where messages are preferably forwarded to users that have a stronger social relation with the target of the message; and assortativity, that limits the exchange of messages to those users with similar "social status". We compare the performance of FairRoute to the state-of-the-art algorithms by extensive simulations on the MIT reality mining dataset. The results show that our algorithm outperforms existing algorithms in the de facto benchmark of throughput vs. forwards. Furthermore, it distributes better the load; the top 10% carry out 26% of the forwards and 28% of the handovers without any loss in performance.

ADSL is becoming the standard form of residential and small-business broadband Internet access due to, primarily, its low deployment cost. These ADSL residential lines are often deployed with 802.11 Access Points (AP) that provide wireless connectivity. Given the density of ADSL deployment, it is often possible for a residential wireless client to be in range of several other APs, belonging to neighbors, with ADSL connectivity. While the ADSL technology has showed evident limits in terms of capacity (with speeds ranging 1-10 Mbps), the short-range wireless communication can guarantee a much higher capacity (up to 20 Mbps). Furthermore, the ADSL links in the neighborhood are generally under-utilized, since ADSL subscribers do not connect 100% of the time. Therefore, it is possible for a wireless client to simultaneously connect to several APs in range and effectively aggregate their available ADSL bandwidth.In this paper, we introduce ClubADSL, a wireless client that can simultaneously connect to several APs in range on different frequencies and aggregate both their downlink and uplink capacity. ClubADSL is a software that runs locally on the client-side, and it requires neither modification to the existing Internet infrastructure, nor any hardware/protocol upgrades to the 802.11 local area network. We show the feasibility of ClubADSL in seamlessly transmitting TCP traffic, and validate its implementation both in controlled scenarios and with current applications over real ADSL lines. In particular we show that a ClubADSL client can greatly benefit from the aggregated download bandwidth in the case of server-client applications such as video streaming, but can also take advantage of the increased upload bandwidth greatly reducing download times with incentive-based P2P applications such as BitTorrent.

Summary form only given. Traditional routing and traffic engineering enable network providers to stir traffic according to their economic and technical constraints. These solutions are best suited to applications like Web and Email with fixed end-points, but donpsilat interact as well with the newest breed of P2P applications. By selecting selfishly the sources that provide the fastest data rates, such applications affect actively the paths from which data flow in, and thus undermine the value of standard traffic control mechanisms. This can create a state of conflict between ISPs and P2P applications, and lead to drastic measures such as throttling or blocking of P2P connections. In this talk, I will present the potential of less radical Traffic Engineering 2.0 control mechanisms that intervene on the P2P overlay construction in order to leverage unexploited P2P service capacity that sits in more favorable network locations. Using real traffic and topology data from one of the worldpsilas largest ISPs, detailed information from 25,000 swarms with more than one million P2P clients, and a combination of analytic, numeric, and emulation methods, we show that ISPs can reclaim control of their network without severely impacting on their customerpsilas QoS.

For half a century, television has been a dominant and pervasive mass media, driving many technological advances. Despite its widespread usage and importance to emerging applications, the ingrained TV viewing habits are not completely understood. This was primarily due to the difficulty of instrumenting monitoring devices at individual homes at a large scale. The recent boom of Internet TV (IPTV) has enabled us to monitor the user behavior and network usage of an entire network. Such analysis can provide a clearer picture of how people watch TV and how the underlying networks and systems can better adapt to future challenges. In this paper, we present the first analysis of IPTV workloads based on network traces from one of the world's largest IPTV systems. Our dataset captures the channel change activities of 250,000 households over a six month period. We characterize the properties of viewing sessions, channel popularity dynamics, geographical locality, and channel switching behaviors. We discuss implications of our findings on networks and systems, including the support needed for fast channel changes. Our data analysis of an operational IPTV system has important implications on not only existing and future IPTV systems, but also the design of the open Internet TV distribution systems such as Joost and BBC's iPlayer that distribute television on the wider Internet.

We consider wireless mesh networks, and exploit the inherent broadcast nature of wireless by making use of multipath routing. We present an optimization framework that enables us to derive optimal flow control, routing, scheduling, and rate adaptation schemes, where we use network coding to ease the routing problem. We prove optimality and derive a primal-dual algorithm that lays the basis for a practical protocol. We use simulation to show on realistic topologies that we can achieve 20-200% throughput improvement compared to single path routing, and several times compared to a recent related opportunistic protocol (MORE).

Designing high throughput wireless mesh networks involves solving interrelated scheduling, routing, and interference problems. In this paper, we exploit the broadcast properties and the path diversity of wireless meshes to implement an efficient multipath routing protocol, Multipath Code Casting (MC2).

User Generated Content (UGC) is re-shaping the way people watch video and TV, with millions of video producers and consumers. In particular, UGC sites are creating new viewing patterns and social interactions, empowering users to be more creative, and developing new business opportunities. To better understand the impact of UGC systems, we have analyzed YouTube, the world's largest UGC VoD system. Based on a large amount of data collected, we provide an in-depth study of YouTube and other similar UGC systems. In particular, we study the popularity life-cycle of videos, the intrinsic statistical properties of requests and their relationship with video age, and the level of content aliasing or of illegal content in the system. We also provide insights on the potential for more efficient UGC VoD systems (e.g. utilizing P2P techniques or making better use of caching). Finally, we discuss the opportunities to leverage the latent demand for niche videos that are not reached today due to information filtering effects or other system scarcity distortions. Overall, we believe that the results presented in this paper are crucial in understanding UGC systems and can provide valuable information to ISPs, site administrators, and content owners with major commercial and technical implications.

Digital media companies have recently started embracing P2P networks as an alternative distribution mechanism. However, with current P2P swarming systems users need to download the full video, and hence have to wait a long time before they can start watching it. While a lot of effort has gone into optimizing the distribution of large files, little research has been done on enabling Video-on-Demand (VoD) functionality with P2P swarming systems. The main challenges reside in ensuring that users can start watching a movie at any point in time, with small start-up times and sustainable playback rates. In this work, we address the issues of providing VoD using P2P mesh-based networks. We investigate scheduling techniques, and network coding in particular. Using both simulations and a prototype implementation, we show that high-quality VoD is feasible, and give guidelines to build play-as-you-download P2P swarming systems with high playback rates and low start-up delays.

We consider the problem of broadcasting a live stream of data in an unstructured network. The broadcasting problem has been studied extensively for edge-capacitated networks. We give the first proof that whenever demand lambda + epsiv is feasible for epsiv > 0, a simple local-control algorithm is stable under demand lambda, and as a corollary a famous theorem of Edmonds. We then study the node-capacitated case and show a similar optimality result for the complete graph. We study through simulation the delay that users must wait in order to playback a video stream with a small number of skipped packets, and discuss the suitability of our algorithms for live video streaming.

Peer-to-peer technologies are increasingly becoming the medium of choice for delivering media content, both professional and homegrown, to large user populations. Indeed, current P2P swarming systems have been shown to be very efficient for large-scale content distribution with few server resources. However, such systems have been designed for generic file distribution and provide a limited user experience for viewing media content. For example, users need to wait to download the full video before they can start watching it. In general, the main challenge resides in designing systems that ensure that users can start watching a movie at any point in time, with small start-up times and sustainable playback rates. In this work, we address the issues of providing a Video-on-Demand (VoD) using P2P mesh-based networks. We show that providing high quality VoD using P2P is feasible using a combination of techniques including (a) network coding, (b) optimized resource allocation across different parts of the video, and (c) overlay topology management algorithms. Our evaluation also shows that systems that do not optimize in all these dimensions could significantly under-utilize the network resources resulting in poor VoD performance. We present our results based on simulations and a prototype implementation. Categories and Subject Descriptors

The eleven papers in this special issue are devoted to advancements in peer-to-peer streaming systems.

In this paper we present the first implementation of a P2P content distribution system that uses Network Coding. Using results from live trials with several hundred nodes, we provide a detailed performance analysis of such P2P system. In contrast to prior work, which mainly relies on monitoring P2P systems at particular locations, we are able to provide performance results from a variety of novel angles by monitoring all components in the P2P distribution.In particular, we show that Network Coding is practical in a P2P setting since it incurs little overhead, both in terms of CPU processing and IO activity, and it results in smooth, fast downloads, and efficient server utilization. We also study the importance of topology construction algorithms in real scenarios and study the effect of peers behind NATs and firewalls, showing that the system is surprisingly robust to large number of unreachable peers. Finally, we present performance results related to verifying network encoded blocks on-the-fly using special security primitives called Secure-Random-Checksums.

Peer-to-peer content distribution networks can suffer from malicious participants that corrupt content. Current systems verify blocks with traditional cryptographic signatures and hashes. However, these techniques do not apply well to more elegant schemes that use network coding techniques for efficient content distribution. Architectures that use network coding are prone to jamming attacks where the introduction of a few corrupted blocks can quickly result in a large number of bad blocks propagating through the system. Identifying such bogus blocks is difficult and requires the use of homomorphic hashing functions, which are computationally expensive. This paper presents a practical security scheme for network coding that reduces the cost of verifying blocks on-the-fly while efficiently preventing the propagation of malicious blocks. In our scheme, users not only cooperate to distribute the content, but (well-behaved) users also cooperate to protect themselves against malicious users by informing affected nodes when a malicious block is found. We analyze and study such cooperative security scheme and introduce elegant techniques to prevent DoS attacks. We show that the loss in the efficiency caused by the attackers is limited to the effort the attackers put to corrupt the communication, which is a natural lower bound in the damage of the system. We also show experimentally that checking as low as 1-5% of the received blocks is enough to guarantee low corruption rates.

Fast and effective distribution of software updates (a.k.a. patches) to millions of Internet users has evolved into a critical task over the last years. In this paper, we characterize "Windows Update", one of the largest update services in the world, with the aim to draw general guidelines on how to best design and architect a fast and effective planet-scale patch dissemination system. To this end, we analyze an extensive set of data traces collected over the period of a year, consisting of billions of queries from over 300 million computers. Based on empirical observations and analytical results, we identify interesting properties of today's update traffic and user behavior.Building on this analysis, we consider alternative patch delivery strategies such as caching and peer-to-peer and evaluate their performance. We identify key factors that determine the effectiveness of these schemes in reducing the server workload and the network traffic, and in speeding-up the patch delivery. Most of our findings are invariant properties induced by either user behavior or architectural characteristics of today's Internet, and thus apply to the general problem of Internet-wide dissemination of software updates.

Wireless wide area networks (WWANs) are becoming ubiquitous across most geographic regions, enabling simultaneous coverage from multiple providers. WWAN channels exhibit both uncorrelated and correlated behaviour on a variety of levels. In this paper we examine the statistical properties of WWAN links, and illustrate the benefits in heterogeneity that can be exploited to improve statistical throughput and multimedia quality. Our results are based on real network measurements. We describe the design and implementation of a high quality multimedia streaming application that implements WWAN streaming optimisations utilising unequal error protection coding techniques, and we evaluate the performance over an operational WWAN network.

We propose a new scheme for content distribution of large files that is based on network coding. With network coding, each node of the distribution network is able to generate and transmit encoded blocks of information. The randomization introduced by the coding process eases the scheduling of block propagation, and, thus, makes the distribution more efficient. This is particularly important in large unstructured overlay networks, where the nodes need to make block forwarding decisions based on local information only. We compare network coding to other schemes that transmit unencoded information (i.e. blocks of the original file) and, also, to schemes in which only the source is allowed to generate and transmit encoded packets. We study the performance of network coding in heterogeneous networks with dynamic node arrival and departure patterns, clustered topologies, and when incentive mechanisms to discourage free-riding are in place. We demonstrate through simulations of scenarios of practical interest that the expected file download time improves by more than 20-30% with network coding compared to coding at the server only and, by more than 2-3 times compared to sending unencoded information. Moreover, we show that network coding improves the robustness of the system and is able to smoothly handle extreme situations where the server and nodes leave the system.

Recently, there has been an increasing interest in wireless broadcast systems as a means to enable scalable content delivery to large numbers of mobile users. However, gracefully providing efficient reconciliation of different versions of a file over such broadcast channels still remains a challenge. Such systems often lack a feedback channel and consequently updates cannot be easily tailored to a specific user. Moreover, given the potentially large number of possible versions of a file, it is impractical to send a tailored update for each particular user.In this paper we consider the problem of efficiently updating files in such wireless broadcast channels. To this extent, we present DeltaCast, a system that combines hierarchical hashes and erasure codes to minimise the amount of battery power and the amount of time needed to synchronise each mobile device. Based on our experimental results, we show that DeltaCast is able to efficiently identify the missing portions of a file and quickly updated each client.

We present a comparative performance study of a wide selection of optimization techniques to enhance application performance in the context of wide-area wireless networks (WWANs). Unlike in traditional wired and wireless IP-based networks, applications running over WWAN cellular environments are significantly affected by the vagaries of the cellular wireless medium. Prior research has proposed and analyzed optimizations at individual layers of the protocol stack. In contrast, we introduce the first detailed experiment-based evaluation and comparison of all such optimization techniques in a commercial WWAN testbed. This paper, therefore, summarizes our experience in implementing and deploying an infrastructure to improve WWAN performance.The goals of this paper are: (1) to perform an accurate benchmark of application performance over such commercially deployed WWAN environments, (2) to implement and characterize the impact of various optimization techniques across different layers of the protocol stack, and (3) to quantify their interdependencies in realistic scenarios. Additionally, we also discuss measurement pitfalls that we experienced and provide guidelines that may be useful for future experimentation in WWAN environments.

We introduce MAR, a commuter mobile access router infrastructure that exploits wireless diversity (e.g. channel diversity, network diversity, and technology diversity) to provide improved data performance for wireless data users. Our system design stems from the observation that rather than choosing a single wireless service provider (e.g. Sprint, AT&T, BT, Vodafone), a single technology (e.g. GPRS, UMTS, CDMA, 802.11), or a single wireless channel, users can obtain significant benefits by using the multiplicity of choices available. MAR is a wireless multi-homed device that can be placed in moving vehicles (e.g. car, bus, train) to enable high-speed data access. MAR dynamically instantiates new channels based on traffic demand, aggregates the bandwidth and dynamically shifts load from poor quality to better quality channels. MAR, thus, provides a faster, more stable, and reliable communication channel to mobile users.We have implemented and tested the MAR system in our testbed which spans the networks of three different cellular providers. Through our experiments we have performed a detailed evaluation to quantify the benefits of MAR for different protocols and applications. For example, even in highly mobile environments, MAR, on average, improves the end-user experience of web-browsing and streaming applications by a factor of 2.8 and 4.4 respectively. Our results show that significant benefits can be obtained by exploiting the diversity in coverage offered by many cellular operators, different technology networks (e.g. GPRS, CDMA), and diverse wireless channels.

Recent observations through experiments that we have performed in current third generation wireless networks have revealed that the achieved throughput over wireless links varies widely depending on the application. In particular, the throughput achieved by file transfer application (FTP) and web browsing application (HTTP) are quite different. The throughput achieved over a HTTP session is much lower than that achieved over an FTP session. The reason for the lower HTTP throughput is that the HTTP protocol is affected by the large Round-Trip Time (RTT) across Wireless links. HTTP transfers require multiple TCP connections and DNS lookups before a HTTP page can be displayed. Each TCP connection requires several RTTs to fully open the TCP send window and each DNS lookup requires several RTTs before resolving the domain name to IP mapping. These TCP/DNS RTTs significantly degrade the performance of HTTP over wireless links. To overcome these problems, we have developed session level optimization techniques to enhance HTTP download mechanisms. These techniques (a) minimize the number of DNS lookups over the wireless link and (b) minimize the number of TCP connections opened by the browser. These optimizations bridge the mismatch caused by wireless links between application-level protocols (such as HTTP) and transportlevel protocols (such as TCP). Our solutions do not require any client-side software and can be deployed transparently on a service provider network to provide 30 50% decrease in end-to-end user perceived latency and 50-100% increase in data throughput across wireless links for HTTP sessions.

Popular content is frequently replicated in multiple servers or caches in the Internet to offload origin servers and improve end-user experience. However, choosing the best server is a nontrivial task and a bad choice may provide poor end user experience. In contrast to retrieving a file from a single server, we propose a parallel-access scheme where end users access multiple servers at the same time, fetching different portions of that file from different servers and reassembling them locally. The amount of data retrieved from a particular server depends on the resources available at that server or along the path from the user to the server. Faster servers deliver bigger portions of a file while slower servers deliver smaller portions. If the available resources at a server or along the path change during the download of a file, a dynamic parallel access automatically shifts the load from congested locations to less loaded parts (server and links) of the Internet. The end result is that users experience significant speedups and very consistent response times. Moreover, there is no need for complicated server selection algorithms and load is dynamically shared among all servers. The dynamic parallel-access scheme presented does not require any modifications to servers or content and can be easily included in browsers, peer-to-peer applications or content distribution networks to speed up delivery of popular content.

Cache cooperation improves the performance of isolated caches, especially for caches with small cache populations. To make caches cooperate on a large scale and effectively increase the cache population, several caches are usually federated in caching architectures. We discuss and compare the performance of different caching architectures. In particular, we consider hierarchical and distributed caching. We derive analytical models to study important performance parameters of hierarchical and distributed caching, i.e., client's perceived latency, bandwidth usage, load in the caches, and disk space usage. Additionally, we consider a hybrid caching architecture that combines hierarchical caching with distributed caching at every level of a caching hierarchy. We evaluate the performance of a hybrid scheme and determine the optimal number of caches that should cooperate at each caching level to minimize client's retrieval latency.

Popular documents are frequently mirrored on multiple sites in an effort to share the load and reduce clients' retrieval latencies. However, choosing the best mirror site is a non-trivial task and a bad choice may give poor performance. We propose a scheme in which clients access multiple mirror sites in parallel to speedup document downloads while eliminating the problem of server selection. In our scheme, clients connect to mirror sites using unicast TCP connections and dynamically request different pieces of a document from different sites. The amount of data retrieved from a particular site varies depending on the network path/server conditions. Dynamic parallel-access can be easily implemented in the current Internet and does not require any modifications at the mirror sites. Using dynamic parallel-access, all clients experience dramatic speedups in downloading documents, and the load is shared among servers without the need for a server selection mechanism. Even in a situation where clients are connected through modem lines, dynamic parallel-access offers transmission rates at least as high as the fastest server.