Content delivery networks serve a major fraction of the Internet traffic, and their geographically deployed infrastructure makes them a good vantage point to observe traffic access patterns. We perform a large-scale investigation to characterize Web traffic patterns observed from a major CDN infrastructure. Specifically, we discover that responses with application/json content-type form a growing majority of all HTTP requests. As a result, we seek to understand what types of devices and applications are requesting JSON objects and explore opportunities to optimize CDN delivery of JSON traffic. Our study shows that mobile applications account for at least 52% of JSON traffic on the CDN and embedded devices account for another 12% of all JSON traffic. We also find that more than 55% of JSON traffic on the CDN is uncacheable, showing that a large portion of JSON traffic on the CDN is dynamic. By further looking at patterns of periodicity in requests, we find that 6.3% of JSON traffic is periodically requested and reflects the use of (partially) autonomous software systems, IoT devices, and other kinds of machine-to-machine communication. Finally, we explore dependencies in JSON traffic through the lens of ngram models and find that these models can capture patterns between subsequent requests. We can potentially leverage this to prefetch requests, improving the cache hit ratio.

The impressive growth of the mobile Internet has motivated several industry reports retelling the story in terms of number of devices or subscriptions sold per regions, or the increase in mobile traffic, both WiFi and cellular. Yet, despite the abundance of such reports, we still lack an understanding of the impact of cellular networks around the world.

HTTP/2 (h2) was standardized in 2015 as an improvement to HTTP/1.1 (h1) to achieve faster webpage load times (PLTs) [5]. Previous studies have shown both improvement and degradation in PLT when using h2 with respect to h1 [6, 8]. The disagreement about h2 performance from these studies motivates further investigation as to whether and under what conditions h2 brings the performance benefits that were originally envisioned [5].

The transition to IPv6 cellular networks creates uncertainty for content providers (CPs) and content delivery networks (CDNs) of whether and how to follow suit. Do CPs that update their CDN contracts to allow IPv6 hosting achieve better, or worse performance in mobile networks? Should CDNs continue to host mobile content over IPv4 networks, or persuade to their CP customers the performance benefits of IPv6 content delivery?

Modern websites use Content Delivery Networks (CDNs) to speed up the delivery of static content. However, we show that DNS-based selection of CDN servers can be refined to fully deliver on the speedup of CDNs. We propose DNS-Proxy (dp), a client-side process that shares load-balancing functionality with CDNs by choosing from among resolved CDN servers based on last mile network performance. Our measurement study of CDN infrastructure deployed by five major CDN providers shows that dp reduces webpage load time by 29% on average. If dp has already resolved the domain, the reduction in webpage load time is as much as 40%. Finally, dp reduces the load time of individual static Web objects by as much as 43%. We argue that dp enables a more effective use of existing content delivery infrastructure and represents a complementary strategy to a continual increase of geographic content availability.

Netflix and Hulu are leading Over-the-Top (OTT) content service providers in the US and Canada. Netflix alone accounts for 29.7% of the peak downstream traffic in the US in 2011. Understanding the system architectures and performance of Netflix and Hulu can shed light on the design of such large-scale video streaming platforms, and help improving the design of future systems. In this paper, we perform extensive measurement study to uncover their architectures and service strategies. Netflix and Hulu bear many similarities. Both Netflix and Hulu video streaming platforms rely heavily on the third-party infrastructures, with Netflix migrating that majority of its functions to the Amazon cloud, while Hulu hosts its services out of Akamai. Both service providers employ the same set of three content distribution networks (CDNs) in delivering the video contents. Using active measurement study, we dissect several key aspects of OTT streaming platforms of Netflix and Hulu, e.g., employed streaming protocols, CDN selection strategy, user experience reporting, etc. We discover that both platforms assign the CDN to a video request without considering the network conditions and optimizing the user-perceived video quality. We further conduct the performance measurement studies of the three CDNs employed by Netflix and Hulu. We show that the available bandwidths on all three CDNs vary significantly over the time and over the geographic locations. We propose a measurement-based adaptive CDN selection strategy and a multiple-CDN-based video delivery strategy that can significantly increase users' average available bandwidth.

Location based social networks (LBSNs) are becoming increasingly popular with the fast deployment of broadband mobile networks and the growing prevalence of versatile mobile devices. This success has attracted great interest in studying and measuring the characteristics of LBSNs, such as Facebook Places, Yelp, and Google+ Local. However, it is often prohibitive, and sometimes too costly, to obtain a detailed and complete snapshot of a LBSN due to its usually massive scale. In this work, taking Foursquare as an example, we focus on sampling and estimating restricted geographic regions in LBSNs, such as a city or a country. By exploiting the application programming interfaces (APIs) provided by Foursquare for geographic search, we first introduce how to obtain the “ground truth”, namely, a complete set of all venues (i.e., places) in a specified region. Then, we propose random region sampling algorithms that allow us to draw representative samples of venues, and design unbiased estimators of regional characteristics of venues. We validate the efficiency of our sampling algorithms on Foursquare using complete datasets obtained from 12 regions, such as Switzerland, New York City and Los Angeles. Our results are applicable to perform sampling and estimation in all GeoDatabases, such as Facebook Places, Yelp, and Google+ Local, which have similar venue search APIs as Foursquare. These location service providers can also benefit from our results to enable efficient online statistic estimation.

The concept of Software-Defined Networking (SDN) has been successfully applied to data centers and campus networks but it has had little impact in the fixed wireline and mobile telecom domain. Although telecom networks demand fine-granular flow definition, which is one of SDN's principal strengths, the scale of these networks and their legacy infrastructure constraints considerably limit the applicability of SDN principles. Instead, telecom networks resort to tunneling solutions using a plethora of specialized gateway nodes, which create high operation cost and single points of failure. We propose extending the concept of SDN so that it can tackle the challenges of the telecom domain. We see vertical forwarding, i.e. programmable en- and decapsulation operations on top of IF, as one of the fundamental features to be integrated into SDN. We discuss how vertical forwarding enables flow-based policy enforcement, mobility and security by replacing specialized gateways with virtualized controllers and commoditized forwarding elements, which reduces cost while adding robustness and flexibility.

In this paper, we provide a detailed analysis on the venue popularity in Foursquare, a leading location-based social network. By collecting 2.4 million venues from 14 geographic regions all over the world, we study the common characteristics of popular venues, and make the following observations. First, venues with more complete profile information are more likely to be popular. Second, venues in the Food category attract the most (43%) public tips (comments) by users, and the Travel & Transport category is the most popular category with the highest per venue check-ins, i.e., each venue in this category attracts on average 376 check-ins. Moreover, the stickiness of users checking in venues in the residence, office, and school categories is higher than in other categories. Last but not least, in general, old venues created at the early stage of Foursquare are more popular than new venues. Our results help to understand the factors that cause venues to become popular, and have applications in venue recommendations and advertisement in location based social networks.

BitTorrent is both the dominant Peer-to-Peer (P2P) protocol for file-sharing and a nightmare for ISPs due to its network agnostic nature. Many solutions exist to localize BitTorrent traffic relying on cooperation between ISPs and the trackers. Recently, BitTorrent users have been abandoning the trackers in favor of Distributed Hash Tables (DHTs). Despite DHTs are complex heterogeneous systems, DHT-based traffic localization is also possible; however, it is unclear how it performs. The goal of this work is to measure DHT-based traffic localization in the wild. We run multiple experiments involving up to five commercial ISPs and a maximum duration of one month, collecting about 400 GB of BitTorrent traffic. Then, we perform an extensive analysis with the following goals: understand the impact of system parameters, verify accuracy of the measurements, estimate the localization benefits.

The endeavor of this work is to study the impact of content popularity in a large-scale Peer-to-Peer network, namely KAD. Based on an extensive measurement campaign, we pinpoint several deficiencies of KAD in handling popular content and provide a series of improvements to address such shortcomings. Our work reveals that keywords, which are associated with content, may become popular for two distinct reasons. First, we show that some keywords are intrinsically popular because they are common to many disparate contents: in such case we ameliorate KAD by introducing a simple mechanism that identifies stopwords. Then, we focus on keyword popularity that directly relates to popular content. We design and evaluate an adaptive load balancing mechanism that is backward compatible with the original implementation of KAD. Our scheme features the following properties: 1) it drives the process that selects the location of peers responsible to store references to objects, based on object popularity; 2) it solves problems related to saturated peers that would otherwise inflict a significant drop in the diversity of references to objects, and 3) if coupled with a load-aware content search procedure, it allows for a more fair and efficient usage of peer resources.

Netflix is the leading provider of on-demand Internet video streaming in the US and Canada, accounting for 29.7% of the peak downstream traffic in US. Understanding the Netflix architecture and its performance can shed light on how to best optimize its design as well as on the design of similar on-demand streaming services. In this paper, we perform a measurement study of Netflix to uncover its architecture and service strategy. We find that Netflix employs a blend of data centers and Content Delivery Networks (CDNs) for content distribution. We also perform active measurements of the three CDNs employed by Netflix to quantify the video delivery bandwidth available to users across the US. Finally, as improvements to Netflix's current CDN assignment strategy, we propose a measurement-based adaptive CDN selection strategy and a multiple-CDN-based video delivery strategy, and demonstrate their potentials in significantly increasing user's average bandwidth.

We consider a system of compute and storage resources geographically distributed over a large number of locations connected via a wide-area network. By distributing the resources, latency to users can be decreased, bandwidth costs reduced and availablility increased. The challenge is to distribute services with varying characteristics among the data centers optimally. Some services are very latency sensitive, others need vast amounts of storage, and yet others are computationally complex but do not require hard deadlines on execution. We propose efficient algorithms for the placement of services to get the maximum benefit from a distributed cloud systems. The algorithms need input on the status of the network, compute resources and data resources, which are matched to application requirements.

Content-centric networks have been proposed as a new network paradigm that is centered around the distribution of content. A key idea of content-centric networks is to address content by name and to enable nodes in the network to respond to content requests. Most proposals for content-centric networks require a "clean slate" approach and a replacement of today's TCP/IP protocol stack, which raises questions about a feasible deployment path. In this paper, we ask the question to which extent the ideas of content-centric networks can be realized on top of today's IP protocol suite. We explore an approach for name-based addressing that extends today's TCP/IP protocols in a fully standard compliant way. We implement our new method in order to demonstrate its feasibility and evaluate the performance of the system using both latency and processing overhead as measures. The obtained results demonstrate that name-based addressing on IP is feasible. We also acknowledge that content-centric networks designed from the ground up go beyond what can be achieved on top of IP.

The endeavor of this work is to study the impact of content popularity in a large-scale Peer-to-Peer network, namely KAD. Armed with the insights gained from an extensive measurement campaign, which pinpoints several deficiencies of the present KAD design in handling popular objects, we set off to design and evaluate an adaptive load balancing mechanism. Our mechanism is backward compatible with KAD, as it only modifies its inner algorithms, and presents several desirable properties: (i) it drives the process that selects the number and location of peers responsible to store references to objects, based on their popularity; (ii) it solves problems related to saturated peers, that entail a significant drop in the diversity of references to objects, and (iii) if coupled with an enhanced content search procedure, it allows a more fair and efficient usage of peer resources, at a reasonable cost. Our evaluation uses a trace-driven simulator that features realistic peer churn and a precise implementation of the inner components of KAD.

Although Tor is the most widely used overlay for providing anonymity services, its users often experience very high delays. Because much of Tor usage is for Web applications, which are sensitive to latency, it is critical to reduce delays in Tor. To take an important step in this direction, we seek an in-depth understanding of delays in Tor. By taking snapshots of the entire Tor network within a short time window, we are able to study the delay distribution of the entire router population. We also monitor delays introduced by individual Tor routers over extended periods of time. Our results indicate that apart from delays introduced by routers, overlay network latency also plays a significant role in delays in Tor. We have also observed that at any time, there exist huge differences in the delays introduced by different routers. Our results reveal key performance characteristics of Tor system behavior and provide valuable insights for improving the Tor performance.

Many different distributed hash tables (DHTs) have been designed, but only few have been successfully deployed. The implementation of a DHT needs to deal with practical aspects (e.g. related to churn, or to the delay) that are often only marginally considered in the design. In this paper, we analyze in detail the content retrieval process in KAD, the implementation of the DHT Kademlia that is part of several popular peer-to-peer clients. In particular, we present a simple model to evaluate the impact of different design parameters on the overall lookup latency. We then perform extensive measurements on the lookup performance using an instrumented client. From the analysis of the results, we propose an improved scheme that is able to significantly decrease the overall lookup latency without increasing the overhead.

Distributed hash tables (DHTs) have been actively studied in literature and many different proposals have been made on how to organize peers in a DHT. However, very few DHT shave been implemented in real systems and deployed on alarge scale. One exception is KAD , a DHT based on Kademlia, which is part of eDonkey2000, a peer-to-peer file sharing system with several million simultaneous users. We have been crawling KAD continuously for about six months and obtained information about the total number of peers online and their geographical distribution.

We consider the case of a virtual world of peers that are organized in an overlay built by Delaunay Triangulation. Application layer routing is used to determine the path taken in the overlay between two peers. Application layer routing incurs a major delay penalty since it ignores the characteristics of the physical network topology.

We present a distributed algorithm for the clustering of peers in a Networked Virtual Environment (NVE) that are organized using a peer-to-peer (P2P) network based on the Delaunay triangulation. The algorithm is dynamic in the sense that whenever a peer joins or leaves the NVE, the clustering will be adapted if necessary by either splitting a cluster or merging clusters. The main idea of the algorithm is to classify links between adjacent peers into short intra-cluster and long inter-cluster links. The advantages of clustering are multiple: clustering allows to limit queries to the peers of a cluster avoiding to flood the entire network. Since clusters can be seen as a level of abstraction that reduces the amount of information/detail exposed about the NVE, clustering allows for faster navigation in the NVE and reduces the number of messages a node receives when he travels through the NVE.

This paper presents a dynamic and distributed algorithm for the clustering of peers in a Networked Virtual Environment (NVE) based on a fully distributed peer-to-peer (P2P) network.The main idea is to classify connections in short intra-cluster connections and long inter-cluster connections. The insertion of new peers or the deletion of existing peers can result in the merging of two clusters or the split of one cluster in two parts.