FBI's Operation Ghost Click, the largest cybercriminal takedown in history, recently took down an ad fraud infrastructure that affected 4 million users and made its owners 14 million USD over a period of four years. The attackers hijacked clicks and ad impressions on victim machines infected by a DNS changer malware to earn ad revenue fraudulently. We experimented with the attack infrastructure when it was in operation and present a detailed account of the attackers' modus operandi. We also study the impact of this attack on real-world users and find that 37 subscriber lines were impacted in our data set. Also, 20 ad networks and 257 legitimate Web content publishers lost ad revenue while the attackers earned revenue convincing a dozen other ad networks that their ads were served on websites with real visitors. Our work expands the understanding of modalities of ad fraud and could help guide appropriate defense strategies.

Recent trends in the networked services industry (e.g., CDN, VPN, VoIP, IPTV) see Internet Service Providers (ISPs) leveraging their existing network connectivity to provide an end-to-end solution. Consequently, new opportunities are available to monitor and improve the end-to-end service quality by leveraging the information from inside the network. We propose a new approach to detect and localize end-to-end service quality issues in such ISP-managed networked services by utilizing traffic data passively monitored at the ISP side, the ISP network topology, routing tables and geographic information. This paper presents the design of a generic service quality monitoring system “Argus”. Argus has been successfully deployed in a tier-1 ISP to monitor millions of users of its CDN service and assist operators to detect and localize end-to-end service quality issues. This operational experience demonstrates that Argus is effective in accurate, quick detection and localization of important service quality issues.

With the recent advent of 4G LTE networks, there has been increasing interest to better understand the performance and power characteristics, compared with 3G/WiFi networks. In this paper, we take one of the first steps in this direction.

Web caching in mobile networks is critical due to the unprecedented cellular traffic growth that far exceeds the deployment of cellular infrastructures. Caching on handsets is particularly important as it eliminates all network-related overheads. We perform the first network-wide study of the redundant transfers caused by inefficient web caching on handsets, using a dataset collected from 3 million smartphone users of a large commercial cellular carrier, as well as another five-month-long trace contributed by 20 smartphone users. Our findings suggest that redundant transfers contribute 18% and 20% of the total HTTP traffic volume in the two datasets. Also they are responsible for 17% of the bytes, 7% of the radio energy consumption, 6% of the signaling load, and 9% of the radio resource utilization of all cellular data traffic in the second dataset. Most of such redundant transfers are caused by the smartphone web caching implementation that does not fully support or strictly follow the protocol specification, or by developers not fully utilizing the caching support provided by the libraries. This is further confirmed by our caching tests of 10 popular HTTP libraries and mobile browsers. Improving the cache implementation will bring considerable reduction of network traffic volume, cellular resource consumption, handset energy consumption, and user-perceived latency, benefiting both cellular carriers and customers.

Cellular networks employ a specific radio resource management policy distinguishing them from wired and Wi-Fi networks. A lack of awareness of this important mechanism potentially leads to resource-inefficient mobile applications. We perform the first network-wide, large-scale investigation of a particular type of application traffic pattern called periodic transfers where a handset periodically exchanges some data with a remote server every t seconds. Using packet traces containing 1.5 billion packets collected from a commercial cellular carrier, we found that periodic transfers are very prevalent in today's smartphone traffic. However, they are extremely resource-inefficient for both the network and end-user devices even though they predominantly generate very little traffic. This somewhat counter-intuitive behavior is a direct consequence of the adverse interaction between such periodic transfer patterns and the cellular network radio resource management policy. For example, for popular smartphone applications such as Facebook, periodic transfers account for only 1.7% of the overall traffic volume but contribute to 30% of the total handset radio energy consumption. We found periodic transfers are generated for various reasons such as keep-alive, polling, and user behavior measurements. We further investigate the potential of various traffic shaping and resource control algorithms. Depending on their traffic patterns, applications exhibit disparate responses to optimization strategies. Jointly using several strategies with moderate aggressiveness can eliminate almost all energy impact of periodic transfers for popular applications such as Facebook and Pandora.

Cellular networks have witnessed tremendous traffic growth recently, fueled by smartphones, tablets and new high speed broadband cellular access technologies. A key application driving that growth is video streaming. Yet very little is known about the characteristics of this traffic class. In this paper, we examine video traffic generated by three million users across one of the world's largest 3G cellular networks. This first deep dive into cellular video streaming shows that HLS, an adaptive bitrate streaming protocol, accounts for one third of the streaming video traffic and that it is common to see changes in encoding bitrates within a session. We also observe that most of the content is streamed at less than 255 Kbps and that only 40% of the videos are fully downloaded. Another key finding is that there exists significant potential for caching to deliver this content.

Smartphone users are increasingly shifting to using apps as "gateways" to Internet services rather than traditional web browsers. App marketplaces for iOS, Android, and Windows Phone platforms have made it attractive for developers to deploy apps and easy for users to discover and start using many network-enabled apps quickly. For example, it was recently reported that the iOS AppStore has more than 350K apps and more than 10 billion downloads. Furthermore, the appearance of tablets and mobile devices with other form factors, which also use these marketplaces, has increased the diversity in apps and their user population. Despite the increasing importance of apps as gateways to network services, we have a much sparser understanding of how, where, and when they are used compared to traditional web services, particularly at scale. This paper takes a first step in addressing this knowledge gap by presenting results on app usage at a national level using anonymized network measurements from a tier-1 cellular carrier in the U.S. We identify traffic from distinct marketplace apps based on HTTP signatures and present aggregate results on their spatial and temporal prevalence, locality, and correlation.

Effective management of large-scale cellular data networks is critical to meet customer demands and expectations. Customer calls for technical support provide direct indication as to the problems customers encounter. In this paper, we study the customer tickets - free-text recordings and classifications by customer support agents - collected at a large cellular network provider, with two inter-related goals: i) to characterize and understand the major factors which lead to customers to call and seek support; and ii) to utilize such customer tickets to help identify potential network problems. For this purpose, we develop a novel statistical approach to model customer call rates which account for customer-side factors (e.g., user tenure and handset types) and geo-locations. We show that most calls are due to customer-side factors and can be well captured by the model. Furthermore, we also demonstrate that location-specific deviations from the model provide a good indicator of potential network-side issues.

Despite the popularity of mobile applications, their performance and energy bottlenecks remain hidden due to a lack of visibility into the resource-constrained mobile execution environment with potentially complex interaction with the application behavior. We design and implement ARO, the mobile Application Resource Optimizer, the first tool that efficiently and accurately exposes the cross-layer interaction among various layers including radio resource channel state, transport layer, application layer, and the user interaction layer to enable the discovery of inefficient resource usage for smartphone applications. To realize this, ARO provides three key novel analyses: (i) accurate inference of lower-layer radio resource control states, (ii) quantification of the resource impact of application traffic patterns, and (iii) detection of energy and radio resource bottlenecks by jointly analyzing cross-layer information. We have implemented ARO and demonstrated its benefit on several essential categories of popular Android applications to detect radio resource and energy inefficiencies, such as unacceptably high (46%) energy overhead of periodic audience measurements and inefficient content prefetching behavior.

No abstract available.

Operators of 3G data networks need to distinguish the performance of each geographic area in their 3G networks to detect and resolve local network problems. This is because the quality of the last mile radio link between 3G base stations and end-user devices is a crucial factor in the end-to-end performance that each user experiences. It is relatively straightforward to measure the performance of all IP traffic in the 3G network from a small number of vantage points in the core network. However, the location information available about each mobile device (e.g., the cell sector/site that it is in) is often too stale to be accurate because of user mobility. Moreover, very costly infrastructure deployment and maintenance of custom equipment would be required to collect fine-grained location information about all mobile devices on an on-going basis in large 3G networks. Thus, it is a challenge to accurately assign IP performance measurements to fine-grained geographic regions of the 3G network using existing standard network components. Fortunately, previous studies have observed that human mobility patterns are very predictable. In this paper, we exploit this predictability to develop a novel clustering algorithm grouping related cell sectors that accurately assigns IP performance measurements to fine-grained geographic regions. We present results from a prototype in a real 3G network that shows our approach provides more accurate performance localization than existing approaches. Eventually, we can either narrow down individual IP performance measurements into only 4 candidate cell sectors consistently with the accuracy of 70% over one week based on a one-day snapshot of fine-grained 3GPP events, or increase the accuracy 20% comparing with site-level accuracy through lightweight handover statistics hourly collected at RNCs. Using our approach, we improve anomaly detection based on IP performance measurements by reducing the number of false positives and false negatives. Our study also sheds light on the mobility patterns of 3G devices.

Despite the tremendous growth in the cellular data network usage due to the popularity of smartphones, so far there is rather limited understanding of the network infrastructure of various cellular carriers. Understanding the infrastructure characteristics such as the network topology, routing design, address allocation, and DNS service configuration is essential for predicting, diagnosing, and improving cellular network services, as well as for delivering content to the growing population of mobile wireless users. In this work, we propose a novel approach for discovering cellular infrastructure by intelligently combining several data sources, i.e., server logs from a popular location search application, active measurements results collected from smartphone users, DNS request logs from a DNS authoritative server, and publicly available routing updates. We perform the first comprehensive analysis to characterize the cellular data network infrastructure of four major cellular carriers within the U.S. in our study.

Traditional DSL troubleshooting solutions are reactive, relying mainly on customers to report problems, and tend to be labor-intensive, time consuming, prone to incorrect resolutions and overall can contribute to increased customer dissatisfaction. In this paper, we propose a proactive approach to facilitate troubleshooting customer edge problems and reducing customer tickets. Our system consists of: i) a ticket predictor which predicts future customer tickets; and ii) a trouble locator which helps technicians accelerate the troubleshooting process during field dispatches. Both components infer future tickets and trouble locations based on existing sparse line measurements, and the inference models are constructed automatically using supervised machine learning techniques. We propose several novel techniques to address the operational constraints in DSL networks and to enhance the accuracy of NEVERMIND. Extensive evaluations using an entire year worth of customer tickets and measurement data from a large network show that our method can predict thousands of future customer tickets per week with high accuracy and signifcantly reduce the time and effort for diagnosing these tickets. This is benefcial as it has the effect of both reducing the number of customer care calls and improving customer satisfaction.

In 3G cellular networks, the release of radio resources is controlled by inactivity timers. However, the timeout value itself, also known as the tail time, can last up to 15 seconds due to the necessity of trading off resource utilization efficiency for low management overhead and good stability, thus wasting considerable amount of radio resources and battery energy at user handsets. In this paper, we propose Tail Optimization Protocol (TOP), which enables cooperation between the phone and the radio access network to eliminate the tail whenever possible. Intuitively, applications can often accurately predict a long idle time. Therefore the phone can notify the cellular network on such an imminent tail, allowing the latter to immediately release radio resources. To realize TOP, we utilize a recent proposal of 3GPP specification called fast dormancy, a mechanism for a handset to notify the cellular network for immediate radio resource release. TOP thus requires no change to the cellular infrastructure and only minimal changes to smartphone applications. Our experimental results based on real traces show that with a reasonable prediction accuracy, TOP saves the overall radio energy (up to 17%) and radio resources (up to 14%) by reducing tail times by up to 60%. For applications such as multimedia streaming, TOP can achieve even more significant savings of radio energy (up to 60%) and radio resources (up to 50%).

The reconvergence of routing protocols in response to changes in network topology can impact application performance. While improvements in protocol specification and implementation have significantly reduced reconvergence times, increasingly performance-sensitive applications continue to raise the bar for these protocols. As such, monitoring the performance of routing protocols remains a critical activity for network operators. We design tool{}, a tool based on passive data plane measurements that we use in conjunction with control plane monitors for offline debugging and analysis of forwarding table dynamics. We discuss practical constraints that affect tool{}, and show how they can be addressed in real deployment scenarios. As an application of tool{}, we study forwarding table updates by backbone routers at a tier-1 ISP. We detect interesting behavior such as delayed forwarding table updates and routing loops due to buggy routers -- confirmed by network operators -- that are not detectable using traditional control plane monitors.

How fast is the network? The speed at which real users can download content at different locations and at different times is an important metric for service providers. Knowledge of this speed helps determine where to provision more capacity and helps detect network problems. However, most network-level estimates of these speeds today are obtained using active "speed tests" that place substantial load on the network and are not necessarily representative of actual user experiences due to limited vantage points. These problems are exacerbated in wireless networks where the physical locations of users play an important role in performance. To redress these problems, this paper presents a new technique to estimate achievable download speed using only flow records collected passively. Estimating achievable speed passively is non-trivial because the measured throughput of real flows is often not comparable to the achievable steady-state TCP rate. This can be because, for example, flows are small and never exit TCP slow start or are rate-limited by the content-provider. Our technique addresses these issues by constructing a ThroughputIndex, a list of flow types that accurately estimate achievable speed. We show that our technique estimates achievable speed more accurately than other techniques in a large 3G wireless network.

3G cellular data networks have recently witnessed explosive growth. In this work, we focus on UMTS, one of the most popular 3G mobile communication technologies. Our work is the first to accurately infer, for any UMTS network, the state machine (both transitions and timer values) that guides the radio resource allocation policy through a light-weight probing scheme. We systematically characterize the impact of operational state machine settings by analyzing traces collected from a commercial UMTS network, and pinpoint the inefficiencies caused by the interplay between smartphone applications and the state machine behavior. Besides basic characterizations, we explore the optimal state machine settings in terms of several critical timer values evaluated using real network traces. Our findings suggest that the fundamental limitation of the current state machine design is its static nature of treating all traffic according to the same inactivity timers, making it difficult to balance tradeoffs among radio resource usage efficiency, network management overhead, device radio energy consumption, and performance. To the best of our knowledge, our work is the first empirical study that employs real cellular traces to investigate the optimality of UMTS state machine configurations. Our analysis also demonstrates that traffic patterns impose significant impact on radio resource and energy consumption. In particular, We propose a simple improvement that reduces YouTube streaming energy by 80% by leveraging an existing feature called fast dormancy supported by the 3GPP specifications.

Enterprise networks are increasingly adopting Layer-3 multiprotocol label switching (MPLS) virtual private network (VPN) technology to connect geographically disparate locations. The any-to-any direct connectivity model of this technology is causing routing tables in the service provider's routers to grow very large. The concept of relaying was proposed earlier to separately minimize the routing table memory footprint of individual VPNs by selecting a small number of hub routers to maintain complete reachability information for each VPN and enabling nonhub spoke routers with reduced routing tables to reach others by routing traffic via a hub. A large service provider network typically hosts thousands of different VPNs. In this paper, we generalize relaying to the multi-VPN environment and consider new constraints on resources shared across VPNs, such as router uplink bandwidth and memory. The hub selection problem involves complex tradeoffs along multiple dimensions including these shared resources and the additional distance traversed by traffic. We formulate the hub selection as a constraint optimization problem and develop an algorithm with provable guarantees to approximate this NP-complete problem. Evaluations using traces and configurations from a large provider indicate that the resulting relaying solution reduces the total router memory requirement by 85% while smoothing out the utilization on each router and requiring only a small increase in the end-to-end path for the relayed traffic.

Since the last in-depth studies of measured TCP traffic some 6-8 years ago, the Internet has experienced significant changes, including the rapid deployment of backbone links with 1-2 orders of magnitude more capacity, the emergence of bandwidth-intensive streaming applications, and the massive penetration of new TCP variants. These and other changes beg the question whether the characteristics of measured TCP traffic in today's Internet reflect these changes or have largely remained the same. To answer this question, we collected and analyzed packet traces from a number of Internet backbone and access links, focused on the "heavy-hitter" flows responsible for the majority of traffic. Next we analyzed their within-flow packet dynamics, and observed the following features: (1) in one of our datasets, up to 15.8% of flows have an initial congestion window (ICW) size larger than the upper bound specified by RFC 3390. (2) Among flows that encounter retransmission rates of more than 10%, 5% of them exhibit irregular retransmission behavior where the sender does not slow down its sending rate during retransmissions. (3) TCP flow clocking (i.e., regular spacing between flights of packets) can be caused by both RTT and non-RTT factors such as application or link layer, and 60% of flows studied show no pronounced flow clocking. To arrive at these findings, we developed novel techniques for analyzing unidirectional TCP flows, including a technique for inferring ICW size, a method for detecting irregular retransmissions, and a new approach for accurately extracting flow clocks.

Enterprise networks are increasingly adopting layer 3 multiprotocol label switching (MPLS) virtual private network (VPN) technology to connect geographically disparate locations. The any-to-any direct connectivity model of this technology involves a very high memory footprint and is causing associated routing tables in the service provider's routers to grow very large. The concept of relaying was proposed earlier [6] to separately minimize the routing table memory footprint of individual VPNs, and involves selecting a small number of hub routers to maintain complete reachability information for that VPN, and enabling non-hub spoke routers with reduced routing tables to achieve any-to-any reachability by routing traffic via a hub. A large service provider network typically hosts many thousands of different VPNs. In this paper, we generalize relaying to the multi-VPN environment, and consider new constraints on resources shared across VPNs, such as router uplink bandwidth and memory. The hub selection problem involves complex tradeoffs along multiple dimensions including these shared resources, and the additional distance traversed by traffic. We formulate the hub selection as a constraint optimization problem and develop an algorithm with provable guarantees to solve this NP-complete problem. Evaluations using traces and configurations from a large provider and many real-world VPNs indicate that the resulting Relaying solution substantially reduces the total router memory requirement by 85% while smoothing out the utilization on each router and requiring only a small increase in the end-to-end path for the relayed traffic.

This paper proposes and evaluates a Network Aware Forward Caching approach for determining the optimal deployment strategy of forward caches to a network. A key advantage of this approach is that we can reduce the network costs associated with forward caching to maximize the benefit obtained from their deployment. We show in our simulation that a 37% increase to net benefits could be achieved over the standard method of full cache deployment to cache all POPs traffic. In addition, we show that this maximal point occurs when only 68% of the total traffic is cached.

Enterprise customers are increasingly adopting MPLS (Multiprotocol Label Switching) VPN (Virtual Private Network) service that offers direct any-to-any reachability among the customer sites via a provider network. Unfortunately this direct reachability model makes the service provider's routing tables grow very large as the number of VPNs and the number of routes per customer increase. As a result, router memory in the provider's network has become a key bottleneck in provisioning new customers. This paper proposes Relaying, a scalable VPN routing architecture that the provider can implement simply by modifying the configuration of routers in the provider network, without requiring changes to the router hardware and software. Relaying substantially reduces the memory footprint of VPNs by choosing a small number of hub routers in each VPN that maintain full reachability information, and by allowing non-hub routers to reach other routers through a hub. Deploying Relaying in practice, however, poses a challenging optimization problem that involves minimizing router memory usage by having as few hubs as possible, while limiting the additional latency due to indirect delivery via a hub. We first investigate the fundamental tension between the two objectives and then develop algorithms to solve the optimization problem by leveraging some unique properties of VPNs, such as sparsity of traffic matrices and spatial locality of customer sites. Extensive evaluations using real traffic matrices, routing configurations, and VPN topologies demonstrate that Relaying is very promising and can reduce routing-table usage by up to 90%, while increasing the additional distances traversed by traffic by only a few hundred miles, and the backbone bandwidth usage by less than 10%.

Reliability is a critical requirement of the Internet. The availability and resilience of the Internet under failures can have significant global effects. However, in the current Internet routing architecture, achieving the high level of reliability demanded by many mission critical activities can be costly. In this paper, we first propose a novel solution framework called reliability as an interdomain service (REIN) that can be incrementally deployed in the Internet and may improve the redundancy of IP networks at low cost. We then present robust algorithms to efficiently utilize network redundancy to improve reliability. We use real IP network topologies and traffic traces to demonstrate the effectiveness of our framework and algorithms.

The management of communication networks increasingly requires detailed knowledge of network usage, acquired by direct measurement. We report on the design and implementation of a backend system for trajectory sampling, a method for consistent sampling of packets across a network domain. This trajectory engine collects trajectory samples and stores them after appropriate preprocessing. It provides a querying and visualization tool to aid in traffic engineering and troubleshooting. We describe the entire system, and in particular the design choices that we took in order to balance the scale of the system (due to large volumes of measured data) with resource usage while providing useful functionality for users. In the preprocessing stage, we focus on reassembly of trajectories from individual samples. We describe the design of the database for efficient storage of trajectories, and its relationship with the query and visualization interface. We test the system using a synthetic stream of trajectory samples derived from configuration and usage data from the network of a major service provider. We walk through several examples that illustrate how a network operator might take advantage of trajectory sampling through such a tool.