Human communication has changed by the advent of smartphones. Using commonplace mobile device features, they started uploading large amounts of content that increases. This increase in demand will overwhelm capacity and limits the providers' ability to provide the quality of service demanded by their users. In the absence of technical solutions, cellular network providers are considering changing billing plans to address this. Our contributions are twofold. First, by analyzing user content upload behavior, we find that the user-generated content problem is a user behavioral problem. Particularly, by analyzing user mobility and data logs of 2 million users of one of the largest US cellular providers, we find that: 1) users upload content from a small number of locations; 2) because such locations are different for users, we find that the problem appears ubiquitous. However, we find that: 3) there exists a significant lag between content generation and uploading times, and 4) with respect to users, it is always the same users to delay. Second, we propose a cellular network architecture. Our approach proposes capacity upgrades at a select number of locations called Drop Zones. Although not particularly popular for uploads originally, Drop Zones seamlessly fall within the natural movement patterns of a large number of users. They are therefore suited for uploading larger quantities of content in a postponed manner. We design infrastructure placement algorithms and demonstrate that by upgrading infrastructure in only 963 base stations across the entire US, it is possible to deliver 50% of content via Drop Zones.

Recent botnets such as Conficker, Kraken, and Torpig have used DNS-based “domain fluxing” for command-and-control, where each Bot queries for existence of a series of domain names and the owner has to register only one such domain name. In this paper, we develop a methodology to detect such “domain fluxes” in DNS traffic by looking for patterns inherent to domain names that are generated algorithmically, in contrast to those generated by humans. In particular, we look at distribution of alphanumeric characters as well as bigrams in all domains that are mapped to the same set of IP addresses. We present and compare the performance of several distance metrics, including K-L distance, Edit distance, and Jaccard measure. We train by using a good dataset of domains obtained via a crawl of domains mapped to all IPv4 address space and modeling bad datasets based on behaviors seen so far and expected. We also apply our methodology to packet traces collected at a Tier-1 ISP and show we can automatically detect domain fluxing as used by Conficker botnet with minimal false positives, in addition to discovering a new botnet within the ISP trace. We also analyze a campus DNS trace to detect another unknown botnet exhibiting advanced domain-name generation technique.

Despite the rapid growth in cellular data traffic, we know very little about the (operational) cellular data service network (CDSN) infrastructure. A key step in the process of developing any such understanding is to first understand the locations and distribution of the basestations in the CDSN infrastructure that serve as physical access points for end users for communicating with the underlying network. Such knowledge not only can provide critical insight into the the CDSN infrastructure, but can also guide the development of innovative (e.g. location-aware) services and applications. In this paper we propose a novel approach for mapping the CDSN basestation infrastructure via (explicit) user geo-intent. The intuition behind the proposed approach is to exploit specific geo-locations (i.e. geo-intent) contained in user queries to location-based services, and correlate them with basestation id's to geo-map the CDSN infrastructure. To investigate the validity of our approach, we employ data (RADIUS/RADA data sessions and application sessions) collected at the core IP network inside a CDSN. We develop heuristics for identifying user geo-intent and for geo-mapping the CDSN infrastructure - in particular, the basestations - and evaluate their efficacy using a subset of basestations with ground-truth GPS locations.

Smartphones have changed the way people communicate. Most prominently, using commonplace mobile device features (e.g., high resolution cameras), they started producing and uploading large amounts of content that increases at an exponential pace. In the absence of viable technical solutions, some cellular network providers are considering to start charging special usage fees to address the problem. Our contributions are twofold. First, we find that the user-generated content problem is a user-behavioral problem. By analyzing user mobility and data logs of close to 2 million users of a cellular network, we find that (i) users upload content from a small number of locations, typically corresponding to their home or work locations; (ii) because such locations are different for different users, we find that the problem appears ubiquitous, since user-generated content uploads grow exponentially at most locations. However, we also find that (Hi) there exists a significant lag between content generation and uploading times. For example, we find that 55% of content that is uploaded via mobile phones is at least 1 day old. Second, based on the above insights, we propose a new cellular network architecture. Our approach proposes capacity upgrades at a select number of locations called Drop Zones. Although not particularly popular for uploads originally, Drop Zones seamlessly fall within the natural movement patterns of a large number of users. They are therefore better suited for uploading larger quantities of content in a postponed manner. We design infrastructure placement algorithms and demonstrate that by upgrading infrastructure in only 963 base-stations across the entire United States, it is possible to deliver 50% of total content via the Drop Zones.

As botnets continue to proliferate and grow in sophistication, so does the need for more advanced security solutions to effectively detect and defend against such attacks. In particular, botnets such as Conficker have been known to encrypt the communication packets exchanged between bots and their command-and-control server, making it costly for existing botnet detection systems that rely on deep packet inspection (DPI) methods to identify compromised machines. In this paper, we argue that, even in the face of encrypted traffic flows, botnets can still be detected by examining the set of server IP-addresses visited by a client machine in the past. However there are several challenges that must be addressed. First, the set of server IP-addresses visited by client machines may evolve dynamically. Second, the set of client machines used for training and their class labels may also change over time. To overcome these challenges, this paper presents a novel incremental LS-SVM algorithm that is adaptive to both changes in the feature set and class labels of training instances. To evaluate the performance of our algorithm, we have performed experiments on two large-scale datasets, including real-time data collected from peering routers at a large Tier-1 ISP. Experimental results showed that the proposed algorithm produces classification accuracy comparable to its batch counterpart, while consuming significantly less computational resources.

Recent Botnets such as Conficker, Kraken and Torpig have used DNS based "domain fluxing" for command-and-control, where each Bot queries for existence of a series of domain names and the owner has to register only one such domain name. In this paper, we develop a methodology to detect such "domain fluxes" in DNS traffic by looking for patterns inherent to domain names that are generated algorithmically, in contrast to those generated by humans. In particular, we look at distribution of alphanumeric characters as well as bigrams in all domains that are mapped to the same set of IP-addresses. We present and compare the performance of several distance metrics, including KL-distance, Edit distance and Jaccard measure. We train by using a good data set of domains obtained via a crawl of domains mapped to all IPv4 address space and modeling bad data sets based on behaviors seen so far and expected. We also apply our methodology to packet traces collected at a Tier-1 ISP and show we can automatically detect domain fluxing as used by Conficker botnet with minimal false positives.

Characterizing the relationship that exists between people's application interests and mobility properties is the core question relevant for location-based services, in particular those that facilitate serendipitous discovery of people, businesses and objects. In this paper, we apply rule mining and spectral clustering to study this relationship for a population of over 280,000 users of a 3G mobile network in a large metropolitan area. Our analysis reveals that (i) People's movement patterns are correlated with the applications they access, e.g., stationary users and those who move more often and visit more locations tend to access different applications. (ii) Location affects the applications accessed by users, i.e., at certain locations, users are more likely to evince interest in a particular class of applications than others irrespective of the time of day. (iii) Finally, the number of serendipitous meetings between users of similar cyber interest is larger in regions with higher density of hotspots. Our analysis demonstrates how cellular network providers and location-based services can benefit from knowledge of the inter-play between users and their locations and interests.

Recently, cellular phone networks have begun allowing third-party applications to run over certain open-API phone operating systems such as Windows Mobile, Iphone and Google's Android platform. However, with this increased openness, the fear of rogue programs written to propagate from one phone to another becomes ever more real. This paper proposes a counter-mechanism to contain the propagation of a mobile worm at the earliest stage by patching an optimal set of selected phones. The counter-mechanism continually extracts a social relationship graph between mobile phones via an analysis of the network traffic. As people are more likely to open and download content that they receive from friends, this social relationship graph is representative of the most likely propagation path of a mobile worm. The counter mechanism partitions the social relationship graph via two different algorithms, balanced and clustered partitioning and selects an optimal set of phones to be patched first as those which have the capability to infect the most number of other phones. The performance of these partitioning algorithms is compared against a benchmark random partitioning scheme. Through extensive trace-driven experiments using real IP packet traces from one of the largest cellular networks in the US, we demonstrate the efficacy of our proposed counter-mechanism in containing a mobile worm.

This paper proposes IPzip, a comprehensive suite of algorithms for compressing IP network packet headers and payloads. We propose an online algorithm for compressing packets in real-time for efficient transfer and an offline algorithm for efficient storage of the network data. In contrast to related approaches, IPzip achieves better compression by exploiting the correlations exhibited by (i) packets that are similar such as those belonging to the same layer-4 flow or those with the same destination port (inter-packet correlation) and (ii) header fields that are correlated to each other (intra-packet correlation). Since reordering of packets and fields is resource intensive, IPzip generates a near-optimal compression plan in an offline phase. Moreover, we propose a methodology to monitor over time the effectiveness of the compression plan being used and switch to a new compression plan when performance of the current compression plan decreases due to changes in the intrinsic traffic structure. Finally, via trace-driven experiments on network traffic obtained from Tier-1 ISPs, we validate that IPzip achieves better performance compared to related approaches.

Though the WWW has come a long way since when it was monikered the World Wide Wait, it is still not reliable during heavy workload conditions. Overloads due to sudden arrival of users (flash crowds) is known to exponentially increase download times. More recently, online banks and portals have been the target of Distributed Denial-of-Service (DDoS) attacks, which send a deluge of requests and drive away the legitimate users. These overloads pose a new set of challenges towards efficient operation at enterprises that host web content which this dissertation addresses by combining knowledge of the network as well as server performance. In particular, this dissertation proposes a web hosting architecture consisting of a grid of clusters, to provide high-performance in the presence of standard overload conditions as well as resilience during attacks. The architecture's high-performance component is provided by a server selection framework which selects the "best server" to serve a request as well as allows for an efficient multiplexing of resources across the entire cluster grid. Traditional approaches assume that minimizing network hop count minimizes client latency. In contrast, the proposed mechanism for server selection collects fine-grained server load and network latency measurements and forwards requests to the server that minimizes the total of estimated network and server delays. The architecture's DDoS- resilience is provided via a combination of anomaly detection and scheduling based mitigation of DDoS attacks. In contrast to prior work, the suspicion mechanism assigns a continuous valued vs. binary suspicion measure to each client session, and the scheduler utilizes these values to determine if and when to schedule a session's requests. Via a combination of analytical modeling and testbed experiments over an online bookstore implementation, the performance benefits achieved by the proposed cluster architecture are justified.

Understanding Internet access trends at a global scale, i.e., what do people do on the Internet, is a challenging problem that is typically addressed by analyzing network traces. However, obtaining such traces presents its own set of challenges owing to either privacy concerns or to other operational difficulties. The key hypothesis of our work here is that most of the information needed to profile the Internet endpoints is already available around us - on the web.

With increasing richness in features such as personalization of content, Web applications are becoming increasingly complex and hence compute intensive. Traditional approaches for improving performance of static content Web sites have been based on the assumption that static content such as images are network intensive. However, these methods are not applicable to the dynamic content applications which are more compute intensive than static content. This paper proposes a suite of algorithms which jointly optimize the performance of dynamic content applications by reducing the client access times while also minimizing the resource utilization. A server migration algorithm allocates servers on-demand within a cluster such that the client access times are not affected even under sudden overload conditions. Further, a server selection mechanism enables statistical multiplexing of resources across clusters by redirecting requests away from overloaded clusters. We also propose a cluster decision algorithm which decides whether to migrate in additional servers at the local cluster or redirect requests remotely under different workload conditions. Through a combination of analytical modeling, trace-driven simulation over traces from large e-commerce sites and testbed implementation, we explore the performance savings achieved by the proposed algorithms.

Enterprise networks are increasingly offloading the responsibility for worm detection and containment to the carrier networks. However, current approaches to the zero-day worm detection problem such as those based on content similarity of packet payloads are not scalable to the carrier link speeds (OC-48 and up-wards). In this paper, we introduce a new system, namely DoWitcher, which in contrast to previous approaches is scalable as well as able to detect the stealthiest worms that employ low-propagation rates or polymorphisms to evade detection. DoWitcher uses an incremental approach toward worm detection: First, it examines the layer-4 traffic features to discern the presence of a worm anomaly; Next, it determines a flow-filter mask that can be applied to isolate the suspect worm flows and; Finally, it enables full-packet capture of only those flows that match the mask, which are then processed by a longest common subsequence algorithm to extract the worm content signature. Via a proof-of-concept implementation on a commercially available network analyzer processing raw packets from an OC-48 link, we demonstrate the capability of DoWitcher to detect low-rate worms and extract signatures for even the polymorphic worms.

Countering Distributed Denial of Service (DDoS) attacks is becoming ever more challenging with the vast resources and techniques increasingly available to attackers. In this paper, we consider sophisticated attacks that are protocol-compliant, non-intrusive, and utilize legitimate application-layer requests to overwhelm system resources. We characterize application-layer resource attacks as either request flooding, asymmetric, or repeated one-shot, on the basis of the application workload parameters that they exploit. To protect servers from these attacks, we propose a counter-mechanism that consists of a suspicion assignment mechanism and a DDoS-resilient scheduler, DDoS Shield. In contrast to prior work, our suspicion mechanism assigns a continuous valued vs. binary measure to each client session, and the scheduler utilizes these values to determine if and when to schedule a session’s requests. Using testbed experiments on a web application, we demonstrate the potency of these resource attacks and evaluate the efficacy of our counter-mechanism. For instance, we effect an asymmetric attack which overwhelms the server resources, increasing the response time of legitimate clients from 0.1 seconds to 10 seconds. Under the same attack scenario, DDoS Shield limits the effects of false-negatives and false-positives and improves the victims’ performance to 0.8 seconds.

Owing to the inter-domain aspects of BGP routing, it is difficult to correlate information across multiple domains in order to analyze the root cause of the routing outages. We present BGP Eye, a tool for visualization-aided root-cause analysis of BGP anomalies. In contrast to previous approaches, BGP Eye performs real-time analysis of BGP anomalies through hierarchical analysis. First, BGP updates are clustered to obtain BGP events that are more representative of an anomaly. These events are then correlated across all border routers to ascertain the extent of the anomaly. Furthermore, BGP Eye provides both the capability to analyze BGP anomalies from an Internet-Centric View through multiple vantage points as well as from a Home-Centric View of a particular Autonomous System. We present the capability for scalable and real-time root-cause analysis provided by BGP Eye through the analysis of two very different anomalies. First, we provide an Internet-Centric view from AS568 of the routing outages during the spread of the Slammer Worm on January 25th, 2003. Second, we provide a Home-Centric view from AS6458 of the routing outages caused by the inadvertent prefix hijacking by AS9121 on December 24th, 2004.

Traditional approaches to mirroring, caching, and content distribution have an underlying assumption that minimizing network hop count minimizes client latency. However, with uncongested backbones and potentially high-latency service times for dynamic content, such techniques are of limited effectiveness. We present an architecture in which dispatchers at an overloaded Internet data center (IDC) redirect requests for dynamic content to a geographically remote but less loaded IDC. We show with both analytical modeling as well as testbed experiments that the delay savings of redirecting requests to a lightly loaded IDC can far outweigh the overhead in interIDC network latency. Consequently, client end-to-end delays are significantly reduced without requiring modifications to clients, servers, or DNS.