Today, large AI training jobs crash for a broad range of reasons and recovery often involves significant amounts of human intervention. At Meta, we observed that 5-20% of our job failures are actually due to kernel bugs in NIC drivers. Unlike other class of bugs, these are challenging to diagnose because we lack visibility into this aspect of the kernel.

Existing research on Content Delivery Networks (CDNs) predominantly focuses on optimizing the delivery of hot content—popular items that attract frequent and repeated access from large user bases. However, at Meta, we have identified that cold content, which is less popular and accessed infrequently, poses significant challenges for user experience, especially with large volumes of direct messaging media. Through extensive analysis of Meta's CDN data, we quantify the impact of cold content performance on user experience and reveal substantial differences across geographic regions. Motivated by these findings, we propose CacheFlare, a suite of production-deployed solutions designed to enhance Quality of Experience (QoE) by increasing CDN hit rates for cold content. Through experiments and production validation, we show that even cold content can benefit from improved caching strategies, yielding up to a 10% improvement across a range of network performance metrics.

Despite the widespread adoption of domain name encryption protocols (e.g., DoH, DoT, and ECH), website fingerprinting attacks remain a significant threat to online privacy due to the visibility of IP connections that can be observed by network-level adversaries. This problem has been investigated in IPv4 thoroughly but remains largely unexplored in IPv6, where it is even more pronounced. From a design perspective, IPv6 offers a vastly larger address space, eliminating the need for virtual hosting and domain co-location -- practices prevalent in IPv4. These practices obscure several domains behind a single IPv4 address. The adoption of IPv6 can theoretically lead to more accurate fingerprinting based on IPv6 connections because each domain is now more likely to be resolved to a unique globally routable IPv6 address unlike IPv4. In this study, we systematically investigate the feasibility, accuracy, and privacy implications of IP-based website fingerprinting in IPv6 environments. Utilizing empirical data collected via active DNS measurements, Web crawling, and entropy-based analyses across half a million dual-stack websites, we conduct the largest evaluation of website fingerprinting in IPv6. We find that dual-stack websites that still have some IPv4-only dependencies are easier to fingerprint, achieving close to 94% accuracy on both IPv4 and IPv6. However, fingerprinting pure dual-stack websites is significantly harder: accuracy drops to 56% over IPv4 and 45% over IPv6. These results reveal distinct trends in hosting infrastructures and indicate that IPv6 itself does not inherently diminish privacy, contrary to existing concerns in the community. Rather, fingerprinting risk is shaped by how hosting providers deploy IPv6 and their choices regarding domain co-location.

Despite decades of investment and regulatory efforts, Africa's Internet ecosystem still relies heavily on infrastructure far outside the continent, routing traffic through Europe and outsourcing critical services like DNS resolution. This dependence on foreign infrastructure exacerbates the impact of subsea cable cuts and exposes a deeper problem. That is, the continent's connectivity fabric remains externally dependent and structurally fragile. Existing measurement tools fail to illuminate this reality, either by missing key components or offering insufficient visibility. We argue for a rethinking of how we monitor and support Africa's Internet infrastructure. Our vision is a purpose-built testbed that combines crowd-sourced vantage points with intentional, context-aware targeting to better capture the unique Internet ecosystem of the continent. Rather than simply retrofitting global solutions, we propose building with the realities of Africa's ecosystem in mind.

Deep Reinforcement Learning-based Congestion Control Algorithms (DRL-based CCA) have shown their great potential to adapt to various environments automatically (e.g., Orca). However, it is difficult for existing DRL-based CCAs to achieve superior QoS consistently, particularly over mobile networks with rapid network fluctuations. The fundamental problem stems from the training of a single model that encompasses a wide range of network conditions. The resulting model can be overly generalized, rendering it less accurately or optimally tailored for a specific network condition. To tackle this challenge, we develop Network-segmented Model Specialization (NMS), a framework that automatically maximizes QoS for any DRLbased CCA under different network conditions. Specifically, NMS generates a set of models offline, each trained for a specific network segment. Online, it selects the model based on the current segment. We showed that NMS not only improves the QoSs of existing DRL-based CCAs consistently, but also opens a new way for the exploration of a clean-slate approach, as opposed to following the hybrid TCP/DRL approach. Hence, we designed Galaxy, a novel clean-slate DRL-based CCA that addresses the inherent limitations in clean-slate approaches by incorporating network segments' knowledge. Extensive evaluations show NMSoptimized Galaxy further explores NMS to achieve superior QoS.

Congestion control mechanisms limit the end hosts' transmission rates, preventing the depletion of network resources. Typically, these systems focus only on fair resource allocation between competing flows. However, thinking only about fairness is insufficient: novel ML-powered congestion control mechanisms may be unpredictable, and even some of the already widely adopted congestion control mechanisms may still harm the network operation in some scenarios. In this context, it is important to also think about the safety of those systems. In this work, we present and analyze formally the concept of safety in congestion control. We use this formalization to design and implement SafeNetCC, a system capable of improving safety of potentially unsafe congestion control mechanisms. Our preliminary results show that SafeNetCC can improve network safety, reducing delays and improving fairness, even when incompatible congestion controls coexist in the same network. In an emulated scenario, we observed an RTT reduction of up to 80% and fairness improvements for incompatible congestion controls.

The Web has shifted towards a mobile-first ecosystem with tools, frameworks, and forums explicitly discussing and catering for the mobile users, both mobile apps and mobile web-pages. Unfortunately, much of the studies and designs are often based on analysis and findings from developed regions (e.g., N. America and Europe) or based on user-generated data (introducing bias). In this paper, we present one of the first studies to understand the interplay between hardware characteristics (e.g., cellular and mobile) on expected network and application level performance in Brazil (the largest developing region in S. America). We analyze more than 170 million measurement sessions collected from within the network of one of the largest Mobile Network Operators in Brazil. Our findings (1) illustrate limitations of existing crowdsourced measurements and inaccuracies in assumptions about adoption patterns and performance in the global south, (2) highlight the differences between recommendations made by standardization bodies and real world performance, (3) disclose a significant change pre- and post-pandemic, and (4) quantify the benefits of using both client side and network data for analysis.

Africa lags behind in broadband connectivity and performance. The internet cost remains significantly higher compared to the Global North, and users experience network delays, availability issues, and slow speeds. Several studies demonstrated that measuring internet performance in Africa will provide a better understanding of the reasons underpinning poor broadband performance and ultimately driving substantial improvements. While internet measurement is critical on the continent, effective measurement requires robust infrastructure. We recognize that robust network measurement is expensive but need to be context-based and involve all the internet stakeholders from users to regulators to service providers and more. However, current measurement tools are mostly designed in the West and do not necessarily reflect Africa's broadband measurement challenges. To address this, we propose in this work a cost-effective and context-based mobile broadband measurement infrastructure.

Although the Internet ecosystem in East Africa has experienced significant growth as evidenced by the 115% increment in Internet users in Sub-Sharan Africa, we understand very little about the infrastructure's performance which is crucial, particularly in DNS resolution, ISP peering, and the use of IXPs. Previous studies highlighting the importance of IXPs and their positive impact on connectivity are qualitative with limited participants, limiting their completeness. To achieve a nuanced understanding of the East African Internet landscape, a more extensive and diverse sample size is needed, integrating both context from qualitative with the scale from quantitative methods. Additionally, prior studies focus on the period following the installation of submarine fiber-optic cables, which may not fully capture the current state of the Internet ecosystem. This study aims to use a mixed methods approach to provide a comprehensive assessment of the Internet infrastructure in East Africa.

Managing the performance of thousands of services across millions of servers demands a networking stack that can dynamically adjust protocol settings to match diverse priorities and network characteristics. Moreover, given the constantly evolving nature of services and their requirements, the set of configurable protocols must remain adaptable. However, current host networking stacks lack the necessary flexibility and adaptability. Although eBPF shows promise in this regard, it lacks essential primitives for efficient development and safe deployment of multiple co-existing services.

There is a growing move to offload functionality, e.g., TCP or key-value stores, into programmable networks - either on SmartNICs or programmable switches. While offloading promises significant performance boosts, these programmable devices often provide little visibility into their performance. Moreover, many existing tools for analyzing and debugging performance problems, e.g., distributed tracing, do not extend into these devices.

Network function (NF) migration alongside (and possibly because of) routing policy updates is a delicate task, making it difficult to ensure that all traffic is processed by its required network functions, in order. Indeed, all previous solutions to this problem adapt routing policy only after NFs have been migrated, in a serial fashion. This paper proposes a design called Nimble for interleaving these tasks to complete both more efficiently while ensuring complete processing of traffic by the required NFs, provided that the route-update protocol enforces a specific property that we define. We demonstrate the benefits of the Nimble design using an implementation in Open vSwitch and the Ryu controller, building on both known routing update protocols and a new protocol of our design that implements specifically the needed property.

Learning-based approaches have shown their great potential to adapt themselves to various environments (e.g., PCC and Sprout). Unfortunately, they do not consistently achieve superior QoS across different network conditions and configurations in mobile networks. Furthermore, although they can offer multiple application objectives by adjusting a preference weight vector, it is challenging for users to accurately express an application objective with a weight vector. In this work, we argue that, if configured correctly, the delay-based TCP scheme can outperform learned ones, and allow users to directly specify their objectives. To this end, we propose Post-QoS Analysis (PQSA), a data-driven framework that trains the key QoS-impacting parameters of the scheme to capture the statistical correlations between QoS objectives, network conditions, and configurations, thereby determining the optimal parameter-set that meets the user-defined QoS objective under different network conditions and configurations. To support this, we enhance conventional delay-based TCP design to develop a Generalized TCP-like Rate controller (GR) by exporting three key parameters. Extensive evaluations show that PQSA-optimized GR outperforms existing schemes in different scenarios consistently, and enables service providers to control the QoS flexibly.

Microservices are terraforming the computing landscape with web-scale infrastructures (e.g., Facebook, Google, Amazon) and telecom infrastructures (e.g., ATT, Ericsson) adopting them. At it’s core, the microservices paradigm promotes a decoupling of applications into multiple services – a decoupling that promotes better scalability, fault-tolerance, and deployability. Unfortunately, this decoupling significantly increases the space of configuration options and performance problems, rendering traditional approaches to management ineffective. Recent efforts to address this problem embrace Artificial Intelligence for IT Operations (AIOps). However, training effective AI models requires significant amounts of data and, in some instances, a framework for quickly exploring or analyzing model performance. Digital twins, or simulators, have effectively enabled AI-based management frameworks within other domains (e.g., manufacturing, industrial and automotive).

IoT devices are increasingly being implicated in cyber-attacks, raising community concern about the risks they pose to critical infrastructure, corporations, and citizens. In order to reduce this risk, the IETF is pushing IoT vendors to develop formal specifications of the intended purpose of their IoT devices, in the form of a Manufacturer Usage Description (MUD), so that their network behavior in any operating environment can be locked down and verified rigorously. This article aims to assist IoT manufacturers in developing and verifying MUD profiles, while also helping adopters of these devices to ensure they are compatible with their organizational policies and track device network behavior using their MUD profile. Our first contribution is to develop a tool that takes the traffic trace of an arbitrary IoT device as input and automatically generates the MUD profile for it. We contribute our tool as open source, apply it to 28 consumer IoT devices, and highlight insights and challenges encountered in the process. Our second contribution is to apply a formal semantic framework that not only validates a given MUD profile for consistency, but also checks its compatibility with a given organizational policy. We apply our framework to representative organizations and selected devices, to demonstrate how MUD can reduce the effort needed for IoT acceptance testing. Finally, we show how operators can dynamically identify IoT devices using known MUD profiles and monitor their behavioral changes in their network.

Software-defined networking (SDN) enables innovative and impressive solutions in the networking domain by decoupling the control plane from the data plane. In an SDN environment, the network control logic for load balancing, routing, and access control is written in software running on a decoupled control plane. As with any software development cycle, the SDN control plane is prone to bugs that impact the network's performance and availability. Yet, as a community, we lack holistic, in-depth studies of bugs within the SDN ecosystem. A bug taxonomy is one of the most promising ways to lay the foundations required for (1) evaluating and directing emerging research directions on fault detection and recovery, and (2) informing operational practices of network administrators. This paper takes the first step towards laying this foundation by providing a comprehensive study and analysis of over 500 `critical' bugs (including ~ 150 with manual analysis) in three of the most widely-used SDN controllers, i.e., FAUCET, ONOS, and CORD. We create a taxonomy of these SDN bugs, analyze their operational impact, and implications for the developers. We use our taxonomy to analyze the effectiveness and coverage of several prominent SDN fault tolerance and diagnosis techniques. This study is the first of its kind in scale and coverage to the best of our knowledge.

Emerging distributed applications, such as big data analytics, generate a large number of flows that concurrently transport data across data center networks. To improve their performance, it is required to account for the behavior of such a collection of flows, i.e., coflows, rather than individual ones. State-of-the-art solutions achieve near-optimal completion time by continuously reordering unfinished coflows at the end-host and using network priorities.This paper shows that dynamically changing flow priorities at the end-host, without considering in-flight packets, can cause high degrees of packet reordering, thus imposing pressure on the congestion control and potentially harming network performance in the presence of switches with shallow buffers. We present pCoflow, a new solution that integrates end-host based coflow ordering with in-network scheduling based on packet history. Our evaluation shows that pCoflow improves in coflow completion time upon state-of-the-art solutions by up to 34% for varying loads.

IETF QUIC, the standardized version of Google’s UDP-based layer-4 network protocol, has seen increasing adoption from large Internet companies for its benefits over TCP. Yet despite its rapid adoption, performance analysis of QUIC in production is scarce. Most existing analyses have only used unoptimized open-source QUIC servers on non-tuned kernels: these analyses are unrepresentative of production deployments which raises the question of whether QUIC actually outperforms TCP in practice.

Configuring a service mesh often involves multiple parties, each of whom is responsible for separate portions of the overall system. This can result in miscommunication, silent and sudden errors, or a failure to meet goals.

Modern network infrastructure has evolved into a complex organism to satisfy the performance and availability requirements for the billions of users. Frequent releases such as code upgrades, bug fixes and security updates have become a norm. Millions of globally distributed infrastructure components including servers and load-balancers are restarted frequently from multiple times per-day to per-week. However, every release brings possibilities of disruptions as it can result in reduced cluster capacity, disturb intricate interaction of the components operating at large scales and disrupt the end-users by terminating their connections. The challenge is further complicated by the scale and heterogeneity of supported services and protocols.

Programmable data planes, PDPs, enable an unprecedented level of flexibility and have emerged as a promising alternative to existing data planes. Despite the rapid development and prototyping cycles that PDPs promote, the existing PDP ecosystem lacks appropriate abstractions and algorithms to support these rapid testing and deployment life-cycles. In this paper, we propose P4Visor, a lightweight virtualization abstraction that provides testing primitives as a first-order citizen of the PDP ecosystem. P4Visor can efficiently support multiple PDP programs through a combination of compiler optimizations and program analysis-based algorithms. P4Visor's algorithm improves over state-of-the-art techniques by significantly reducing the resource overheads associated with embedding numerous versions of a PDP program into hardware. To demonstrate the efficiency and viability of P4Visor, we implemented and evaluated P4Visor on both a software switch and an FPGA-based hardware switch using fourteen of different PDP programs. Our results demonstrate that P4Visor introduces minimal overheads and is one order of magnitude more efficient than existing PDPs primitives for concurrently supporting multiple programs.

Over the last few years, we have experienced a massive transformation of the Software Defined Networking ecosystem with the development of SDNEnhancements, e.g., Statesman, ESPRES, Pane, and Pyretic, to provide better composability, better utilization of TCAM, consistent network updates, or congestion free updates. The end-result of this organic evolution is a disconnect between the SDN applications and the data-plane. A disconnect which can impact an SDN application's performance and efficacy.

Though centrally managed by a controller, a software-defined network (SDN) can still encounter routing inconsistencies among its switches due to the non-atomic updates to their forwarding tables. In this paper, we propose a new method to rectify these inconsistencies that is inspired by causal consistency, a consistency model for shared-memory systems. Applied to SDNs, causal consistency would imply that once a packet is matched to ("reads") a forwarding rule in a switch, it can be matched in downstream switches only to rules that are equally or more up-to-date. We propose and analyze a relaxed but functionally equivalent version of this property called suffix causal consistency (SCC) and evaluate an implementation of SCC in Open vSwitch and P4 switches, in conjunction with the Ryu and P4Runtime controllers. Our results show that SCC provides greater efficiency than competing consistent-update alternatives while offering consistency that is strong enough to ensure high-level routing properties (black-hole freedom, bounded looping, etc.).

Programmable data planes promise unprecedented flexibility and innovation. But enormous management issues arise when these programmable data-planes, and the in-network compute functionality they enable, are deployed within production networks. In this paper, we present an overview of these management challenges, then explore the limitations of existing management techniques. Finally, we propose a system, Harmony, that encapsulates new abstractions and primitives to address these problems.

Managing large-scale distributed systems is a difficult task. System administrators are responsible for the upkeep and maintenance of numerous components with complex dependencies. With the shift to microservices-based architectures, these systems can consist of 100s to 1000s of interconnected nodes. To combat this difficulty, administrators rely on analyzing logs and metrics collected from the different services. However, the number of available metrics for large systems presents complexity and scaling issues. To combat these issues, we present Minerva, an unsupervised Machine Learning (ML) framework for performing network diagnosis analysis. Minerva is composed of a multi-stage pipeline, where each component can act individually or cohesively to perform various management tasks. Our system offers a unified and extensible framework for managing the complexity of large networks, and presents administrators with a swiss-army knife for diagnosing the overall health of their systems. To demonstrate the feasibility of Minerva, we evaluate its performance on a production-scale system. We present use cases for the various management tools made available by Minerva, and show how these tools can be used to make strong inferences about the system using unsupervised techniques.

Programmable data planes, PDPs, enable an unprecedented level of flexibility and have emerged as a promising alternative to existing data planes. Despite the rapid development and prototyping cycles that PDPs promote, the existing PDP ecosystem lacks appropriate abstractions and algorithms to support these rapid testing and deployment life-cycles. In this paper, we propose P4Visor, a lightweight virtualization abstraction that provides testing primitives as a first-order citizen of the PDP ecosystem. P4Visor can efficiently support multiple PDP programs through a combination of compiler optimizations and program analysis-based algorithms. P4Visor s algorithm improves over state-of-the-art techniques by significantly reducing the resource overheads associated with embedding numerous versions of a PDP program into hardware. To demonstrate the efficiency and viability of P4Visor, we implemented and evaluated P4Visor on both a software switch and an FPGA-based hardware switch using fourteen different PDP programs. Our results demonstrate that P4Visor introduces minimal overheads (less than 1%) and is one order of magnitude more efficient than existing PDPs primitives for concurrently supporting multiple programs.

Over the last decade, in an attempt to improve the end-user experience, the community has proposed a multitude of changes to the configuration parameters of the networking protocol stack of modern web servers. These changes range from improving security (e.g., TLS 1.2) to improving performance (e.g., HTTP/2). While the performance implications of several of these configuration parameters have been studied in isolation. To date, there is no holistic and general tool to infer, analyze, and under-stand the actual configuration parameters employed by popular web services. Moreover, it is unclear how these parameters are tuned to account for differences in network conditions, devices characteristics, or web page complexity. Although little is known, the configuration of these parameters impact attempts to model and understand performance expectations across the internet. To this end, we present InspectorGadget, a framework for characterizing and fingerprinting the configuration parameters of a server's network stack. InspectorGadget leverages some domain-specific heuristics for reverse engineering configuration parameters and options (protocol versions). To demonstrate the efficacy of InspectorGadget, we implemented a prototype of InspectorGadget and used this prototype to survey the configuration parameters for the top 10K online content providers across different regions and end-user devices.

The last few years have seen a massive and organic transformation of the Software Defined Networking ecosystem with the development of enhancements, e.g., Statesman, ESPRES, PANE, and Athens, to provide better composability, better utilization of TCAM, consistent network updates, or congestion free updates. The end-result of this organic evolution is a disconnect between the SDN applications and the dataplane. A disconnect which can impact an SDN application's performance or correctness.

The First Workshop on Internet of Things Security and Privacy is held in Dallas, TX, USA on November 3, 2017, co-located with the ACM Conference on Computer and Communications Security (CCS). The workshop aims to address the security and privacy challenges of the emerging Internet-of-Things landscape. The workshop aims to bring together academic and industrial researchers, and to that end, we have put together an exciting program offering a a mix of current and potential challenges. The workshop will also features 12 papers, 4 posters, and an invited keynote.

SDN controllers demand tight performance guarantees over the control plane actions performed by switches. For example, traffic engineering techniques that frequently reconfigure the network require guarantees on the speed of reconfiguring the network. Initial experiments show that poor performance of Ternary Content-Addressable Memory (TCAM) control actions (e.g., rule insertion) can inflate application performance by a factor of 2x! Yet, modern switches provide no guarantees for these important control plane actions -- inserting, modifying, or deleting rules.

Cloud computing has evolved to meet user demands, from arbitrary VMs offered by IaaS to the narrow application interfaces of PaaS. Unfortunately, there exists an intermediate point that is not well met by today's offerings: users who wish to run arbitrary, already available binaries (as opposed to rewriting their own application for a PaaS) yet expect their applications to be long-lived but mostly idle (as opposed to the always-on VM of IaaS). For example, end users who wish to run their own email or DNS server.

Network failures are inevitable. Interfaces go down, devices crash and resources become exhausted. It is the responsibility of the control software to provide reliable services on top of unreliable components and throughout unpredictable events. Guaranteeing the correctness of the controller under all types of failures is therefore essential for network operations. Yet, this is also an almost impossible task due to the complexity of the control software, the underlying network, and the lack of precision in simulation tools.

No abstract available.

Despite Software Defined Network's (SDN) proven benefits, there remains significant reluctance in adopting it. Among the issues that hamper SDN's adoption two stand out: reliability and fault tolerance. At the heart of these issues is a set of fate-sharing relationships: The first between the SDN-Apps and controllers, where-in the crash of the former induces a crash of the latter, and thereby affecting availability; and, the second between the SDN-App and the network, where-in a byzantine failure e.g., black-holes and network-loops, induces a failure in the network, and thereby affecting network availability. The principal position of this paper is that availability is of utmost concern -- second only to security. To this end, we present a re-design of the controller architecture centering around a set of abstractions to eliminate these fate-sharing relationships, and make the controllers and network resilient to SDN-App failures. We illustrate how these abstractions can be used to improve the reliability of an SDN environment, thus eliminating one of the barriers to SDN's adoption.

The current IaaS model has several shortcomings. First, several IaaS providers only offers VM (virtual machine) with predefined sizes, thus enterprise tenants must judiciously determine the VM size that best fit their application. This is challenging as overprovisioning VMs can lead to waste of resources while underprovisioned VMs can lead to poor performance. Second, when an application requires more resources than a VM can provide, tenants are currently limited to either scaling-out or scaling-up their applications. However, in both situations the granularity is at the level of VMs which leads to sizing issues discussed earlier. Third, scaling-up is ineffective as it incurs a significant amount of downtime/poor performance while the new VM is being provisioned and not all applications support scaling-out. For example while, Web servers can be easily scaled-out other legacy applications can not [1], thus limiting its applicability.

The progress of a big data job is often a function of storage, networking and processing. Hence, for efficient job execution, it is important to collectively optimize all three components. Prior proposals [1], in contrast, have focused on mainly on one or two of the three components. This narrow focus constraints the extent to which these proposals can support efficient operation of big data applications.

Enterprises today face several challenges when hosting line-of-business applications in the cloud. Central to many of these challenges is the limited support for control over cloud network functions, such as, the ability to ensure security, performance guarantees or isolation, and to flexibly interpose middleboxes in application deployments. In this paper, we present the design and implementation of a novel cloud networking system called CloudNaaS. Customers can leverage CloudNaaS to deploy applications augmented with a rich and extensible set of network functions such as virtual network isolation, custom addressing, service differentiation, and flexible interposition of various middleboxes. CloudNaaS primitives are directly implemented within the cloud infrastructure itself using high-speed programmable network elements, making CloudNaaS highly efficient. We evaluate an OpenFlow-based prototype of CloudNaaS and find that it can be used to instantiate a variety of network functions in the cloud, and that its performance is robust even in the face of large numbers of provisioned services and link/device failures.

The effects of data center traffic characteristics on data center traffic engineering is not well understood. In particular, it is unclear how existing traffic engineering techniques perform under various traffic patterns, namely how do the computed routes differ from the optimal routes. Our study reveals that existing traffic engineering techniques perform 15% to 20% worse than the optimal solution. We find that these techniques suffer mainly due to their inability to utilize global knowledge about flow characteristics and make coordinated decision for scheduling flows.

Studying network configuration evolution can improve our understanding of the evolving complexity of networks and can be helpful in making network configuration less error-prone. Unfortunately, the nature of changes that operators make to network configuration is poorly understood. Towards improving our understanding, we examine and analyze five years of router, switch, and firewall configurations from two large campus networks using the logs from version control systems used to store the configurations. We study how network configuration is distributed across different network operations tasks and how the configuration for each task evolves over time, for different types of devices and for different locations in the network. To understand the trends of how configuration evolves over time, we study the extent to which configuration for various tasks are added, modified, or deleted. We also study whether certain devices experience configuration changes more frequently than others, as well as whether configuration changes tend to focus on specific portions of the configuration (or on specific tasks). We also investigate when network operators make configuration changes of various types. Our results concerning configuration changes can help the designers of configuration languages understand which aspects of configuration might be more automated or tested more rigorously and may ultimately help improve configuration languages.

ISPs are increasingly offering a variety of network-based services such as VPN, VPLS, VoIP, Virtual-Wire and DDoS protection. Although both enterprise and residential networks are rapidly adopting these services, there is little systematic work on the design challenges and trade-offs ISPs face in providing them. The goal of our paper is to understand the complexity underlying the layer-3 design of services and to highlight potential factors that hinder their introduction, evolution and management. Using daily snapshots of configuration and device metadata collected from a tier-1 ISP, we examine the logical dependencies and special cases in device configurations for five different network-based services. We find: (1) the design of the core data-plane is usually service-agnostic and simple, but the control-planes for different services become more complex as services evolve; (2) more crucially, the configuration at the service edge inevitably becomes more complex over time, potentially hindering key management issues such as service upgrades and troubleshooting; and (3) there are key service-specific issues that also contribute significantly to the overall design complexity. Thus, the high prevalent complexity could impede the adoption and growth of network-based services. We show initial evidence that some of the complexity can be mitigated systematically.

Although there is tremendous interest in designing improved networks for data centers, very little is known about the network-level traffic characteristics of data centers today. In this paper, we conduct an empirical study of the network traffic in 10 data centers belonging to three different categories, including university, enterprise campus, and cloud data centers. Our definition of cloud data centers includes not only data centers employed by large online service providers offering Internet-facing applications but also data centers used to host data-intensive (MapReduce style) applications). We collect and analyze SNMP statistics, topology and packet-level traces. We examine the range of applications deployed in these data centers and their placement, the flow-level and packet-level transmission properties of these applications, and their impact on network and link utilizations, congestion and packet drops. We describe the implications of the observed traffic patterns for data center internal traffic engineering as well as for recently proposed architectures for data center networks.

Few studies so far have examined the nature of reachability policies in enterprise networks. A better understanding of reachability policies could both inform future approaches to network design as well as current network configuration mechanisms. In this paper, we introduce the notion of a policy unit, which is an abstract representation of how the policies implemented in a network apply to different network hosts. We develop an approach for reverse-engineering a network's policy units from its router configuration. We apply this approach to the configurations of five productions networks, including three university and two private enterprises. Through our empirical study, we validate that policy units capture useful characteristics of a network's policy. We also obtain insights into the nature of the policies implemented in modern enterprises. For example, we find most hosts in these networks are subject to nearly identical reachability policies at Layer 3.