We report on experiences with Swift congestion control in Google datacenters. Swift targets an end-to-end delay by using AIMD control, with pacing under extreme congestion. With accurate RTT measurement and care in reasoning about delay targets, we find this design is a foundation for excellent performance when network distances are well-known. Importantly, its simplicity helps us to meet operational challenges. Delay is easy to decompose into fabric and host components to separate concerns, and effortless to deploy and maintain as a congestion signal while the datacenter evolves. In large-scale testbed experiments, Swift delivers a tail latency of <50μs for short RPCs, with near-zero packet drops, while sustaining ~100Gbps throughput per server. This is a tail of <3x the minimal latency at a load close to 100%. In production use in many different clusters, Swift achieves consistently low tail completion times for short RPCs, while providing high throughput for long RPCs. It has loss rates that are at least 10x lower than a DCTCP protocol, and handles O(10k) incasts that sharply degrade with DCTCP.

Network virtualization stacks are the linchpins of public clouds. A key goal is to provide performance isolation so that workloads on one Virtual Machine (VM) do not adversely impact the network experience of another VM. Using data from a major public cloud provider, we systematically characterize how performance isolation can break in current virtualization stacks and find a fundamental tradeoff between isolation and resource multiplexing for efficiency. In order to provide predictable performance, we propose a new system called PicNIC that shares resources efficiently in the common case while rapidly reacting to ensure isolation. PicNIC builds on three constructs to quickly detect isolation breakdown and to enforce it when necessary: CPU-fair weighted fair queues at receivers, receiver-driven congestion control for backpressure, and sender-side admission control with shaping. Based on an extensive evaluation, we show that this combination ensures isolation for VMs at sub-millisecond timescales with negligible overhead.

The problem of overbuffering in the current Internet (termed as bufferbloat) has drawn the attention of the research community in recent years. Cellular networks keep large buffers at base stations to smooth out the bursty data traffic over the time-varying channels and are hence apt to bufferbloat. However, despite their growing importance due to the boom of smart phones, we still lack a comprehensive study of bufferbloat in cellular networks and its impact on TCP performance. In this paper, we conducted extensive measurement of the 3G/4G networks of the four major U.S. carriers and the largest carrier in Korea. We revealed the severity of bufferbloat in current cellular networks and discovered some ad-hoc tricks adopted by smart phone vendors, which mitigate the impact of bufferbloat but result in performance degradation under various practical scenarios. To address the problem, we propose dynamic receiver window adjustment (DRWA) that requires slight TCP modification only in smart phones, thus guarantees quick deployment via over-the-air updates. Our extensive real-world tests confirm that DRWA reduces the latency of TCP flows by 25-49 percent and increase TCP throughput by up to 51 percent in certain scenarios. It is further verified that DRWA has significant effect on the latency of Voice-over-IP traffic and on the traffic going through TCP split scenarios with performance enhancing proxy.

Datacenter transports aim to deliver low latency messaging together with high throughput. We show that simple packet delay, measured as round-trip times at hosts, is an effective congestion signal without the need for switch feedback. First, we show that advances in NIC hardware have made RTT measurement possible with microsecond accuracy, and that these RTTs are sufficient to estimate switch queueing. Then we describe how TIMELY can adjust transmission rates using RTT gradients to keep packet latency low while delivering high bandwidth. We implement our design in host software running over NICs with OS-bypass capabilities. We show using experiments with up to hundreds of machines on a Clos network topology that it provides excellent performance: turning on TIMELY for OS-bypass messaging over a fabric with PFC lowers 99 percentile tail latency by 9X while maintaining near line-rate throughput. Our system also outperforms DCTCP running in an optimized kernel, reducing tail latency by $13$X. To the best of our knowledge, TIMELY is the first delay-based congestion control protocol for use in the datacenter, and it achieves its results despite having an order of magnitude fewer RTT signals (due to NIC offload) than earlier delay-based schemes such as Vegas.

In Named Data Networking (NDN), forwarding lookup is based on tokenized variable-length names instead of fixed-length host addresses, and therefore it requires a new approach for designing a fast packet forwarding lookup engine. In this paper, we propose a design of an NDN software forwarding lookup engine based on hash tables and evaluate its performance with different design options. With a good hash function and table design combined with Bloom filters and data prefetching, we demonstrate that our design reaches about 1.5MPPS with a single thread on an Intel 2.0GHz Xeon processor.

Locality-aware P2P file sharing systems have drawn the attention of the research community as a promising technique to alleviate the tussle between P2P traffic and ISPs. However, existing locality-based schemes mainly focus on how to distinguish whether a peer is local or not and pay less attention to how to utilize the locality information. Typically, they simply bias the peer selection towards local peers in the hope that it will reduce cross-ISP traffic. In this paper, we argue that such coarse-grained control is inadequate and propose a fine-grained mechanism called Swarm-over-Swarm (SOS). Through explicit coordination on piece selection among local peers, SOS is much more effective in reducing redundant cross-ISP traffic than existing schemes. We implement SOS in both NS-2 and a real BitTorrent client and demonstrate the effectiveness of our solution via both simulations and Internet experiments.

The problem of overbuffering in the current Internet (termed as bufferbloat) has drawn the attention of the research community in recent years. Cellular networks keep large buffers at base stations to smooth out the bursty data traffic over the time-varying channels and are hence apt to bufferbloat. However, despite their growing importance due to the boom of smart phones, we still lack a comprehensive study of bufferbloat in cellular networks and its impact on TCP performance. In this paper, we conducted extensive measurement of the 3G/4G networks of the four major U.S. carriers and the largest carrier in Korea. We revealed the severity of bufferbloat in current cellular networks and discovered some ad-hoc tricks adopted by smart phone vendors to mitigate its impact. Our experiments show that, due to their static nature, these ad-hoc solutions may result in performance degradation under various scenarios. Hence, a dynamic scheme which requires only receiver-side modification and can be easily deployed via over-the-air (OTA) updates is proposed. According to our extensive real-world tests, our proposal may reduce the latency experienced by TCP flows by 25% ~ 49% and increase TCP throughput by up to 51% in certain scenarios.

Overload happens in Session Initiation Protocol (SIP) networks when SIP servers have insufficient resources to handle all messages they receive. Under overload, SIP networks suffer from congestion collapse due to its ineffective overload control mechanism. In this paper we propose a backpressure-based SIP overload control mechanism called Bassoon. It consists of two parts: the first part is a provably optimal load balancing algorithm that ensures full utilization of the available network resources. The second part is an end-to-end load control algorithm that intelligently throttles excessive traffic at the edge of the network. We show that, by combining optimal load balancing and end-to-end load control, Bassoon effectively controls overload in SIP networks and outperforms existing schemes in terms of goodput, fairness and responsiveness.