The cloud vendors provide sharing resources for millions of tenants across the world to achieve economies of scale. At the same time, the cloud network keeps the performance isolation between different tenants as if they use their private dedicated resources. However, heavy hitters caused by a single tenant at cloud gateways will break such isolation, undermining the predictable performance expected by other cloud tenants. To prevent it, heavy hitter detection becomes a key concern at the performance-critical cloud gateways but faces the dilemma between fine granularity and low overhead. In this work, we presentCloudSentry, a scalable two-stage heavy hitter detection system dedicated to multi-tenant cloud gateways against such a dilemma. CloudSentry uses CPU utilization as an indicator of heavy hitters and conducts a lightweight coarse-grained detection running 24/7 to detect such CPU spikes. Then it invokes a fine-grained detection to precisely dump and analyze the potential heavy-hitter packets at the CPU spikes. After that, a more comprehensive analysis is conducted to associate heavy hitters with the cloud service scenarios and invoke a corresponding backpressure procedure. CloudSentry significantly reduces memory, computation and storage overhead compared with existing approaches. In a gateway cluster under an average traffic throughput of 251 Gbps, CloudSentry consumes only a fraction of 2%–5% CPU utilization with 8 KB run-time memory, producing only 10 MB heavy hitter logs during one month. Additionally, as it has been deployed in Alibaba Cloud for over two years, we share case studies and a lot of deployment experiences in this article.

Accurate per-flow counting is beyond the capability of network switches due to the sheer flow number. The conventional divide-and-conquer method by distributing the traffic to multiple servers for software processing is costly. The solution therefore quests for a combination of hardware and software where the hardware with limited resources aims to undertake a part of the job and reduce the workload of software, achieving a desirable balance of cost and performance. To this end we design FASTeller to be deployed on SmartNICs. It is tuned to maximize the counting aggregation level in hardware, leaving the server a much lower workload for accurate per-flow counting and sparing the server capacity for post-counting functions such as network intrusion detection. The novelty lies in the multi-tier hardware caching data structure which is tailored for the flow distribution properties of real traffic. We build an FPGA-based prototype and evaluate the performance of FASTeller. The low-cost implementation achieves the highest performance among the methods in comparison and can easily sustain the accurate perflow counting for 100Gbps traffic with the least software load.

In this paper, we study service function chain (SFC) offloading in the edge-cloud network. Two offloading options are available for fully-loaded edge nodes: edge to edge (E2E) offloading and edge to cloud (E2C) offloading. Both E2E offloading and E2C offloading have been optimized in existing research, and Deep Reinforcement Learning (DRL) methods were adopted to achieve excellent performances. However, DRL-based SFC E2E and E2C joint offloading is still a research gap. In this paper, we propose a DRL-based SFC E2E and E2C joint offloading optimization algorithm for the edge-cloud network to maximize the utilization efficiency of edge resources. Twin Delayed Deep Deterministic policy gradient (TD3) algorithm is applied to the optimization problem. The simulation results indicate that the proposed algorithm has excellent convergence performance and improves the utilization efficiency of edge resources in edge-cloud network scenarios of diverse scales.

Persistent packet loss in the cloud-scale overlay network severely compromises tenant experiences. Cloud providers are keen to diagnose such problems efficiently. However, existing work is either designed for the physical network or insufficient to present the concrete reason of packet loss. We propose to record and analyze the on-site forwarding condition of packets during packet-level tracing. The cloud-scale overlay network presents great challenges to achieve this goal with its high network complexity, multi-tenant nature, and diversity of root causes. To address these challenges, we present VTrace, an automatic diagnostic system for persistent packet loss over the cloud-scale overlay network. Utilizing the “fast path-slow path” structure of virtual forwarding devices (VFDs), e.g., vSwitches, VTrace installs several “coloring-matching-logging” rules in VFDs to selectively track the target packets and inspect them in depth. The detailed forwarding situation at each hop is logged and then assembled to perform analysis with an efficient path reconstruction scheme. Experiments are conducted to demonstrate VTrace’s low overhead and quick response. Besides, based on the idea “coloring-matching-counting”, VTrace can be easily extended toVTrace-statsto identify the culprit device for transient packet loss. We share experiences of how VTrace andVTrace-statsefficiently work after deploying them in Alibaba Cloud for years.

The cloud network provides sharing resources for tens of thousands of tenants to achieve economics of scale. However, heavy hitters caused by a single tenant will probably interfere with the processing of the cloud gateways, undermining the predictable performance expected by other cloud tenants. To prevent it, heavy hitter detection becomes a key concern at the performance-critical cloud gateways but faces the dilemma between fine granularity and low overhead. In this work, we present CloudSentry, a scalable two-stage heavy hitter detection system dedicated to multi-tenant cloud gateways against such a dilemma. CloudSentry contains a lightweight coarse-grained detection running 24/7 to localize infrequent CPU spikes. Then it invokes a fine-grained detection to precisely dump and analyze the potential heavy-hitter packets at the CPU spikes. After that, a more comprehensive analysis is conducted to associate heavy hitters with the cloud service scenarios and invoke a corresponding backpressure procedure. CloudSentry significantly reduces memory, computation and storage overhead compared with existing approaches. Additionally, it has been deployed world-wide in Alibaba Cloud for over one year, with rich deployment experiences. In a gateway cluster under an average traffic throughput of of 251Gbps, CloudSentry consumes only a fraction of 2%-5% CPU utilization with 8KB run-time memory, producing only 10MB heavy hitter logs during one month.

Persistent packet loss in the cloud-scale overlay network severely compromises tenant experiences. Cloud providers are keen to automatically and quickly determine the root cause of such problems. However, existing work is either designed for the physical network or insufficient to present the concrete reason of packet loss. In this paper, we propose to record and analyze the on-site forwarding condition of packets during packet-level tracing. The cloud-scale overlay network presents great challenges to achieve this goal with its high network complexity, multi-tenant nature, and diversity of root causes. To address these challenges, we present VTrace, an automatic diagnostic system for persistent packet loss over the cloud-scale overlay network. Utilizing the "fast path-slow path" structure of virtual forwarding devices (VFDs), e.g., vSwitches, VTrace installs several "coloring, matching and logging" rules in VFDs to selectively track the packets of interest and inspect them in depth. The detailed forwarding situation at each hop is logged and then assembled to perform analysis with an efficient path reconstruction scheme. Experiments are conducted to demonstrate VTrace's low overhead and quick responsiveness. We share experiences of how VTrace efficiently resolves persistent packet loss issues after deploying it in Alibaba Cloud for over 20 months.

Large-scale data centers support various applications and process/issue terabits per second traffic from/to Internet. On the boundary of data center, the gateway needs to execute a series of network functions for each incoming packet. The Network Function Virtualization (NFV) technology leverages commodity servers to flexibly implement network functions. This solution provides satisfying processing and storage capability. However, state-of-the-art NFV platforms can merely process network functions at the line rate of 10~40Gbps. Supporting throughput of terabits per second requires dozens or even hundreds of servers operating exclusively for network functions, which is not only expensive but also difficult to maintain. On the other hand, programmable packet processing hardwares proposed in recent years offer a new platform for implementing network functions. They can execute user-defined packet processing logics at ultra-high line rate while containing limited processing and storage resources.

Recent academic and industrial work is exploring the challenges of using RDMA over Ethernet, to support highly reliable, latency-sensitive services in today's datacenters. Previous work on the high-speed packet I/O like netmap, DPDK, etc., and high-performance user-level stacks like mTCP, IX etc., rekindles our inspirations to implement a high-performance software RDMA over commodity Ethernet devices.

As big data analytics frameworks are developing towards larger degrees of parallelism and shorter task durations to provide lower latency, millions of scheduling decisions per second pose a great challenge to centralized schedulers. Therefore, increasing efforts are devoted to the study of distributed scheduling approaches to avoid the throughput limitation of centralized designs. Among these approaches, Sparrow is a leading design. However, due to Sparrow's sample-based techniques, some tasks in subsequent jobs may be scheduled earlier than those in the head-of-line job, which results in scheduling disorder and inevitably causes poor response times and unfairness. To address these problems, this paper proposes a simple algorithm called probe sharing: jobs that arrive at the same Sparrow scheduler can share their probes to ensure that all tasks in the head-of-line job can be scheduled earlier than subsequent jobs. We have performed theoretical analysis and proved that probe sharing makes a good improvement on Sparrow. We have implemented probe sharing in Sparrow and shown that probe sharing reduces scheduling delays by 2.2× and provides 100% fairness. Trace-driven simulations have been also used to evaluate probe sharing when scaling to large clusters. In addition, the simplicity of probe sharing makes it applicable to many schedulers that use Sparrow's techniques (e.g., Hopper, Tarcil and Eagle).

Today's data center servers are equipped with high speed and complex network adaptors, featuring an array of functions, e.g. hardware TX/RX queues, packet filters, rate limiters, etc. Recent work like IX, Arrakis, MultiStack has made us rekindle the user-level network stacks' innovation utilizing these commodity network adaptors. In this paper, we revisit the idea to move stacks' design from in-kernel shared space into user-level application-specific dedicated one, for high performance and ease of development and deployment. We provide an unified control plane TAPM to exploit and manage the hardware adaptors' resources, and a dedicated data plane hwTAP to support different user-level stacks. TAPM and hwTAP highlight the utilization of hardware features from commodity network adaptors, to support the innovation of different user-level stacks. Experiments show that the hardware switching module can keep the input rate without any overheads and costs. TAPM could configure the hwTAP dynamically. Our run-to-completion user-level stack also achieves high throughput and low latency.

DCTCP is a potential TCP replacement to satisfy the requirements of data center network. It receives wide concerns in both academic and industrial circles. However, DCTCP could only support tens of concurrent flows well and suffers timeouts and throughput collapse facing numerous concurrent flows. This is far from the requirement of data center network. Data centers employing partition/aggregation pattern usually involve hundreds of concurrent flows. In this paper, after tracing DCTCP's dynamic behavior through experiments, we explored two roots for DCTCP's failure under the high fan-in traffic pattern: (1) The regulation mechanism of sending window is ineffective when cwnd is decreased to the minimum size, (2) The bursts induced by synchronized flows with small cwnd cause fatal packet loss leading to severe timeouts. We enhance DCTCP to support massive concurrent flows by regulating the sending time interval and desynchronizing the sending time in particular conditions. The new protocol called DCTCP+ outperforms DCTCP when the number of concurrent flows increases to several hundreds. DCTCP+ can normally work to effectively support the short concurrent query responses in the benchmark from real production clusters, and keep the same good performance with the mixture of background traffic.

With the rapid development of wireless communication technology, a large number of wireless networks and devices that have different PHY and MAC layers coexist with each other. 2.4GHz Industrial, Scientific and Medical (ISM) band is becoming increasingly crowded. Wireless Sensor Networks (WSNs) which use low-power communication standard IEEE802.15.4 share the unlicensed spectrum with a plethora of other devices and technologies, such as WiFi systems underlying IEEE802.11, Bluetooth systems underlying IEEE802.15.1, and even non-communication appliance like microwave ovens. The ability to detect what radios are operating in the neighborhood is a fundamental need of WSNs, ranging from network management to network security. Since there are no explicit mechanisms to recognize such heterogeneous interference sources, WSNs often have no reasonable way to guard against them. In this paper, we describe the main working principle of heterogeneous interference sources, and present HEIR, a detector that is able to accurately identify heterogeneous interference sources. HEIR builds on the insight that there are hidden repeating patterns of signals which can be used to construct unique signatures and identify different types in most wireless protocols. The method can be implemented through signal sampling, interference estimation, feature extraction, and device classification. We show the experimental evaluation in an indoor testbed that HEIR is accurate in several different scenarios, and can live on sensor nodes' hardware. Since no any channel changes, the network topology is not interrupted, and the stable communication in real-time is ensured then.