Lossless Ethernet uses hop-by-hop backpressure to prevent buffer overflow and has become the mainstream choice for running Remote Direct Memory Access (RDMA) in AI and cloud data centers. Despite preventing congestion-induced drops, lossless networks introduce congestion contagion, which causes head-of-line blocking, congestion spreading, and deadlocks. Congestion control schemes have been introduced to mitigate the drawbacks of lossless Ethernet. However, congestion control mechanisms struggle with bursty traffic, face a dilemma, and can be sidelined by backpressure. In this paper, we propose congestion quarantine (CQ) as a complementary congestion management mechanism for lossless Ethernet. CQ uses a separate queue to quarantine congested flows, preventing congestion contagion, resolving the CC dilemma, and preventing CC from being sidelined. Results show that congestion quarantine can eliminate head-of-line blocking in scenarios with multiple congestion trees. Large-scale simulations demonstrate that CQ reduces the average and 99th percentile FCT slowdown of normal flows by 25–86% and 33.4–90%, respectively, with negligible impact on bursty traffic.

In cloud datacenter operations, telemetry and logs are indispensable, enabling essential services such as network diagnostics, auditing, and knowledge discovery. The escalating scale of data centers, coupled with increased bandwidth and finer-grained telemetry, results in an overwhelming volume of data. This proliferation poses significant storage challenges for telemetry systems. In this article, we introduce HyperDrive, an innovative system designed to efficiently store large volumes of telemetry and logs in data centers using programmable switches. This in-network approach effectively mitigates bandwidth bottlenecks commonly associated with traditional endpoint-based methods. To our knowledge, we are the first to use a programmable switch to directly control storage, bypassing the CPU to achieve the best performance. With merely 21% of a switch’s resources, our HyperDrive implementation showcases remarkable scalability and efficiency. Through rigorous evaluation, it has demonstrated linear scaling capabilities, efficiently managing 12 SSDs on a single server with minimal host overhead. In an eight-server testbed, HyperDrive achieved an impressive throughput of approximately 730 Gbps, underscoring its potential to transform data center telemetry and logging practices.

With the growing importance of real-time communications (RTC), designing congestion control (CC) algorithms for RTC that achieve high network performance and QoE is gaining attention. Recently, data-driven, reinforcement learning (RL)-based CC algorithms for RTC have shown great potential, outperforming traditional rule-based counterparts. However, there are no open platforms tailored for training, evaluation, and validation of the algorithms that can facilitate this emerging research area.

Datacenter generates operation data at an extremely high rate, and data center operators collect and analyze them for problem diagnosis, resource utilization improvement, and performance optimization. However, existing data collection methods fail to efficiently aggregate and store data at extremely high speed and scale. In this paper, we explore a new approach that leverages programmable switches to aggregate data and directly write data to the destination storage. Our proposed data collection system, ALT, uses programmable switches to control NVMe SSDs on remote hosts without the involvement of a remote CPU. To tolerate loss, ALT uses an elegant data structure to enable efficient data recovery when retrieving the collected data. We implement our system on a Tofino-based programmable switch for a prototype. Our evaluation shows that ALT can saturate SSD’s peak performance without any CPU involvement.

One of the key challenges in deploying RL to real-world applications is to adapt to variations of unknown environment contexts, such as changing terrains in robotic tasks and fluctuated bandwidth in congestion control. Existing works on adaptation to unknown environment contexts either assume the contexts are the same for the whole episode or assume the context variables are Markovian. However, in many real-world applications, the environment context usually stays stable for a stochastic period and then changes in an abrupt and unpredictable manner within an episode, resulting in a segment structure, which existing works fail to address. To leverage the segment structure of piecewise stable context in real-world applications, in this paper, we propose a \textit{\textbf{Se}gmented \textbf{C}ontext \textbf{B}elief \textbf{A}ugmented \textbf{D}eep~(SeCBAD)} RL method. Our method can jointly infer the belief distribution over latent context with the posterior over segment length and perform more accurate belief context inference with observed data within the current context segment. The inferred belief context can be leveraged to augment the state, leading to a policy that can adapt to abrupt variations in context. We demonstrate empirically that SeCBAD can infer context segment length accurately and outperform existing methods on a toy grid world environment and Mujuco tasks with piecewise-stable context.

With the rapid growth of hardware devices, a single host may have simultaneous connections that vary in network bandwidth and CPU processing capability as several orders of magnitude. State-of-art flow control mechanism, i.e., TCP auto-tuning, still needs to configure the maximum receive buffer, which cannot be applied to all connections in one host. In this paper, we reveal that improper receive buffer restrained by this configuration either (i) underutilizes the available network and CPU resources or (ii) occupies too much memory and then causes overall throughput collapse. To fully utilize resources with less memory occupancy, we present Receive Buffer Adaptive-regulating (RBA) algorithm, which regulates receive buffer according to the estimation of network bandwidth and receiver's processing capability. Testbed experiments show that RBA adapts to different scenarios and brings substantial performance improvement compared to TCP auto-tuning.

Memcached is a widely used in-memory caching solution in large-scale searching scenarios. The most crucial metric of Memcached systems is the response time, which is affected by various factors such as workload, service rate, unbalanced load distribution, and cache miss ratio. This article aims to quantify the influence of each factor on the response time of Memcached systems. First, we establish a theoretical model for Memcached systems that captures their main features, including burst and concurrent key arrival, unbalanced load distribution, and cache miss process. By solving this model using queuing and stochastic theories, we obtain an estimate of the response time in Memcached systems. Intensive experiments based on real-world components demonstrate that the estimate always matches perfectly with the actual value. Furthermore, we obtain a comprehensive and quantitative understanding of all factors. The main insights are threefold. 1) There exists an optimum range of utilization at Memcached servers in which the response time is kept at a low level with a small penalty. 2) The influence of the cache miss ratio on the response time is logarithmic rather than linear. 3) The number of keys generated from an end-user request has the greatest impact in Memcached systems.

Nowadays, optical circuit switching is becoming an increasingly favored technology in scaling data center networks for its definitive advantages in data rate, power consumption, and device cost. Concurrently, reducing coflow completion time (CCT) is of great significance for improving application-level performance. However, minimizing CCT in circuit switched networks is totally different from that in traditional packet switched networks due to port constraints and circuit reconfiguration delays. To address this issue, this article proposes Grouped Optimization-based Scheduling (GOS), a CCT minimization algorithm for circuit switched networks integrating circuit and coflow scheduling. We first formalize the CCT minimization problem into a 0-1 programming problem, then relax and solve the problem in 2 steps to obtain the coflow order and flow grouping decisions on each circuit. Thus intra-group reconfiguration delays are saved, and small coflows can be prioritized at the group level. Theoretical analysis proves GOS is a 4-approximation algorithm in average CCT. To reduce computing overheads, we further propose a heuristic approximation algorithm. Extensive simulations show that the heuristic algorithm has satisfactory CCT performance (0.12× Varys, 0.36× Sunflow) as well as high throughput (16.74× Varys, 1.32× Sunflow), and well adapts to a wide range of reconfiguration delays and algorithm decision time.

In recent years, the growth in popularity of mobile video streaming services is unbroken. There are tremendous demands for video streaming over wireless networks. Currently, most video streaming is over HTTP. Up to now, HTTP-based adaptive video streaming is standardized as DASH, where a client-side video player can dynamically pick the bitrate level according to the perceived network conditions. Actually, not only the available bandwidth drastically varies due to wireless network properties, but also the chunk sizes in the same bitrate level significantly fluctuate, which also influences the bitrate adaptation. However, existing bitrate adaptation algorithms mostly focus on available bandwidth but do not involve chunk size variation, leading to performance losses. In this paper, we theoretically analyze the influence of chunk size variation on bitrate adaptation performance in wireless networks. Based on DASH system features, we build a general model describing playback buffer evolution. Applying stochastic theories, we respectively analyze the influence of the chunk size variation on rebuffering probability, average bitrate, and bitrate switching interval. Furthermore, based on theoretical insights, we provide several suggestions for algorithm designing and rate encoding, and also design a simple bitrate adaptation algorithm. Extensive simulations verify our insights, suggestions, and designed algorithm effectiveness.

Recently, high performance packet I/O frameworks continue to flourish for their ability to process packets from high-speed links. To achieve high throughput and low latency, high performance packet I/O frameworks usually employ busy polling. As busy polling will burn all CPU cycles even if there's no packet to process, these frameworks are quite power inefficient. However, exploiting power management techniques such as DVFS and LPI in the frameworks is challenging, because neither the OS nor the frameworks can provide information (e.g., actual CPU utilization, available idle period, or the target frequency) required by these techniques. In this article, we establish a model that can formulate the packet processing flow of high performance packet I/O to help and address the above challenges. From the model, we can deduce the information needed for power management techniques, and gain the insights to balance the power and latency. After suggesting to use pause instruction to reduce CPU power within short idle period, we propose two approaches to conduct power conservation for high performance packet I/O: one with the aid of traffic information and the other without. Experiments with Intel DPDK show that both approaches can achieve significant power reduction with little latency increase.

The Remote Procedure Call (RPC) frameworks are widely deployed in industry. Applications supported by RPC frameworks are often latency-sensitive which strictly require to be responded before the deadline. For meeting this requirement, RPC frameworks adopt the application-level flow control mechanism. This mechanism gives an appropriate threshold determining the number of RPC requests that the server can process, thus avoids missing the deadline. However, this threshold at the application-level is a fixed empirical value so that it is hard to obtain respectable performance because an endpoint's processing capacity can take a huge quantity of values by varying workload and different hardware configurations. While other methods based on specialized transport protocols are adaptive, they will introduce extra costs for message reporting from server to client. Furthermore, adopting specialized transport protocols will also introduce extra transplanting efforts for TCP-based applications. In this paper, we provide an active and adaptive application level flow control mechanism at the client side. We first design an algorithm to find the appropriate threshold to achieve the desired response time. Then based on this algorithm, we control the threshold to bound the response time as expected. We implement our flow control mechanism using a memcached testbed. Experiments prove that our mechanism can accurately reduce the mean and 99th percentile response time by at least 71.3% and 69.4% respectively, while keeping a relatively high QPS. Furthermore, compared to static-threshold mechanism, our flow control mechanism is more efficient under low latency constraints.

Many applications in distributed systems rely on underlying lossless networks to achieve required performance. Existing lossless network solutions propose different hop-by-hop flow controls to guarantee zero packet loss. However, another crucial problem called network deadlock occurs concomitantly. Once the system traps in a deadlock, a large part of network would be disabled. Existing deadlock avoidance solutions focus all their attentions on breaking the cyclic buffer dependency to eliminate circular wait (one necessary condition of deadlock). These solutions, however, impose many restrictions on network configurations and side-effects on performance.

Short connections are generally used to transfer small-size messages, which contribute a large part of workload in modern applications. The maximum sustainable short connection rate, which is called short connection capacity, is an important index for admission control, Web QoS control, and energy saving. A capacity estimation mechanism aims to find the workload just saturating the server, and it relies on both workload information and system information. Past researches point out that kernel space network stack becomes the bottleneck when a huge number of concurrent short connections coexist. On the other hand, high performance user level network stacks have been proved to eliminate such bottleneck, thus become a hot research topic in both academia and industry. However, they also bring challenges for estimating short connection capacity, making traditional methods ineffective. Therefore, it is important to find a new method to estimate short connection capacity on high performance user level network stacks. In this paper, we prove that the effective CPU utilization is an adaptive index to different workload patterns and application complexities, which can reflect the server state. Then we design and implement an online capacity estimator on the Seastar platform. We conduct experiments to verify the effectiveness of our online capacity estimator. The results show that our estimator can actually estimate the capacity online. When the server is near saturated, the 90th percentile relative estimating error is no more than 9.18%. Furthermore, our capacity estimator only introduces no more than 1.38% of capacity loss in our experiments.

Recently, high performance packet I/O frameworks are expected an extensive application for their ability to process packets from 10Gbps or higher speed links. To achieve high throughput and low latency, high performance packet I/O frameworks usually employ busy polling technique. As busy polling will burn all CPU cycles even if there's no packet to process, these frameworks are quite power inefficient. Meanwhile, exploiting power management techniques such as DVFS and LPI in high performance packet I/O frameworks is challenging, because neither the OS nor the frameworks can provide information (e.g., the actual CPU utilization, available idle period, or the target frequency) required by power management techniques. In this paper, we establish an analytical model that can formulate the packet processing flow of high performance packet I/O to help address the above challenges. From the analytical model, we can deduce the actual CPU utilization and average idle period in different traffic load, and gain the insight to choose CPU frequency that can appropriately balance the power consumption and packet latency. Then, we propose two simple but effective approaches to conduct power conservation for high performance packet I/O: one with the aid of traffic information and the other without. Experiments with Intel DPDK show that both approaches can achieve significant power reduction (35.90% and 34.43% on average respectively) while incurring < 1 μs of latency increase.

Recent academic and industrial work is exploring the challenges of using RDMA over Ethernet, to support highly reliable, latency-sensitive services in today's datacenters. Previous work on the high-speed packet I/O like netmap, DPDK, etc., and high-performance user-level stacks like mTCP, IX etc., rekindles our inspirations to implement a high-performance software RDMA over commodity Ethernet devices.

Memcached is a widely used in-memory caching solution in large-scale searching scenarios. The most pivotal performance metric in Memcached is latency, which is affected by various factors including the workload pattern, the service rate, the unbalanced load distribution and the cache miss ratio. To quantitate the impact of each factor on latency, we establish a theoretical model for the Memcached system. Specially, we formulate the unbalanced load distribution among Memcached servers by a set of probabilities, capture the burst and concurrent key arrivals at Memcached servers in form of batching blocks, and add a cache miss processing stage. Based on this model, algebraic derivations are conducted to estimate latency in Memcached. The latency estimation is validated by intensive experiments. Moreover, we obtain a quantitative understanding of how much improvement of latency performance can be achieved by optimizing each factor and provide several useful recommendations to optimal latency in Memcached.

Recently, HTTP-based adaptive video streaming has been widely adopted in the Internet. Up to now, HTTP-based adaptive video streaming is standardized as Dynamic Adaptive Streaming over HTTP (DASH), where a client-side video player can dynamically pick the bitrate level according to the perceived network conditions. Actually, not only the available bandwidth is varying, but also the chunk sizes in the same bitrate level significantly fluctuate, which also influences the bitrate adaptation. However, existing bitrate adaptation algorithms do not accurately involve the chunk size variation, leading to performance losses. In this paper, we theoretically analyze the influence of chunk size variation on bitrate adaptation performance. Based on DASH system features, we build a general model describing the playback buffer evolution. Applying stochastic theories, we respectively analyze the influence of the chunk size variation on rebuffering probability and average bitrate level. Furthermore, based on theoretical insights, we provide several recommendations for algorithm designing and rate encoding, and also propose a simple bitrate adaptation algorithm. Extensive simulations verify our insights as well as the efficiency of the proposed recommendations and algorithm.

Today's data center servers are equipped with high speed and complex network adaptors, featuring an array of functions, e.g. hardware TX/RX queues, packet filters, rate limiters, etc. Recent work like IX, Arrakis, MultiStack has made us rekindle the user-level network stacks' innovation utilizing these commodity network adaptors. In this paper, we revisit the idea to move stacks' design from in-kernel shared space into user-level application-specific dedicated one, for high performance and ease of development and deployment. We provide an unified control plane TAPM to exploit and manage the hardware adaptors' resources, and a dedicated data plane hwTAP to support different user-level stacks. TAPM and hwTAP highlight the utilization of hardware features from commodity network adaptors, to support the innovation of different user-level stacks. Experiments show that the hardware switching module can keep the input rate without any overheads and costs. TAPM could configure the hwTAP dynamically. Our run-to-completion user-level stack also achieves high throughput and low latency.

Congestion control is an indispensable mechanism in the new trend of enhanced Ethernet as a unified fabric for traditional LAN, SAN, and high-performance computing networks. A congestion management framework for Converged Ethernet (CE) networks has been standardized by IEEE 802.1 Qau work group, and QCN is recommended as the congestion control scheme in the standard draft. QCN is heuristically designed for 1/10Gbps Ethernet without considering the impact of delays. Recent work find that QCN will encounter stability issues with feedback delays, and these issues will be more serious as Ethernet extends to 40/100Gbps and the delays become heterogeneous and time-varying. This work aims to mitigate the negative impact of delays on congestion control scheme in CE. Specially, considering the delays are heterogeneous and time-varying, we build a model for Converged Ethernet with the standard congestion management framework. The model provides a new congestion detector to estimate the real congestion status under the impact of delays and regards the heterogeneous and time-varying feature as disturbances. Leveraging the new congestion detector and tolerating the disturbance through the sliding mode control method, we design the Delay-tolerant Sliding Mode (DSM) congestion control scheme. Extensive simulations show that DSM outperforms other congestion control schemes when the Ethernet ranges from 1Gbps to 100Gbps and the delays are heterogeneous and time-varying.

Owing to Ethernet's low cost, high bandwidth and architecture openness, much attention has been paid to develop converged Ethernet to support both time-critical services and conventional communication services on a unified network infrastructure. The greatest challenge here is providing low and deterministic latency for time-critical packets. Recently, the IEEE time sensitive networking task group is launched to address it. However, their framework is complex and unsuitable for commodity switch architecture. In this paper, we propose a concise and efficient converged real-time Ethernet framework called XpressEth, which leverages Dual Preemption mechanism to minimize the delay of time-critical packets, and employs a lightweight Slot Assignment Scheduler to minimize the conflicts among time-critical packets at sources. XpressEth cuts off great burden from both forwarding and scheduling. The simulation results verify that XpressEth can provide ultra-low and deterministic latency for time-critical packets (1.024μ s per hop and zero jitter in 1Gbps network), which is 13× better than time sensitive networking solution, and the side-effect on conventional communication traffic is negligible.

The shuffle transfer pattern is widely adopted in today's cluster computing applications and the completion time of each group of transmissions directly affects application performance. Because of the restriction on the number of concurrent threads and the TCP Incast problem, the randomized data fetching strategy is widely employed in this kind of communication in practice. In this paper, to assess the performance of randomized data fetching, we build a general analytical model and define two metrics - link overload probability and K-deviation load balancing probability - to evaluate the degree of link overload and load balancing respectively, since they are closely related to the transfer completion time. Leveraging our model, we theoretically analyze the transfer performance in three typical scenarios and provide recommendations for setting the number of concurrent connections per receiver. Finally, we validate the theoretical analysis as well as the recommendations through extensive simulations.