The Remote Procedure Call (RPC) frameworks are widely deployed in industry. Applications supported by RPC frameworks are often latency-sensitive which strictly require to be responded before the deadline. For meeting this requirement, RPC frameworks adopt the application-level flow control mechanism. This mechanism gives an appropriate threshold determining the number of RPC requests that the server can process, thus avoids missing the deadline. However, this threshold at the application-level is a fixed empirical value so that it is hard to obtain respectable performance because an endpoint's processing capacity can take a huge quantity of values by varying workload and different hardware configurations. While other methods based on specialized transport protocols are adaptive, they will introduce extra costs for message reporting from server to client. Furthermore, adopting specialized transport protocols will also introduce extra transplanting efforts for TCP-based applications. In this paper, we provide an active and adaptive application level flow control mechanism at the client side. We first design an algorithm to find the appropriate threshold to achieve the desired response time. Then based on this algorithm, we control the threshold to bound the response time as expected. We implement our flow control mechanism using a memcached testbed. Experiments prove that our mechanism can accurately reduce the mean and 99th percentile response time by at least 71.3% and 69.4% respectively, while keeping a relatively high QPS. Furthermore, compared to static-threshold mechanism, our flow control mechanism is more efficient under low latency constraints.

Low response time is the requirement for high performance computing applications. The network latency is an important influence factor on response time. Busy poll socket (BPS) is a new mechanism that can reduce network latency by introducing extra energy cost to busily poll RX queue. Unlike hardware dependent solutions such as RDMA, busy poll socket can improve performance without specialized hardware and transplanting applications. BPS now becomes mature and is supported by many off-the-shelf network interface cards. In this paper, we compare the mechanisms of BPS and NAPI. We make a comprehensive analysis about both BPS and NAPI mechanisms. We analyze which factors would influence latencies in BPS and NAPI modes respectively. We find out that BPS does not definitely outperform NAPI, and the reasons are explored in detail. Our findings can provide suggestions on the future deployment of BPS based applications.

Short connections are generally used to transfer small-size messages, which contribute a large part of workload in modern applications. The maximum sustainable short connection rate, which is called short connection capacity, is an important index for admission control, Web QoS control, and energy saving. A capacity estimation mechanism aims to find the workload just saturating the server, and it relies on both workload information and system information. Past researches point out that kernel space network stack becomes the bottleneck when a huge number of concurrent short connections coexist. On the other hand, high performance user level network stacks have been proved to eliminate such bottleneck, thus become a hot research topic in both academia and industry. However, they also bring challenges for estimating short connection capacity, making traditional methods ineffective. Therefore, it is important to find a new method to estimate short connection capacity on high performance user level network stacks. In this paper, we prove that the effective CPU utilization is an adaptive index to different workload patterns and application complexities, which can reflect the server state. Then we design and implement an online capacity estimator on the Seastar platform. We conduct experiments to verify the effectiveness of our online capacity estimator. The results show that our estimator can actually estimate the capacity online. When the server is near saturated, the 90th percentile relative estimating error is no more than 9.18%. Furthermore, our capacity estimator only introduces no more than 1.38% of capacity loss in our experiments.

Recently, high performance packet I/O frameworks are expected an extensive application for their ability to process packets from 10Gbps or higher speed links. To achieve high throughput and low latency, high performance packet I/O frameworks usually employ busy polling technique. As busy polling will burn all CPU cycles even if there's no packet to process, these frameworks are quite power inefficient. Meanwhile, exploiting power management techniques such as DVFS and LPI in high performance packet I/O frameworks is challenging, because neither the OS nor the frameworks can provide information (e.g., the actual CPU utilization, available idle period, or the target frequency) required by power management techniques. In this paper, we establish an analytical model that can formulate the packet processing flow of high performance packet I/O to help address the above challenges. From the analytical model, we can deduce the actual CPU utilization and average idle period in different traffic load, and gain the insight to choose CPU frequency that can appropriately balance the power consumption and packet latency. Then, we propose two simple but effective approaches to conduct power conservation for high performance packet I/O: one with the aid of traffic information and the other without. Experiments with Intel DPDK show that both approaches can achieve significant power reduction (35.90% and 34.43% on average respectively) while incurring < 1 μs of latency increase.

Recent academic and industrial work is exploring the challenges of using RDMA over Ethernet, to support highly reliable, latency-sensitive services in today's datacenters. Previous work on the high-speed packet I/O like netmap, DPDK, etc., and high-performance user-level stacks like mTCP, IX etc., rekindles our inspirations to implement a high-performance software RDMA over commodity Ethernet devices.

Today's data center servers are equipped with high speed and complex network adaptors, featuring an array of functions, e.g. hardware TX/RX queues, packet filters, rate limiters, etc. Recent work like IX, Arrakis, MultiStack has made us rekindle the user-level network stacks' innovation utilizing these commodity network adaptors. In this paper, we revisit the idea to move stacks' design from in-kernel shared space into user-level application-specific dedicated one, for high performance and ease of development and deployment. We provide an unified control plane TAPM to exploit and manage the hardware adaptors' resources, and a dedicated data plane hwTAP to support different user-level stacks. TAPM and hwTAP highlight the utilization of hardware features from commodity network adaptors, to support the innovation of different user-level stacks. Experiments show that the hardware switching module can keep the input rate without any overheads and costs. TAPM could configure the hwTAP dynamically. Our run-to-completion user-level stack also achieves high throughput and low latency.

Since TCP Incast has been identified as a catastrophic problem in many typical data center applications, a lot of efforts have been made to analyze or solve it. The analysis work intends to model Incast problem from certain perspective, and the solutions try to solve the problem through designing enhanced mechanisms or algorithms. However, the proposed models are either closely coupled with particular protocol version or dependent on empirical observations, and the solutions cannot eliminate Incast problem entirely because the underlying issues are not identified completely. There is little work which attempts to close the gap between “analyzing” and “solving”, and present a comprehensive understanding. In this paper, we provide an in-depth understanding of how TCP Incast problem happens. We build up an interpretive model which emphasizes particularly on describing qualitatively how various factors, including system parameters and mechanism variables, affect network performances in Incast traffic pattern, but not on calculating the accurate throughput. With this model, we give plausible explanations why the various solutions for TCP Incast problem can help, but do not solve it entirely.