Virtual switch is a fundamental component of cloud computing as it provides core networking functionalities for VMs and containers. Open vSwitch (OVS) is widely adopted in cloud environments due to its open-source nature, programmability, and rich set of features. At ByteDance, we initially adopted OVS in our public cloud, but as our cloud business grew, its generic design along with its complex code-base quickly became obstacles to improvements. Hence, we developed Byte vSwitch (BVS), a high-performance virtual switch that was specifically designed to address the performance, scalability, serviceability, and operational efficiency needs of our cloud services. More specifically, BVS adopts a simple architecture with an optimized hash table to maximize forwarding performance. In addition, we introduced several optimizations to improve BVS scalability, operability, and serviceability in cloud environments. Our evaluations show that BVS achieves up to 3.3× higher PPS and 25% lower latency compared to OVS. BVS has been deployed at scale across all regions of the ByteDance public cloud for over four years, and this paper presents our experience in designing, deploying, and operating BVS in production.

Layer-4 load balancers play a critical role in large-scale data centers. Recently, load balancers implemented on programmable switches have attracted much attention since they overcome the inflexibility of dedicated load balancers and high latency of software load balancers. However, keeping per-connection state easily leads to storage exhaustion, especially under resource exhaustion attacks. Although several stateless load balancers are proposed to address this issue, the state management burden is offloaded to backend servers, causing high deployment and running costs. In this paper, a load balancer called Loom with compressed states is proposed for large-scale data centers. Firstly, we propose a novel classifier-based load balancer idea to avoid directly maintaining per-connection state. Then, a circulating Bloom filter structure is proposed that can efficiently classify connections as well as be implemented on existing programmable switches. Theoretical analysis shows that Loom can maintain 11 ~ 30x more concurrent connections than those directly storing the 5-tuple of connections. Loom is implemented in hardware P4 switches and experimental results indicate that 11 ~ 29x more concurrent connections can be maintained in Loom, which is close to the theoretical results. Besides, Loom is resistant to resource exhaustion attacks and reduces the percentage of broken connections by up to 57% with an SYN flood.

Large-scale services are generally hosted on multiple application servers to scale out in today's data centers. Load balancers distribute users' requests across these servers. Software load balancer and switch-based load balancer are two typical classes of load balancers. However, most of the existing mechanisms either exhibit high processing latency at load balancers or likely lead to unbalanced requests distribution without considering the disparity of the application servers. In this paper, we study how the disparity of application servers significantly impacts the response time of requests. A fast switch-based Load Balancer considering Application Server states (LBAS) then is proposed to minimize the processing latency at both load balancers and application servers. The data plane of LBAS is well designed to store millions of connections in limited storage capacity without violating per-connection consistency. Besides, a partial dynamic weighting algorithm based on the Ridge Regression theory is designed and implemented to decrease the processing latency at application servers. We implement LBAS using the P4 programming language and conduct a series of extensive experiments to evaluate the performance. The results demonstrate that the proposed LBAS mechanism significantly reduces the response time of requests compared with Uniform random, Static weight, and Spotlight in various scenarios.