Virtual switch is a fundamental component of cloud computing as it provides core networking functionalities for VMs and containers. Open vSwitch (OVS) is widely adopted in cloud environments due to its open-source nature, programmability, and rich set of features. At ByteDance, we initially adopted OVS in our public cloud, but as our cloud business grew, its generic design along with its complex code-base quickly became obstacles to improvements. Hence, we developed Byte vSwitch (BVS), a high-performance virtual switch that was specifically designed to address the performance, scalability, serviceability, and operational efficiency needs of our cloud services. More specifically, BVS adopts a simple architecture with an optimized hash table to maximize forwarding performance. In addition, we introduced several optimizations to improve BVS scalability, operability, and serviceability in cloud environments. Our evaluations show that BVS achieves up to 3.3× higher PPS and 25% lower latency compared to OVS. BVS has been deployed at scale across all regions of the ByteDance public cloud for over four years, and this paper presents our experience in designing, deploying, and operating BVS in production.

A Virtual Private Cloud (VPC) that enables both RDMA and TCP provides advantages for both tenants and cloud providers. It serves the flexible demands of RDMA and TCP of tenant applications while delivering a cost-effective solution compared to the construction of two distinct overlay networks. In this study, we introduce Jakiro, an innovative framework of vNIC design that supports both RDMA and TCP within ByteDance Cloud. Jakiro holds the capability to support fundamental VPC features such as QoS, security groups, etc., for both RDMA and TCP streams while maintaining compatibility with applications and intra-host RDMA optimization techniques. We benchmark Jakiro's performance using basic test cases and real-world high-performance computing applications and distributed machine learning training. The results indicate that the RDMA performance of Jakiro is close to that of the physical RDMA. Concurrently, Jakiro guarantees a weighted fair QoS between RDMA and TCP. Jakiro has been deployed in ByteDance Cloud for one year, we share our critical design and deployment decisions, as well as experiences and lessons from production.

Software-Defined Networking (SDN), enabled by OpenFlow, represents a paradigm shift from traditional network to the future Internet. Compared with traditional networks, its controllers and network applications have higher requirements on the availability and scalability. This paper proposes OFBGP, a new scalable, highly available architecture of BGP totally based on distributed design, without any centralized component. OFBGP is an application of SDN controller. In the design of OFBGP, BGP functionalities are divided according to their characteristics and requirements into two modules: BGP Protocol and BGP Decision. By doing so, the application can easily scale out when the number of BGP neighbors increases, and the process of BGP routing decisions can be parallelized. The paper also presents a new solution to implement BGP NSR through snapshots and backup event flow so that from a historical snapshot, replaying real-time backup event flow can rebuild the state that has happened behind at any moment. When a fault occurs, we can easily restore a state to ensure BGP neighbors do not perceive failure. We have also built a prototype to demonstrate our architecture. After the test of the performance of some core modules, the result shows that our new architecture can help improve the availability and scalability of BGP.

Software-Defined Networking (SDN), enabled by Open Flow, represents a paradigm shift from traditional network to the future Internet. Replicate or distributed controllers have been proposed to address the issues of availability and scalability that a centralized controller suffers from. However, it lacks a flexible mechanism to balance load among distributed controllers. To address this problem, this paper presents DALB, a dynamic and adaptive algorithm for controller load balancing totally based on distributed architecture, without any centralized component. This algorithm is running as a module of SDN controller. On one hand, it adopts an adjustable load collection threshold so as to reduce the overhead of exchanging messages for load collection, and on the other hand it can make policy and election locality in order to reduce the decision delay caused by network transmission. In this paper, we build the prototype system on floodlight to demonstrate our design and test the performance of our algorithm.

Network controller is the core of the OpenFlow-Based Software-Defined Networks (SDN). High availability of network controller is an urgent need objectively. The appearance of controller instance fault should not be perceived by data plane. During the fault recovery, OpenFlow messages, especially asynchronous message from data plane, should not be discarded. Otherwise the control platform would hold the outdated network status. This is an interesting problem which we call Asynchronous Message Discard (ASMD) trouble. This paper proposes a Non-stop network controller (NSNC). It can troubleshoot the ASMD problem based on highly available TCP (HA-TCP) technology. During the fault recovery on distributed control platform, we want to elect the optimal controller to take over the network devices managed by the failed controller instance. We analyses the minimum metrics set for controller election. The view of this paper is verified on Floodlight controller. Experiments show that when the controller instance fails, asynchronous messages from switch will not be discarded, and using HA-TCP only adds negligible overhead to throughput and latency. We test the performance of election algorithm based on Mininet. Currently, there is only a theoretical analysis of metrics related to the election. We will test the validity of the election strategy in a real scenario as part of our future work.