Stateful data plane network applications are indispensable but their efficiency is hindered by prevalent network device architectures which utilize a Blocking Scheme to maintain state consistency. The Blocking Scheme results in poor throughput and latency performance due to its frequent halts during packet processing. In response to this issue, we propose an innovative Non-Blocking Scheme and construct a theoretical model based on the G/GI/m queueing model. The new scheme leverages the speculative execution method, avoiding unnecessary blocking by taking advantage of the fact that the state update ratio is usually much smaller than the incoming packet rate. In case of speculation failures, the affected packets are reprocessed to guarantee state consistency. We provide an approximate model for the Blocking Scheme, and show that, even with relaxed approximations, the Blocking Scheme still performs worse than the Non-Blocking Scheme. The superior performance of the Non-Blocking Scheme is further corroborated through rigorous simulations, conducted under both realistic and synthetic traces. Based on our model, we propose an enhanced architecture: SN_RAPID (Sequence Number and unidirectional Reverse path-Augmented PIpeline Dataplane), to support speculative execution. The architecture is simpler than the other architectures supporting stateful network functions. Serving as a design foundation for future iterations, we implement a prototype of SN_RAPID in FPGA which can run at line speed, and also develop a software ASIC emulator. The experiments show the superiority of the improved architecture.

Network dataplane calls for better programmability. Current programmable network processing chips are based on either pipeline or multi-core Run-To-Completion (RTC) architecture with various trade-offs in flexibility, performance, and cost. The existing attempts to amalgamate the strengths of the two are stilted and inflexible. In this paper, we challenge the status quo by introducing a more fluid and organic programmable chip architecture, OptimusPrime, built from identical hardware blocks. Unlike the conventional static hybrid architecture, OptimusPrime allows each block to be transformed into either a pipeline stage processor or a multi-core RTC processor through software-defined configuration, enabling versatile data plane programming tailored to a wide range of applications (e.g., stateful packet processing and in-network computing). We integrate the C and P4 languages for application programming and develop algorithms to map a user program to the optimal distribution of pipeline stages and RTC cores. We demonstrate the viability of OptimusPrime through practical use cases such as in-network aggregation, in-network caching, and network function integration. We developed an FPGA-based prototype and a software-based ASIC simulator to validate the feasibility of OptimusPrime, which can be used by switches and smartNICs to enhance their programmability to a new level with high performance and low cost.