Network packets have traditionally been processed by purpose built, fixed-function appliances using proprietary hardware and software designs. Today, these designs are transforming into collections of flexible software blocks running on general purpose computing hardware, with expectations of high performance and scalability. Among these pieces, software packet switching (i.e. vSwitch) plays a fundamental role, as all packets need to traverse the switch on a given platform, and quite often repeatedly. In this paper, we demonstrate that certain performance and scaling deficiencies of packet switching in software can be attributed to the centralized design and deployment of the switching software threads. While multithreading is crucial for scaling-out software performance, improper use of the multi-processor resources can lead to undesired impacts on performance, due to fixed-partitioning, sub-optimal scheduling and complex cache coherency protocol interactions involving non-negligible data-access latencies. In this study, we take advantage of symmetric multi-processor threading (SMT) and lay out principles for better software packet switching with a distributed approach. With proof of concept implementations based on the very popular Open vSwitch project and DPDK, we show distributed software switching can easily scale with the network function workload threads, and achieve more than 2× throughput by exploiting the data locality within distributed nature of the CPU cache-subsystem.

Software switch has emerged as a critical component in software defined networking and network virtualization areas. Open vSwitch (OvS) is a widely used software switch which uses tuple space search algorithm for packet classification, and an exact match cache (EMC) for caching most frequently used flows. In this paper, we propose two new optimizations for OvS to further improve its performance and scalability. First aims to completely remove the sequential search overhead of the tuple space search layer of OvS, and second is a dynamic cache insertion optimization for the EMC to improve EMC effectiveness. We show that the optimizations can improve OvS's throughput by up to 3.5X for millions of active flows.

As the number of cores in a multicore system increases, core-to-core (C2C) communication is increasingly limiting the performance scaling of workloads that share data frequently. The traditional way cores communicate is by using shared memory space between them. However, shared memory communication fundamentally involves coherence invalidations and cache misses, which cause large performance overheads and incur a high amount of network traffic. Many important workloads incur significant C2C communication and are affected significantly by the costs, including pipelined packet processing which is widely used in software-based networking solutions. In these workloads, threads run on different cores and pass packets from one core to another for different stages of processing using software queues.

To provide the ultimate mobile user experience, extended battery life is critical to small form-factor mobile platforms such as smartphones and tablets. Dynamic voltage and frequency scaling (DVFS) and low-power CPU/platform sleep states are commonly used power management features, as they allow dynamic control of power and performance to the time-varying needs of workloads. Despite the potential power saving benefit from synergistic integration of DVFS and sleep-state selection, it is challenging to optimize them jointly for mobile workloads (e.g., video streaming), and most existing work considers them only individually. To address this problem, we study joint optimization of CPU frequency (a.k.a. CPU P-states) and CPU/platform sleep-state selections to reduce energy consumption in mobile platforms. This joint optimization becomes feasible with advanced power management techniques and power aware software development methodologies that regulate (e.g., coalesce/align) system activities, making workload characteristics and system idle duration more deterministic and predictable. We then analyze the optimal operating state that minimizes the expected platform energy consumption based on workload characteristics, and present an algorithm to adapt to it at run time. Our evaluation results on mobile workloads show that the proposed scheme can reduce system power consumption by up to 24%, compared to the conventional CPU-utilization-based approach, which seeks mainly to minimize processor energy.

Battery life remains to be a critical competitive metric for today's mobile platforms that offer ubiquitous connectivity through their wireless communication interfaces. With most usage models being driven by always-on communication activities, e.g, Internet video streaming, web browsing, etc., it is imperative to understand the impact of network activities on the overall platform power, and optimize power consumption for such activities. As shown by our investigation, various real-world network-driven workloads exhibit bursty and random behavior, which motivates our work on regulating and coalescing incoming packets to reduce platform wake events. To understand the performance impact of packet coalescing, we conduct an extensive investigation to study how coalescing may affect the throughput and user experience. Armed with the deep understandings, we propose, implement and evaluate an Adaptive Traffic Coalescing (ATC) scheme that monitors the incoming traffic at the Network Interface Card (NIC), and adaptively coalesces the packets for a limited duration in the NIC buffer, thus requiring no network or eco-system support. The proposed ATC scheme effectively reduces platform wake events, and enables the platform to enter and stay in the low-power state longer for energy efficiency. We have implemented the scheme in commercial wireless NICs. Using various mobile platforms, we evaluate the power savings and performance impact of the proposed ATC scheme. Experiments show that ATC achieves significant power saving for major platform components, around 20% for real-world Internet workloads, without impacting performance and user experience.

Mobile platforms (e.g. laptops) offer ubiquitous network connectivity through its wireless communication interfaces, with most of usage models driven by always-on communication activities. However this in turn creates significant power challenges as the battery life is a critical resource for all mobile platforms. While the communication device itself consumes a relatively small portion of the total power, the impact of communication on the overall platform power is significant, due to the non-deterministic nature of the network traffic, which tends to keep the platform busy more than necessary. In this paper, we extensively investigate characteristics of various real- world network-centric workloads, e.g., VoIP, web browsing, etc., and their impact on the mobile platform in terms of power consumption and energy. Based on the understandings, we propose, implement and evaluate an Interrupt /DMA (Direct Memory Access) Coalescing (IDC) scheme at the wireless network interface card (NIC). This scheme effectively reduces platform wakeups due to incoming network traffic, Another advantage of the scheme is that it requires only NIC modification at the receiving node, and is transparent to the user and the network. Using a commercial NIC with a laptop testing platform, we evaluate the power savings and performance impact of our proposed scheme. The measurements show that our scheme achieves significant platform power saving, around 25% of platform components, without impacting user experience.