RDMA is the preferred ultra-low-latency transport in modern datacenters. As services decompose into latency-critical microservices, servers maintain hundreds to thousands of connections across tiers. However, RDMA's polling-based RPC notification struggles at high core and connection counts, forcing a tradeoff between idle polling, inter-core synchronization costs, and load imbalance—each limiting throughput for$\mu \mathrm{s}$-scale services. We introduce Sassy, a smartNIC-based notification mechanism that breaks this tradeoff, optimizing the performance and scalability of servicing$\mu \mathrm{s}$-scale RPCs over high connection counts. Sassy eliminates performance overheads stemming from idle polling and inter-core synchronization, while balancing the load of active RDMA connections across available cores. We implement Sassy on an FPGA-enhanced NIC and evaluate it on a range of traffic patterns, as well as a high-performance key-value store. For deployment scenarios with high connection counts, Sassy improves throughput under SLO by up to$2.3 \times$.

In modern cloud infrastructures, multi-tenancy and virtualization are common, yet optimizing resource allocation while maintaining performance is challenging due to complex Solid State Device (SSD) physics. Over-provisioning resources often leads to revenue loss and higher infrastructure costs. To address this, we introduce DILATON, an adaptive closed-loop scheduler that balances bandwidth and QoS across tenants. By predicting die loading and using real-time adaptation, DILATON maximizes resource allocation, achieves performance gains, and reduces costs while meeting SLAs. This paper uses simulations to emphasize the necessity of a closed-loop QoS control.

Online services are decomposed into fine-grained software components that communicate over the network using fine-grained Remote Procedure Calls (RPCs). Inter-server communication often exhibits patterns of wide RPC fan-outs between software tiers, raising the well-known tail at scale effect and necessitating mechanisms that curb long response tail latencies. When handling µs-scale RPCs, request distribution across the cores of multicore servers is a major determinant of the resulting tail latency. Software approaches for inter-core RPC balancing introduce considerable overheads, throttling a server’s peak throughput. On the other hand, existing NIC-based hardware mechanisms ameliorate software and inter-core synchronization overheads, but result in inter-core load imbalance that leaves significant performance improvement headroom.We introduce Turbo, a hardware on-NIC load-balancing mechanism that achieves near-optimal inter-core load distribution for the most fine-grained, light-tailed RPCs with service times of only a couple of µs. We implement Turbo on a programmable NIC and evaluate it on a range of different service time distributions and with a high-performance Key-Value store. Compared to hardware NIC-based mechanisms that statically spread load across cores, Turbo boosts throughput under a 99% latency Service Level Objective (SLO) of 30× the service time by up to 5×, and by up to 95× for a more aggressive 10× SLO target.