Optimizing request routing in large microservice-based applications is difficult, especially when applications span multiple geo-distributed clusters. In this paper, inspired by ideas from network traffic engineering, we propose Service Layer Traffic Engineering (SLATE), a new framework for request routing in microservices that span multiple clusters. SLATE leverages global knowledge of cluster states and multi-hop application graphs to centrally control the flow of requests in order to optimize end-to-end application latency and cost. Realizing such a system requires tackling several technical challenges unique to service layer, such as accounting for different request traffic classes, multi-hop call trees, and application latency profiles. We identify such challenges and build a preliminary prototype that addresses some of them. Preliminary evaluations of our prototype show how SLATE outperforms the state-of-the-art global load balancing approach (used by Meta's Service Router and Google's Traffic Director) by up to 3.5× in average latency and reduces egress bandwidth cost by up to 11.6×.

Datacenter systems require real-time troubleshooting so as to minimize downtimes. In doing so, datacenter operators employ streaming analytics for collecting and processing datacenter telemetry over a temporal window. Quantile computation is key to this telemetry monitoring since it can summarize the typical and abnormal behavior of the monitored system. However, computing quantiles in real-time is resource-intensive as it requires processing hundreds of millions of events in seconds while providing high accuracy. To address these challenges, we propose AOMG, an efficient and accurate quantile approximation algorithm that capitalizes insights from our workload study. AOMG improves performance through two-level hierarchical windowing while offering small value errors in a wide range of quantiles by taking into account the density of underlying data distribution. Our evaluations show that AOMG estimates the exact quantiles with less than 5% relative value error for a variety of use cases while providing high throughput.