Container orchestration platforms have become the backbone of modern private cloud infrastructure, offering flexibility, scalability, and reliability for managing containerized workloads. Among them, Kubernetes has emerged as the de facto standard, widely adopted across the industry, with many organizations operating tens or even hundreds of clusters globally. However, as infrastructure scales to ultra-large clusters (exceeding 10,000 nodes) and faces extreme provisioning scenarios—such as abrupt workload surges, irrespective of available resource headroom—scheduling latency becomes a critical bottleneck. This issue is not adequately addressed by the default Kubernetes scheduler or existing alternative solutions, which struggle to maintain low latency under such conditions.

Over the last few years, at ByteDance, our compute infrastructure scale has been expanding significantly due to expedited business growth. In this journey, to meet hyper-scale growth, some business groups resorted to managing their own compute infrastructure stack running different scheduling systems such as Kubernetes, YARN which created two major pain points: the increasing resource fragmentation across different business groups and the inadequate resource elasticity between workloads of different business priorities. Isolation across different business groups (and their compute infrastructure management) leads to inefficient compute resource utilization and prevents us from serving the business growth needs in the long run.

In many application areas, including control systems, careful management of system resources is key to providing the best application performance. Traditional control systems with multiple control loops statically allocate a fixed portion of the system resources to each controller based on their average or worst-case resource requirements. However, controllers' resource needs vary depending on the jobs they perform and the state of the systems they control. A controller of a plant operating close to its equilibrium requires fewer resources than a controller of a plant operating far from its equilibrium point. The Draco dynamic rate control system exploits this fact by dynamically allocating resources to control systems based on system state. Our research demonstrates that Draco provides significantly better overall control performance with much less resources than static controllers. Our experimental evaluation shows that in the control scenarios we examined Draco provides up to 25% better control performance with 30% less resources.

The simple notion of soft real-time processing has fractured into a spectrum of diverse soft real-time types with a variety of different resource and time constraints. Schedulers have been developed for each of these types, but these are essentially point solutions in the space of soft real-time and no detailed unified definition of soft real-time has previously been provided that includes all types of soft realtime processing. We present a complete real-time taxonomy covering the spectrum of processes from best-effort to hard real-time. The taxonomy divides processes into nine classes based on their resource and timeliness requirements and includes four soft real-time classes, each of which captures a group of soft real-time applications with similar characteristics. We exploit the different features of each of the soft real-time classes to integrate all of them into a single scheduler together with hard real-time and best-effort processes and present results demonstrating their performance

Modern operating systems frequently support applications with a variety of timing constraints including hard real-time, soft real-time, and best-effort. To guarantee performance, critical applications typically over-reserve resources based on worst-case resource usage estimates, while others may reserve based on average-case or other estimates. When resources are fully subscribed, the performance of soft- and non-real-time applications depends upon the effective distribution of dynamic slack - reserved, but unused resources - from other tasks. Motivated by several representative examples, we derive four general principles for the effective management of slack. We have implemented these principles in four progressively better slack schedulers that demonstrate their effectiveness. BACKSLASH, which employs all four principles, misses fewer soft realtime deadlines than all of the other slack schedulers we examined

In many application areas, including control systems, careful management of system resources is key to providing the best application performance. Most traditional resource management techniques for real-time systems with multiple control loops are based on open-loop strategies that statically allocate a constant CPU share to each controller, independent of their current resource needs. This provides average control performance with minimal overhead but in general fails to provide the best performance possible within the available resources. We show that by using feedback to dynamically allocate resources to controllers as a function of the current state of their controlled systems, control performance can be significantly improved. We present an optimal resource allocation policy that maximizes control performance within the available resources and provide experimental results showing that the optimal policy 1) significantly increases control performance compared to traditional control system implementations (by more than 20% in our experiments), 2) maximizes control performance over other feedback-based policies, 3) saves resources when perturbations occur infrequently, and 4) incurs negligible overhead.

Real-time systems are growing in complexity and real-time and soft real-time applications are becoming common in general-purpose computing environments. Thus, there is a growing need for scheduling solutions that simultaneously support processes with a variety of different timeliness constraints. Toward this goal we have developed the resource allocation/dispatching (RAD) integrated scheduling model and the rate-based earliest deadline (RBED) integrated multi-class real-time scheduler based on this model. We present RAD and the RBED scheduler and formally prove the correctness of the operations that RBED employs. We then describe our implementation of RBED and present results demonstrating how RBED simultaneously and seamlessly supports hard real-time, soft real-time, and best-effort processes.