The rapid growth of Artificial Intelligence (AI) workloads has introduced unprecedented challenges to modern cloud-native systems, particularly in Kubernetes (K8s)-based environments. These workloads often demand low-latency communication, high resource locality, and efficient utilization of heterogeneous hardware devices such as Graphics Processing Units (GPUs) and specialized accelerators. However, the existing scheduling mechanisms in K8s are typically unaware of the underlying physical topology, leading to performance degradation and inefficient resource usage. This paper presents Sakkara, a novel topology-aware scheduling framework designed to optimize the placement of AI workloads in K8s clusters. Sakkara incorporates a hierarchical model of the Data Center (DC), including nodes and racks, enabling flexible scheduling strategies that account for resource availability and risk-aware metrics that mitigate performance interference and constraint violations caused by topology-unaware placement. Sakkara extends existing scheduling logic in K8s with placement strategies that guide pod allocation using configurable topology constraints, aiming to minimize communication costs and maximize workload performance. We evaluated Sakkara on a representative AI workload, a distributed training application under different cluster configurations. Experimental results show that Sakkara improves job completion time, throughput, and memory utilization compared to available K8s schedulers, achieving improvements of up to 10%. Sakkara, available as open-source, offers a promising pathway toward topology-conscious orchestration of AI workloads in next-generation cloud environments.

The rapid explosion of Artificial Intelligence (AI) workloads utilizing a growing number of accelerators has placed unprecedented demand on the network. These workloads typically leverage Remote Direct Memory Access (RDMA) and require a high-performance network fabric. While many purpose-built cloud networking solutions can provide high performance, efficiently utilizing these costly infrastructures require a fabric that is multi-tenant for ease of consumption. Furthermore, the fabric must be resilient to faults and for operational manageability. Resilient cloud networks typically employ mature Ethernet segmentation techniques over Clos topologies with Equal Cost Multi Path (ECMP) routing. ECMP hashes flows to paths, which in case of collisions can significantly degrade performance for large RDMA over Converged Ethernet (RoCE) flows. To mitigate ECMP penalties, we evaluate routing strategies with varying levels of operational complexity. We explore load balancing and path pinning solutions that leverage non-proprietary, mature technologies over commodity Ethernet. Our evaluation follows a three-fold strategy, focusing on the key dimensions of performance, resiliency, and operational complexity. By applying this methodology to representative implementations, we highlight the trade-offs. While all techniques are resilient, path pinning-based solutions excel at performance but introduce greater complexity. Specifically, path pinning achieves up to 1.6× improvement over ECMP for RoCE test traffic and up to 2.5× for NCCL AllReduce. These results validate the promising performance benefits of path pinning and highlight the need to explore less complex implementations for broader adoption. Our methodology can be used to rigorously evaluate future implementations in support of AI network design.

Vela is a cloud-native system designed for LLM training workloads built using off-the-shelf hardware, Linux KVM-based virtualization, and a virtualized RDMA over Converged Ethernet (RoCE) network. Vela virtual machines (VMs) support peer-to-peer DMA between the GPUs and SRIOV-based network interface. In this paper, we share Vela's key architectural aspects with details from an NVIDIA A100 GPU-based deployment in one of the IBM Cloud data centers. Throughout the paper, we share insights and experiences from designing, building, and operating the system over a ~2.5 year timeframe to highlight the capabilities of readily available software and hardware technologies and the improvement opportunities for future AI systems, thereby making AI infrastructure more accessible to a broader community. As we evaluated the system for performance at ~1500 GPU scale, we achieved ~80% of the ideal throughput while training a 50 billion parameter decoder model using model parallelism, and ~70% per GPU FLOPS compared to a single VM with the High-Performance Linpack benchmark.

Modern Artificial Intelligence (AI) and HighPerformance Computing (HPC) workloads impose diverse demands on data center networks in terms of both performance and reliability. Applications such as inferencing are usually compute-bound, while distributed training and HPC workloads are network-bound, requiring high bandwidth and low, predictable latency. In terms of reliability, some workloads require seamless failure recovery, while others can tolerate failures through checkpointing or application/transport-layer reliability. In this paper, we address the challenge of designing a highperformance network for private data centers at scales of 1-2K Graphics Processing Units (GPUs) or other accelerators, supporting a range of workloads over Remote Direct Memory Access (RDMA), RDMA over Converged Ethernet (RoCE), and Transmission Control Protocol (TCP) with off-the-shelf hardware and software. We first present a network control plane design that accommodates flexible reliability and performance requirements, and we present our approach across key system components to efficiently utilize multiple equal-cost paths in a two-level leafspine topology. We then use simulations to guide our design choices, including a 1:4 speed ratio between server-facing and spine-facing ports, along with switch partitioning at the spine layer to mitigate the impact of flow collisions. Leveraging Ansible automation for efficient configuration management, we integrate these findings into a 12 -node cluster with 8 GPUs and 1.6 Tbps bandwidth per server. Using network micro-benchmarks that mimic the demands of intensive workloads, our results show that the network sustains near-line-rate throughput under traffic patterns where up to two-thirds of traffic traverses the spine, while also supporting a flexible reliability model.

The exponential growth in Artificial Intelligence (AI) adoption presents unique challenges and opportunities for deploying AI workloads in modern Data Center (DC) networks, particularly in terms of performance, scalability, and reliability. AI workloads, such as inference and distributed training, impose different network demands: inference is primarily computebound and typically requires low network latency, while distributed training is network-bound and requires high bandwidth, placing significant strain on the network. This paper focuses on the network requirements of widely known AI communication patterns, and studies their impact on modern DC architectures by analyzing the effects of different orchestration strategies-specifically packing and spreading-on throughput, response time, and network congestion. The results show that packing strategies generally deliver higher performance for most covered AI collectives. However, spreading strategies can be beneficial in certain scenarios, such as when larger workloads span across higher number of racks, as they can help mitigate network congestion between the switches of leaf-spine network configurations. This paper offers valuable insights into optimizing the orchestration of popular AI collectives in data center networks, presenting informed strategies to improve performance in response to growing AI demands, with findings demonstrating completion time reductions of up to 30 %.

Efficient placement of advanced HPC and AI workloads with application constraints is raising challenges for resource schedulers on shared infrastructures, such as the Cloud. In this work, we propose a novel Constraints- and Heuristics-based scheduler on HIerarchical Topologies for High-Performance Computing workloads in the Cloud (chic-sched, for short). Our heuristics-based algorithm enables placement across multiple levels in a network hierarchy with loosely specified constraints, and it works without retries by providing suboptimal placements to minimize placement failures. This allows for fast scheduling at scale, and the O(N log N) complexity enables placement decisions within tens of milliseconds for groups of hundreds of virtual machines (VM). We introduce a new and simple metric to quantify the goodness of group placements. With this metric, in terms of deviation from ideal placements, we show that chic-sched is 20-50% better than the common bestFit or worstFit algorithms in all scenarios of two-level placements with spreading and packing constraints. We evaluate chic-sched with publicly available VM-request traces from a production Cloud, and, comparing against bestFit, we show that it achieves 8% lower placement failure rates and more than 40% better placement locality. Finally, to quantify the goodness of constraints-based placements, we conduct experiments with a realistic MPI workload on synthetically allocated VM clusters in a public cloud. We measure a 9% performance improvement over an adverse placement in a scenario where our heuristics-based scheduler would return a good, but not perfect, placement.

In order to meet the ever-increasing traffic demands, the combination of fiber and Millimeter Wave (mmWave) is expected to play a key role for 5G Centralized-Radio Access Networks (C-RANs). Due to the inefficiency of the Common Public Radio Interface for the Baseband Unit (BBU)-Remote Radio Head (RRH) communication, analog-Radio-over-Fiber (a-RoF) technology is considered a promising solution, mainly due to the RRH simplification and lower fronthaul requirements it imposes. In such mmWave a-RoF C-RANs, efficient Medium Transparent-Medium Access Control (MT-MAC) protocols are needed able to meet the challenging 5G requirements. To this end, in this paper, we propose a gated service MT-MAC protocol which authorizes each user to transmit the amount of data it requested. A detailed delay model is proposed, which is validated through simulations for different fiber lengths, network load conditions and number of available optical wavelengths. Moreover, the proposed protocol is compared with the state-of-the-art (SoA) and is shown to achieve up to 20 times higher throughput, 2 times lower delay with 100% lower blocking probability and 5 times higher data wavelength utilization, while being able to adapt to varying network traffic conditions. Our proposal also attains sub-ms latency in most cases, constituting it a promising candidate for 5G mmWave a-RoF C-RANs.

Fifth Generation (5G) Cloud-Radio Access Networks (C-RANs) are about to exploit both optical and Millimeter Wave (mmWave) technology to meet the ever-increasing traffic demands. In this new type of converged Fiber-Wireless (FiWi) systems efficient Medium Transparent-Medium Access Control (MT-MAC) protocols should be designed, able to satisfy the very strict 5G service requirements. To this end, in this paper, we propose an MT-MAC protocol for mmWave Analog Radio-over-Fiber (A-RoF) C-RANs, which employs gated service, according to which users are granted transmission windows equal to the number of bytes contained in their buffer. An analytical model is also proposed for the mean packet delay, which is verified by means of simulation for different fiber length values, network load conditions and optical capacity values. Our results not only prove the accuracy of the proposed model but also the suitability of the proposed MT-MAC protocol to meet the sub-ms delay challenge of latency-critical 5G network requirements.

The upcoming generation of mobile networks is expected to serve numerous mobile users with high quality-of-service (QoS) demands, requiring high-capacity fronthaul. As the provision of fiber connections directly to the end users is not cost-efficient, the integrated fiber wireless (FiWi) fronthaul design based on wireless networking and passive optical networks (PONs) has been proposed. The FiWi design involves modern networking technologies that can accommodate the need for data rates in the Gb/s scale and low delay, such as the wavelength division multiplexing (WDM) in the optical domain and the multiple input multiple output (MIMO) communication over millimeter wave (mmWave) spectrum in the wireless domain. The co-existence of two network types requires resource management in a medium transparent manner, i.e., the sharing of the bandwidth in the wireless domain should allow the organization of the data packets in optical frames. As the traffic circulating in the FiWi fronthaul involves packets of different priorities, i.e., different QoS classes, the resource management scheme should support QoS differentiation. To this end, we propose a resource management scheme for FiWi fronthaul and we extensively study its performance in terms of experienced delay and throughput. Our simulation results demonstrate that the proposed scheme significantly reduces the delay of the high priority class.