Internet companies are constructing large-scale AI clusters with commodity Ethernet switches for AI model training to support their businesses. AI training workloads impose stringent network requirements, mandating that cluster networks deliver high peak throughput while maintaining robustness and resilience in the face of link failures. We present SGLB, a distributed, global congestion-aware load balancing system for AI clusters. SGLB operates a control-plane protocol, SyncMesh, to enable a new load balancing abstraction in modern commodity switches—Global Load Balancing (GLB) engine—which utilizes global congestion information to distribute traffic across all available paths. We address three key challenges in designing SGLB: fast routing convergence to minimize downtime in the event of link failures, scalable maintenance of congestion profiles within the constraints of limited switch hardware resources, and preventing GLB throughput suppression in scenarios where path bandwidths are asymmetric. We prototype SGLB and conduct extensive experiments to evaluate SGLB. SGLB ensures rapid routing convergence in the event of link failures, recovering in as little as 45 μs to guarantee network robustness for long-term, stable model training. Additionally, SGLB effectively load-balances traffic across paths, avoiding those with global congestion, which accelerates All-to-All collective communication by up to 60%.

On-chip photonic waveguides have been proposed as a feasible replacement for the long interconnects that cause speed and power bottlenecks. Along with recent advancements in nanophotonic technologies, we believe that combining on-chip waveguides with high-radix Network on Chip (NoC) topologies is a promising way to improve NoC performance. In this paper, we propose the BufferLess phOtonic ClOs Network (BLOCON) to exploit silicon photonics. We propose a scheduling algorithm named Sustained and Informed Dual Round-Robin Matching (SIDRRM) to solve the output contention problem, a path allocation scheme named Distributed and Informed Path Allocation (DIPA) to solve the Clos network routing problem, and a methodology to achieve an optimal off-chip laser-power budget. In the simulation results, we show that with SIDRRM and DIPA, BLOCON improves the delay and on-chip power performance of the compared electrical and photonic NoC architectures over synthetic traffic patterns and SPLASH-2 traces.

On-chip photonic waveguides have been proposed as a feasible replacement for the long interconnects that cause speed and power bottlenecks. Along with recent advancements in nanophotonic technologies, we believe that combining on-chip waveguides with high-radix Network on Chip (NoC) topologies is a promising way to improve NoC performance. In this paper, we propose the BLOCON (BufferLess phOtonic ClOs Network) to exploit silicon photonics. We propose a scheduling algorithm named Sustained and Informed Dual Round-Robin Matching (SIDRRM) to solve the output contention problem, and a path allocation scheme named Distributed and Informed Path Allocation (DIPA) to solve the Clos network routing problem. In the simulation results, we show that with SIDRRM and DIPA, BLOCON improves the delay and power performance of the compared electrical and photonic NoC architectures over synthetic traffic patterns and SPLASH-2 traces.

Many high-radix network-on-chip (NoC) topologies have been proposed to improve network performance with an ever-growing number of processing elements (PEs) on a chip. We believe high-radix Clos network-on-chip (CNoC) is the most promising with its low average hop counts and good load-balancing characteristics. In this paper, we propose: 1) a high-radix router architecture with virtual output queue (VOQ) buffer structure and packet mode dual round-robin matching (PDRRM) scheduling algorithm to achieve high speed and high throughput in CNoC; 2) the design of hierarchical round-robin arbiter for high-radix high-speed NoC routers; and 3) a heuristic floor-planning algorithm to minimize the power consumption caused by the long wires. Experimental results show that the throughput of a 64-node three-stage CNoC under uniform traffic increases from 62% to 78% by replacing the baseline virtual channel routers with PDRRM VOQ routers. We also compared the delay, power, and area performance of the 64-node CNoC with other NoC topologies under various synthetic traffic patterns and SPLASH-2 benchmark traces. The simulation results show that in general CNoC improves the throughput, low-load delay, and energy efficiency over the compared NoC topologies.

Many high-radix Network-on-Chip (NOC) topologies have been proposed to improve network performance with an ever-growing number of processing elements (PEs) on a chip. We believe Clos Network-on-Chip (CNOC) is the most promising with its low average hop counts and good load-balancing characteristics. In this paper, we propose (1) a high-radix router architecture with Virtual Output Queue (VOQ) buffer structure and Packet Mode Dual Round-Robin Matching (PDRRM) scheduling algorithm to achieve high speed and high throughput in CNOC, (2) a heuristic floor-planning algorithm to minimize the power consumption caused by the long wires. Experimental results show that the throughput of a 64-node 3-stage CNOC under uniform traffic increases from 62% to 78% by replacing the baseline routers with PDRRM VOQ routers. We also compared CNOC with other NOC topologies, and found that using the new design techniques, CNOC has the highest throughput, lowest zero-load latency, and best power efficiency.