Multi-tenant GPU clusters are designed to concurrently run multiple distributed ML training workloads. However, frequent data transfers among GPUs via collective operations can slow down training, as collectives from different tenants compete for network bandwidth. Recent work (e.g., CASSINI) has considered collective coordination to prevent network contention, but primarily focused on static job-level optimizations at deployment time, oblivious to runtime network conditions and the specific traffic pattern of each workload. In this paper, we present Symphony, an application-layer solution that dynamically coordinates collective operations across tenants at runtime. Symphony integrates seamlessly with existing clusters with minimal modifications to the collective communication library and includes a lightweight online scheduling mechanism that requires no advance information about the workloads or their collectives. We evaluate Symphony using both a real GPU testbed implementation and trace-driven simulations. Specifically, using realistic ML workloads in our testbed, we observe improvements of up to 13.2% in average communication time and 9.6% in training time compared to state-of-the-art solutions.

Next generation applications for datacenters, such as Distributed Machine Learning (DML) and Big Data, have complex communication patterns that demand a scalable, stateless and application-aware optimal transport protocol to maximize network utilization and improve application performance. Recent transport protocols either provide limited benefits due to lack of information sharing between application and network; or implement complex stateful mechanisms to improve the application performance. In this paper, we present Omni- Transport Mechanism (Omni-TM) as a message-based congestion control protocol. Omni-TM allows exchanging message information with the network to negotiate the optimal transmission rate without maintaining a per-flow state at the switches (i.e., stateless). Omni- Tmis designed to reach maximum link capacity in one shot. Our simulation results show that Omni- Tmdemonstrates better traffic control decisions (i.e., close to zero queue length while maintaining high link utilization). Furthermore, Omni- Tmreduces Flow Completion Time (FCT) up to 45 % in a realistic workload compared to DCTCP.

The recent applications in communications are required to meet low-latency transmission with high traffic rates and reliabilities. From the latency point of view, most of the state-of-the-art techniques consider the average latency which does not directly apply to delay-sensitive scenarios. In this paper, we propose a novel approach to tackle the scheduling problem by directly addressing the max-delay constraint; this is an NP-hard problem. Our main contributions are first, proposing the Super State Monte-Carlo Tree Search (SS-MCTS) as a version of regular MCTS modified for large-scale probabilistic environments with less computational complexity, and second, addressing the scheduling problem with maximum delay constraint on flows. Our numerical results demonstrate that the proposed approach significantly improves the packet delivery rate while meeting the maximum delay constraint in large-scale scenarios compared to the state-of-the-art technologies.