With the growing popularity of large-scale deep neural networks, efficient communication scheduling has become crucial in distributed deep learning systems to reduce overall training time. In multi-job distributed training scenarios, current communication scheduling methods do not effectively utilize the periodic communication patterns of deep learning training (DLT) jobs to reduce the potential link contention. When multiple tenants run concurrent jobs and compete for network resources, training performance can degrade due to increased network contention. In this paper, we focus on exploring the potential of leveraging periodic communication patterns in scheduling DLT jobs. We analyze the performance of static shift-based scheduling strategies based on the least common multiple (LCM) alignment in handling multi-job communication conflicts. Through theoretical analysis and validation via real-world experiments, we expose fundamental limitations of shift-based scheduling strategies, which fail to improve training throughput in about 73 % of multi-job scenarios. Our research work provides guidance for future research on understanding traffic patterns of DLT jobs and lays the groundwork for communication scheduling optimization in multi-tenant clusters.

PolyTOIL is a new statically typed polymorphic object-oriented programming language that is provably typesafe. By separating the definitions of subtyping and inheritance, providing a name for the type of self, and carefully defining the type-checking rules, we have obtained a language that is very expressive while supporting modular type-checking of classes. Thematchingrelation on types, which is related to F-bounded quantification, is used both in stating type-checking rules and expressing the bounds on type parameters for polymorphism. The design of PolyTOIL is based on a careful formal definition of type-checking rules and semantics. A proof of type safety is obtained with the aid of a subject reduction theorem.

We establish sufficient and necessary conditions for a natural sink to be a colimit in DCPO. Based on these conditions, we show how to construct a colimit for any functor F from a small category Ω into the category DCPO. This demonstrates that the category DCPO is cocomplete. We also investigate under what conditions the colimit object is algebraic.