Distributed training (DT) has emerged as a solution to address the growing computational resource demands of training large-scale machine learning models. To meet this need, major cloud providers typically build GPU clusters to accommodate DT jobs. Specifically, for an incoming DT job request, cloud providers need to determine in which GPUs place workers (i.e., worker placement). Existing approaches usually place workers on as few idle machines as possible to minimize communication time. However, this scheme will lead to resource fragmentation problem, which degrades the efficiency of the GPU cluster and increases training costs for cloud providers. In this paper, we propose Titan, a novel worker placement scheme that mitigates the influence of resource fragmentation by enhancing the utilization of non-idle machines. Titan formulates a multi-objectives non-linear optimization problem that incorporates the collective communication constraint and proves its NP-hardness. To solve the problem, Titan presents an effective submodular-based greedy algorithm with a tight approximation ratio ($1-\frac{1}{e}$). We evaluate Titan with a large-scale simulation employing real-world job traces and a small-scale testbed consisting of 8 servers with 32 logical GPUs. Experimental results show that Titan can achieve near-optimal training throughput while improving the efficiency of the cluster by 74.9% compared to the state-of-the-art solutions.

Software Defined Networking (SDN) is beneficial to many applications, such as intra-datacenter communication, inter-datacenter transportation, etc., due to its centralized control. However, this centralized control frequently makes the controller a bottleneck, due to the large amount of interactions between the controller and switches. In this paper, we characterize such interactions as control traffic, and propose RouteStitch to minimize such kind of traffic. RouteStitch exploits existing route entries in switches to build new paths. To this end, RouteStitch first builds a graph model to describe existing route entries. Then, on such a model, a novel minimum color-alternation routing problem is defined to minimize control traffic, after which an optimal algorithm is proposed on a fixed routing path. For general paths, an O(log2L)-competitive online algorithm is designed to build new paths in an online manner that preserves fundamental property of switch Ternary Content Addressable Memory (TCAM) capacity and allowed maximum hop length L. Extensive simulation results based on realistic topology show that RouteStitch has good performance in terms of reducing control traffic, by 40%.

Many corporations rely on Internet service provider (ISP) network to provide reliable communication services. However, the current communication networks are vulnerable to disruptive events, such as natural disaster or power outage. Such disastrous events may destroy multiple network facilities in a specific region and result in a long term recovery of ISP networks. The disconnected communication will lead to enormous economic loss even if corporation's infrastructure is not directly destroyed during the disaster. Therefore, corporations need a self-rescue mechanism to actively respond to the emergency instead of simply relying on the ISP. This paper proposes Ambula, an easy-to-deploy platform to realize fast congestion-aware recovery for corporation's communication lifeline. Our platform leverages current widely-deployed public cloud services to build a scalable peer-to-peer overlay routing system. By so doing, the corporation is capable of controlling the packets forwarding path to bypass the affected region and congested routes. To this end, Ambula first carefully selects a small set of virtual machines (VMs) from geographically distributed public clouds, and then apply the self-developed congestion-aware routing protocol to achieve automatic and fast routing recovery. Simulations on both random generated and real network topologies show that the high recovery ratio of 80% can be achieved. The congestion avoidance algorithm can significantly reduce the impact of congestion. Our prototype on Emulab shows it can recover within hundreds of milliseconds. To the best of our knowledge, no effective disaster recovery mechanism currently exists for corporations during emergency. Ambula will facilitate the business continuity management of corporations in present of hazard events.

With the wide deployment of network facilities and the increasing requirement of network reliability, the disruptive event like natural disaster, power outage or malicious attack has become a non-negligible threat to the current communication network. Such disruptive event can simultaneously destroy all devices in a specific geographical area and affect many network based applications for a long time. Hence, it is essential to build disaster-resilient network for future highly survivable communication services. In this paper, we focus on the integrated approach through the technique of software defined networking to mitigate disaster risks while cut down the investment and management costs. Our design consists of a sub-graph based proactive protection approach for fast rerouting at the network nodes and a splicing approach at the controller for effective post-disaster restoration. Such a systematic design is implemented in OpenFlow framework through the Mininet emulator and Nox controller. Numerical results show that our approach can achieve high reliability, fast recovery and low control overhead.