Distributed training is the most common way to scale out and accelerate Deep Neural Network (DNN) training. Distributed DNN training requires synchronization of gradient aggregation among all workers through collective communication operations before proceeding to the next training round. Any long-tail delay would be an obstacle to model training performance and convergence. This paper proposes a load balancing scheme SOLB for distributed DNN training acceleration, which utilizes a synchronization-objective load balancing to ensure packets with the same gradient indices from the same training job arrive at the receiver back to back, enabling optimal synchronous communication. Furthermore, leveraging the independence of parameter updates, we design an order-agnostic transmission protocol to avoid the overhead of packet reordering without affecting the training accuracy. Through both testbed and simulation experiments, our scheme significantly reduces gradient aggregation time by up to 82.5% and accelerates overall model training by up to 84.4% compared to the state-of-the-art load balancing schemes.

Most existing flow scheduling schemes aim to minimize the flow completion time in data center network. However, these schemes either lack fine-grained flow differentiation, sacrificing performance for deployability or require precise flow information and significant hardware modifications for nearoptimal transmission. Thus, we present Sadra, a novel flow scheduling solution leveraging multiple priority queues available in existing commodity switches to minimize FCT by assigning different initial priorities and demotion thresholds to flows with various sizes. Our intuition is that the average queuing length of small flows increases as the number of large flows grows, yet reducing queuing length improves latency of small flows with negligible impact on throughput of large flows. Sadra consists of two key parts: i) A novel scheduling scheme that utilizes imprecise predictions to assign initial priorities to flows with different sizes, enabling differentiation from the outset of transmission; ii) A size-aware demotion threshold strategy that assigns different demotion thresholds to further differentiate flows with the same initial priority, thereby reducing the blocking of small flows by large ones. We have implemented a Sadra prototype and evaluated Sadra through both testbed experiments and NS3 simulations. Simulation and testbed evaluations show that Sadra significantly reduces the average and tail FCT of small flows by up to 61% and 89% compared with the state-of-the-art schemes, respectively.

In this paper, a novel cross-layer optimization scheme for video streaming in wireless relay networks is proposed. It takes into account of delay constraints at the application (APP) layer, relay node selection at the MAC layer and dynamic fading of wireless channels at the physical (PHY) layer in an integrated mode to improve the quality of video transmission. At the PHY layer, a finite state Markov channels (FSMC) model is used to estimate the channel states and a modified RTS/CTS scheme is proposed to facilitate the path selection at the MAC layer. The impacts of delay constraints and other factors at the APP layer are investigated. At the MAC layer, the source node can choose the proper path to transmit video data while considering video delay requirements and wireless channel states simultaneously. A typical three-node uploading scenario in video surveillance applications is considered as an example for performance evaluation. The experimental results show that the proposed scheme could improve video transmission quality and achieve a near-optimal performance.

In this paper, we improve the soft-threshold by using Markov random field (MRF) theory to preserve the edges of images. According to MRF theory, the similarity problem of the edge structure can be transformed into an energy function minimization problem. Through computing all the energies of the edge structures in clique c, we can find the edge structure with the minimum energy, and regard it as the edge structure of this place. In this way, we can gain the edge information of an image. Here we detect the edge of image in wavelet domain. In our method we preserve the wavelet coefficients which belong to edge set; for the other wavelet coefficients, the soft threshold method is applied. Then, we reconstruct the treated wavelet coefficients by using inverse wavelet transform. The experiment result shows that our method is better than the hard threshold and soft threshold method. It will not only smooth away noise, but also preserve the edge of image.