Synchronous stochastic gradient descent (SSGD) has been widely used in distributed deep learning. However, since the local gradients need to be shared among workers at every iteration, SSGD performance is significantly influenced by network bottlenecks caused by either heterogeneous environment or bandwidth contention. To solve this problem, asynchronous parallel (ASP) strategy allows each worker to update parameters independently without synchronization, while suffering from accuracy loss and convergence inefficiency. In this paper, we propose a novel similarity-based synchronization scheme called SIM, which mitigates the impact of network bottlenecks and ensures convergence efficiency. Specifically, SIM reduces the number of aggregation workers based on the gradient similarity between global and local gradients, therefore shrinking the waiting time for the stragglers. We provide a theoretical analysis of convergence efficiency and conduct large-scale testbed experiments on CIFAR-10 and SQUAD dataset. The experimental results show that SIM reduces the convergence time of four classical deep learning models by up to 40%.

In lossless data center networks (DCN), many existing congestion control schemes are used to address the impact caused by priority-based flow control (PFC), such as congestion spreading, and victim flow problems. However, in some special cases, this problem is not solved. Through observation, we examine the interaction between flow control and congestion control, and realize that the mismatch between hop-by-hop flow control and end-to-end congestion feedback, as well as inaccurate rate regulation, are the root causes of frequent PFC triggering. Therefore, we propose Egress Queue Congestion Information Notification (EQCIN). EQCIN implements threshold-based flow identification to avoid packet buildup due to congestion spreading being considered as the root cause of congestion, while using direct feedback from the congestion side to reduce unnecessary link loss. For different flow identifiers, EQCIN adopts different algorithms to achieve targeted rate control. Experimental results show that EQCIN can reduce the number of PFC PAUSEs tends to zero, compared to TIMELY, DCQCN, DCQCN+TCD and improve the link utilization by 7%-77%, respectively.

Abstract Nonlinear feedback shift register (NFSR) is one of the most important cryptographic primitives in lightweight cryptography. At ASIACRYPT 2010, Knellwolf et al. proposed conditional differential attack to perform a cryptanalysis on NFSR-based cryptosystems. The main idea of conditional differential attack is to restrain the propagation of the difference and obtain a detectable bias of the difference of the output bit. QUARK is a lightweight hash function family which is designed by Aumasson et al. at CHES 2010. Then the extended version of QUARK was published in Journal of Cryptology 2013. In this paper, we propose an improved conditional differential attack on QUARK. One improvement is that we propose a method to select the input difference. We could obtain a set of good input differences by this method. Another improvement is that we propose an automatic condition imposing algorithm to deal with the complicated conditions efficiently and easily. It is shown that with the improved conditional differential attack on QUARK, we can detect the bias of output difference at a higher round of QUARK. Compared to the current literature, we find a distinguisher of U-QUARK/D-QUARK/S-QUARK/C-QUARK up to 157/171/292/460 rounds with increasing 2/5/33/8 rounds respectively. We have performed the attacks on each instance of QUARK on a 3.30 GHz Intel Core i5 CPU, and all these attacks take practical complexities which have been fully verified by our experiments. As far as we know, all of these results have been the best thus far.