We study the resilience to Byzantine failures of distributed implementations of Stochastic Gradient Descent (SGD). So far, distributed machine learning frameworks have largely ignored the possibility of failures, especially arbitrary (i.e., Byzantine) ones. Causes of failures include software bugs, network asynchrony, biases in local datasets, as well as attackers trying to compromise the entire system. Assuming a set of $n$ workers, up to $f$ being Byzantine, we ask how resilient can SGD be, without limiting the dimension, nor the size of the parameter space. We first show that no gradient aggregation rule based on a linear combination of the vectors proposed by the workers (i.e, current approaches) tolerates a single Byzantine failure. We then formulate a resilience property of the aggregation rule capturing the basic requirements to guarantee convergence despite $f$ Byzantine workers. We propose \emph{Krum}, an aggregation rule that satisfies our resilience property, which we argue is the first provably Byzantine-resilient algorithm for distributed SGD. We also report on experimental evaluations of Krum.

We report on Krum, the first provably Byzantine-tolerant aggregation rule for distributed Stochastic Gradient Descent (SGD). Krum guarantees the convergence of SGD even in a distributed setting where (asymptotically) up to half of the workers can be malicious adversaries trying to attack the learning system.

This paper presents a simple broadcast algorithm suited to dynamic systems where links can repeatedly appear and disappear. The algorithm is proved correct and a simple improvement is introduced, that reduces the number and the size of control messages. As it extends in a simple way a classical network traversal algorithm to the dynamic context, the proposed algorithm has also pedagogical flavor.

A notion of a universal construction suited to distributed computing has been introduced by M. Herlihy in his celebrated paper "Wait-free synchronization" (ACM TOPLAS, 1991). A universal construction is an algorithm that can be used to wait-free implement any object defined by a sequential specification. Herlihy's paper shows that the basic system model, which supports only atomic read/write registers, has to be enriched with consensus objects to allow the design of universal constructions.

A message adversary is a daemon that suppresses messages in round-based message-passing synchronous systems in which no process crashes. A property imposed on a message adversary defines a subset of messages that cannot be eliminated by the adversary. It has recently been shown that when a message adversary is constrained by a property denoted TOUR (for tournament), the corresponding synchronous system and the asynchronous crash-prone read/write system have the same computability power for task solvability.

The ability to identify people that share one’s own interests is one of the most interesting promises of the Web 2.0 driving user-centric applications such as recommendation systems or collaborative marketplaces. To be truly useful, however, information about other users also needs to be associated with some notion of trust. Consider a user wishing to sell a concert ticket. Not only must she find someone who is interested in the concert, but she must also make sure she can trust this person to pay for it. This paper addresses the need for trust in user-centric applications by proposing two novel distributed protocols that combine interest-based connections between users with explicit links obtained from social networks à-la Facebook. Both protocols build trusted multi-hop paths between users in an explicit social network supporting the creation of semantic overlays backed up by social trust. The first protocol, TAPS2, extends our previous work on TAPS (Trust-Aware Peer Sampling), by improving the ability to locate trusted nodes. Yet, it remains vulnerable to attackers wishing to learn about trust values between arbitrary pairs of users. The second protocol, PTAPS (Private TAPS), improves TAPS2 with provable privacy guarantees by preventing users from revealing their friendship links to users that are more than two hops away in the social network. In addition to proving this privacy property, we evaluate the performance of our protocols through event-based simulations, showing significant improvements over the state of the art.

This paper is on the design of a consensus object in the context of asynchronous shared memory systems where any number of process can suffer a crash failure. These systems are becoming more and more important with the advent of multicore architectures. To circumvent the impossibility of implementing a consensus object in such a context, the paper considers that the base read/write system model is enriched with an eventual leader failure detector (traditionally denoted Ω). This failure detector can easily be used to ensure that all the invocations of the consensus object issued by processes that do not crash eventually terminate(wait-freedom termination property). Hence, when one has to implement a consensus object in such an enriched system model, the main issue consists in designing an object (from base atomic read/write registers) on which the implementation can rely to ensure that no two different values can be decided from the consensus object. This paper presents such an object, called closing set. The main feature of this object is that it takes advantage of the system asynchrony by reducing the number of values that can be deposited: only concurrent deposits of values in an empty set are successful. The paper presents then a simple consensus algorithm based on closing sets. This algorithm is round-based and uses a closing set per round.

This paper continues our quest for the weakest failure detector which allows the k-set agreement problem to be solved in asynchronous message-passing systems prone to process failures. It has two main contributions which will be instrumental to complete this quest. The first contribution is a new failure detector (denoted PiSigma(x, y)) that has several noteworthy properties. (a) It is stronger than Sigma(k) which has been shown to be necessary. (b) It is equivalent to the pair (Sigma, Omega) when x=y=1 (optimal to solve consensus). (c) It is equivalent to the pair (Sigma(n-1), Omega(n-1)) when x=y=n-1 (optimal for (n-1)-set agreement). (d) It is strictly weaker than the pair (Sigma(x), anti-Omega(y)) (which has been investigated in previous works). (e) It is operational: the paper presents a PiSigma(x, y)-based algorithm that solves k-set agreement for k greater or equal to xy (intuitively, x refers to the maximum number of isolated groups of processes and y to the number of leaders in each of these groups). The second contribution of the paper is a proof that, for k strictly between 1 and n-1, the eventual leaders failure detector Omega(k) (which eventually provides each process with the same set of k process identities, this set including at least one correct process) is not necessary to solve k-set agreement problem.

This short paper shows how to capture failure detectors so that the base asynchronous read/wite model and the distributed iterated model have the same computational power when both are enriched with the same failure detector. To that end it introduces the notion of a "strongly correct" process and presents simulations that prove the computational equivalence when both models are enriched with the same failure detector. Interestingly, these simulations, which work for a large family of failure detector classes, can be easily extended to the case where the wait-freedom requirement is replaced by the notion of t-resilience. A noteworthy and first class feature of the proposed approach lies in its simplicity.