Federated learning (FL) is an emerging paradigm through which decentralized devices can collaboratively train a common model. However, a serious concern is the leakage of privacy from exchanged gradient information between clients and the parameter server (PS) in FL. To protect gradient information, clients can adopt differential privacy (DP) to add additional noises and distort original gradients before they are uploaded to the PS. Nevertheless, the model accuracy will be significantly impaired by DP noises, making DP impracticable in real systems. In this work, we propose a novel noise information secretly sharing (NISS) algorithm to alleviate the disturbance of DP noises by sharing negated noises among clients. We theoretically prove that: 1) if clients are trustworthy, DP noises can be perfectly offset on the PS and 2) clients can easily distort negated DP noises to protect themselves in case that other clients are not totally trustworthy, though the cost lowers model accuracy. NISS is particularly applicable for FL across multiple Internet of Things (IoT) systems, in which all IoT devices need to collaboratively train a model. To verify the effectiveness and the superiority of the NISS algorithm, we conduct experiments with the MNIST and CIFAR-10 data sets. The experimental results verify our analysis and demonstrate that NISS can improve model accuracy by 19% on average and obtain better privacy protection if clients are trustworthy.

With the growth of participating clients, the centralized parameter server (PS) will seriously limit the scale and efficiency of Federated Learning (FL). A straightforward approach to scale up the FL system is to construct a Parallel FL (PFL) system with multiple PSes. However, it is unclear whether PFL can really achieve a faster convergence rate or not. Even if the answer is yes, it is non-trivial to design a highly efficient parameter average algorithm for a PFL system. In this paper, we propose a completely parallelizable FL algorithm called P-FedAvg under the PFL architecture. P-FedAvg extends the well-known FedAvg algorithm by allowing multiple PSes to cooperate and train a learning model together. In P-FedAvg, each PS is only responsible for a fraction of total clients, but PSes can mix model parameters in a dedicatedly designed way so that the FL model can well converge. Different from heuristic-based algorithms, P-FedAvg is with theoretical guarantees. To be rigorous, we conduct theoretical analysis on the convergence rate of P-FedAvg, and derive the optimal weights for each PS to mix parameters with its neighbors. We also examine how the overlay topology formed by PSes affects the convergence rate and robustness of a PFL system. Lastly, we perform extensive experiments with real datasets to verify our analysis and demonstrate that P-FedAvg can significantly improve convergence rates than traditional FedAvg and other competitive baselines. We believe that our work can help to lay a theoretical foundation for building more efficient PFL systems.

Federated collaborative filtering (Fed-CF) is a variant of federated learning (FL) models, which can protect user privacy in recommender systems. In Fed-CF, the recommendation model is collectively trained across multiple decentralized clients by exchanging gradients only. However, the decentralized nature of Fed-CF makes it vulnerable to shilling attacks, which can be realized by inserting fake ratings of target items to distort recommendation results. Unfortunately, previous detection algorithms cannot work well in the FL framework, as all original data samples are not disclosed at all. In this paper, we are the first to systematically study the problem of shilling attacks in the context of federated learning, and propose an effective detection method called Federated Shilling Attack Detector (FSAD) to detect shilling attackers in Fed-CF. We first show the feasibility of shilling attacks in Fed-CF. Next, we dedicatedly design four novel features based on exchanged gradients among clients. By incorporating these gradient-based features, we train a semi-supervised Bayes classifier to identify shilling attackers effectively. Finally, we conduct extensive experiments based on real-world datasets to evaluate the performance of our proposed FSAD method. The experimental results show that FSAD can detect shilling attackers in Fed-CF with high accuracy, with the F1value as high as 0.90 on the Netflix dataset, which approaches the performance of the optimal detector that utilizes complete private user information for detection.