In federated learning, the central server aggregates local model updates from the participants in the network to generate a global model. For the purpose of protecting clients' privacy, the server is designed to have no visibility into how these updates are generated. The nature of federated learning makes detecting and defending against malicious model up-dates a challenging task. Unlike existing works that struggle to defend against poisoning attacks from a spatial perspective, the paper considers mitigating the impact of attacks from a spatial-temporal perspective. This paper proposes Fedmvae, a robust federated learning framework. Fedmvae uses multiple variational autoencoder models to detect and exclude malicious model updates from a spatial perspective. Moreover, to handle poisoning attacks with time-varying features, we propose generating a robust global model update according to momentum-based update speculation and historical global updates. Fedmvae is tested with extensive experiments on both IID and non-IID datasets, showing a competitive performance over existing aggregation methods under both Byzantine attacks and backdoor attacks.

The industry has widely deployed federated learning (FL) due to its promise to protect clients’ privacy. However, FL is vulnerable to adversarial attacks when the participants are compromised. The defense against adversarial attacks is a challenging problem in FL. Moreover, existing defense methods optimize the dimensionality reduction and anomaly detection models separately, leading to a disappointing projection space and low detection accuracy. We propose a deep metric learning-based anomaly detection to project the model gradients into a metric space where the malicious gradients are separated from benign ones. Meanwhile, while existing methods require an auxiliary dataset to train the defense model, the auxiliary dataset is usually unavailable to the server in the FL setting. We propose a self-supervised method to distill the data between the training epochs of our defense model. To handle radical changes in malicious model gradients, we utilize a median-based aggregated gradient filter to discard improper aggregated gradients. We show experimentally that our algorithm has a competitive performance over existing methods under Byzantine attacks and backdoor attacks with various triggers.

In federated learning, the central server combines local model updates from the clients in the network to create an aggregated model. To protect clients' privacy, the server is designed to have no visibility into how these updates are generated. The nature of federated learning makes detecting and defending against malicious model updates a challenging task. Unlike existing works that struggle to defend against Byzantine clients, the paper considers defending against targeted model poisoning attack in the federated learning setting. The adversary aims to reduce the model performance on targeted subtasks while maintaining the main task's performance. This paper proposes Fedcvae, a robust and unsupervised federated learning framework where the central server uses conditional variational autoencoder to detect and exclude malicious model updates. Since the reconstruction error of malicious updates is much larger than that of benign ones, it can be used as an anomaly score. We formulate a dynamic threshold of reconstruction error to differentiate malicious updates from normal ones based on this idea. Fedcvae is tested with extensive experiments on IID and non-IID federated benchmarks, showing a competitive performance over existing aggregation methods under Byzantine attack and targeted model poisoning attack.

In asynchronous distributed learning, the parameter server updates the global model as soon as a new gradient is received from any device. The asynchronous systems are designed to address the existence of lagging devices which is inevitable due to device heterogeneity and network unreliability. However, the lack of synchrony incurs additional noise and makes detecting and defending against malicious model gradients a challenging task. Unlike existing works that struggle to design robust methods to tolerate untargeted model poisoning gradients, the paper considers detecting and removing targeted model poisoning gradients from the normal asynchronous training process. This paper proposes Asynvae, a robust distributed asynchronous learning framework where the parameter server uses variational autoencoder to detect and exclude malicious gradients. Since the reconstruction error of malicious updates is much larger than that of benign ones, it can be used as an anomaly score. We formulate a threshold of reconstruction error to differentiate malicious updates from normal ones based on this idea. Asynvae is tested with extensive experiments on distributed learning benchmarks, showing a competitive performance over existing distributed learning methods under untargeted model poisoning attack, targeted model poisoning attack and lagging attack.