Due to the growing emphasis on privacy and data governance in machine learning, federated unlearning, an emerging concept in the domain of federated learning, stems from the growing need to address the dynamic nature of data and the evolving requirements related to privacy, compliance, and data management. However, there are some security risks during the unlearning process, including the potential for adversarial manipulation of model integrity, privacy breaches, and performance degradation in a federated learning framework. Although existing research has proposed various defenses to mitigate these risks, significant vulnerabilities remain that can be exploited to undermine the integrity and effectiveness of the unlearning process. Current attack methods are limited by their detectability during training, lack of persistence, and reliance on test-time triggers, which reduces their overall effectiveness. In this paper, we introduce camouflaged poisoning attacks, a novel attack paradigm relevant to federated unlearning. In this approach, some adversary clients initially infuse a small number of meticulously designed points into the dataset, ensuring that the model's predictions are barely influenced. The adversary then makes a request to the exclusion of some of these malicious clients. At this juncture, the attack is activated, leading to a detrimental impact on the model's predictions. The outcomes reveal a substantial potential for these strategies to compromise the effectiveness of models in unlearning scenarios. The essence of this attack involves the creation of deceptive clients that conceal the influence of a contaminated dataset during the federated unlearning process.

With growing attention to data privacy in federated learning, federated unlearning has become an important solution to meet increasing demands for privacy compliance. However, unlearning may bring in new security concerns, such as dangers of adversarial manipulation, where the adversary may launch malicious updates or inputs to hurt the model performance or prediction, privacy-attacks, as the sensitive data can be possibly deduced from the process of unlearning, and performance degradation, because the unlearning process may break the consistency or performance of the model. In this paper, to address such issues and acquire a good and adaptive unlearning policy without causing much negative effect to the federated system, we present a reinforcement learning based method to facilitate the data unlearning method in federated learning. Our approach iteratively disposes of clients through partial unlearning, complete unlearning, or no unlearning using a DQN combined with clients’ properties like contribution, privacy cost, and computational overhead. We show that by utilizing the reinforcement learning technique, the performance decay can be defended effectively, and adversarial behaviors are indeed a common concern for the federated unlearning scenario. Our analysis can inform the development of federated unlearning frameworks that defend against performance and security threats.

Federated learning (FL) trains a global model by aggregating local updates from multiple clients under a server's guidance. Despite its potential, FL is vulnerable to poisoning attacks where malicious clients intentionally corrupt their updates, compromising the global model's accuracy. Current defense strategies aim to tolerate or remove such corrupt updates, but they are not fully effective to prevent malicious clients from sending poisonous updates to the server, leaving the global model at risk. We propose a novel approach based on the evolution of cooperation, which promotes system-wide collaboration. Our defense method allows the server to selectively engage clients in the training process, encouraging them to provide clean updates or exclude those persistently malicious. We also introduce an attack framework where clients initially send clean updates to gain trust before sending malicious ones later. This model, designed to simulate advanced threats, can adapt to various attack types to increase its impact. Our experimental results show that this defense significantly improves resilience against such attacks, effectively safeguarding the global model even under complex threat scenarios.

Machine learning models are susceptible to a range of adversarial activities. These attacks are designed to either infer private information from the target model or deceive it. For instance, an attacker may attempt to discern if a given data example is from the model’s training set (membership inference attacks) or create adversarial examples to mislead the model to make incorrect predictions (adversarial example attacks). Numerous defense methods have been proposed to counter these attacks. However, these methods typically share two common limitations. Firstly, most are not designed to address label-only attacks, which is a newly emerged kind of attacks that rely solely on the hard labels predicted by the target model. Secondly, they are often developed to mitigate specific attacks rather than universally various attacks. To address these limitations, this paper proposes a novel defense method that focuses on the most challenging attacks, i.e., label-only attacks, and can handle various types of label-only attacks. The key idea is to strategically modify the target model’s predicted labels using a meta-reinforcement learning technique. This ensures that attackers receive incorrect labels while benign users continue to receive correct labels. Notably, the defender, i.e., the owner of the target model, can make effective decisions without knowledge of the attacker’s behavior. The experimental results demonstrate that our proposed method is an effective defense against a range of attacks, including label-only model stealing, label-only membership inference, label-only model inversion, and label-only adversarial example attacks.

As deep learning models mature, one of the most prescient questions we face is: what is the ideal tradeoff between accuracy, fairness, and privacy (AFP)? Unfortunately, both the privacy and the fairness of a model come at the cost of its accuracy. Hence, an efficient and effective means of fine-tuning the balance between this trinity of needs is critical. Motivated by some curious observations in privacy-accuracy tradeoffs with differentially private stochastic gradient descent (DP-SGD), where fair models sometimes result, we conjecture that fairness might be better managed as an indirect byproduct of this process. Hence, we conduct a series of analyses, both theoretical and empirical, on the impacts of implementing DP-SGD in deep neural network models through gradient clipping and noise addition. The results show that, in deep learning, the number of training epochs is central to striking a balance between AFP because DP-SGD makes the training less stable, providing the possibility of model updates at a low discrimination level without much loss in accuracy. Based on this observation, we designed two different early stopping criteria to help analysts choose the optimal epoch at which to stop training a model so as to achieve their ideal tradeoff. Extensive experiments show that our methods can achieve an ideal balance between AFP.