Incentive mechanisms are essential for boosting client engagement in differential private federated learning (DP-FL). However, existing Stakelberg games-based incentive mechanisms typically assume that client decisions are one-dimension and that data is independent and identically distributed (IID) across clients. In reality, data distributions are often non-IID and clients have two-dimensional resources decisions, including data quantity and privacy. Therefore, in this paper, we present a novel two-dimensional Stackelberg game-based incentive mechanism for DP-FL with non-IID data, aiming to maximize the total utility of clients and server by seeking a balance between the clients' two-dimensional decisions and the server's payment. Specifically, we first formulate the utility functions of both server and clients under two-dimensional decisions and then model the interactions between server and clients as a single-leader-multiple-followers Stackelberg game. To derive the optimal decisions that maximize their utilities, we theoretically prove the existence of a Stackelberg equilibrium between server and clients. Due to the difficulty to directly calculate the Stackelberg equilibrium, we propose a bi-level multi-agent reinforment learning algorithm to learn the optimal decisions for both server and clients by trial and error. Extensive simulation results demonstrate that our proposed method outperforms the baselines in terms of total utility.

Language Models (LMs) are prone to ''memorizing'' training data, including substantial sensitive user information. To mitigate privacy risks and safeguard the right to be forgotten, machine unlearning has emerged as a promising approach for enabling LMs to efficiently ''forget'' specific texts. However, despite the good intentions, is textual unlearning really as effective and reliable as expected? To address the concern, we first propose Unlearning Likelihood Ratio Attack+ (U-LiRA+), a rigorous textual unlearning auditing method, and find that unlearned texts can still be detected with very high confidence after unlearning. Further, we conduct an in-depth investigation on the privacy risks of textual unlearning mechanisms in deployment and present the Textual Unlearning Leakage Attack (TULA), along with its variants in both black- and white-box scenarios. We show that textual unlearning mechanisms could instead reveal more about the unlearned texts, exposing them to significant membership inference and data reconstruction risks. Our findings highlight that existing textual unlearning actually gives a false sense of unlearning, underscoring the need for more robust and secure unlearning mechanisms.

Federated learning (FL) is susceptible to poisoning attacks. To defend against such threats, robust aggregation rules (AGRs) are typically deployed on the server to identify or filter clients’ potentially malicious submissions based on statistical similarity. Recently, knowledge distillation (KD) has been widely used in FL to facilitate collaborative learning among clients that have heterogeneous model architectures by aggregating and distilling architecture-independent model outputs (i.e., logits). However, the KD process introduces a novel poisoning attack surface, where adversaries can manipulate local model output logits to ruin the global model performance. To fully reveal and explore such a new security vulnerability and effectively poison the global model in the existence of robust AGRs, in this paper, we propose the first untargeted poisoning attack scheme to KD-based FL under robust AGRs, named ManipulatingKD. It manipulates compromised clients to send well-designed malicious logits during the KD process. To ensure attack effectiveness and stealthiness, ManipulatingKD models attacks as constrained optimization problems. This allows for crafting satisfactory malicious logits that are statistically similar to benign logits but can generate poisoned aggregated logits to provide deviated supervision and mislead the global model. Extensive experiments demonstrate the effectiveness of ManipulatingKD under both non-robust and robust AGRs. Particularly, under robust AGRs, the global model accuracy degradation caused by our attacks can exceed 2× that of state-of-the-art attacks.

To efficiently execute tasks, computation resource requesters (CRRs) with limited resources can offload their tasks to nearby computation resource providers (CRPs) with spare computing capacity. These CRPs require appropriate incentives to compensate for their incurred costs when helping process the offloaded tasks. Although several mechanisms have been designed to incentivize CRPs, none of them have investigated the incentive mechanism considering price-setting and price-taking CRPs simultaneously. In this work, we propose an incentive framework for task offloading in the edge computing marketplace that includes both price-setting and price-taking CRPs. We model the CRR's interactions with both types of CRPs as a three-stage Stackelberg game to maximize the profit for both the CRR and CRPs. We prove the existence of a unique subgame perfect equilibrium (SPE) of the formulated game and further develop iterative algorithms for the CRR and price-setting CRPs to achieve the equilibrium. Through the designed algorithms, each CRP does not require complete information about the CRR and other CRPs. Extensive simulations demonstrate that offloading tasks to both price-setting and price-taking CRPs achieves higher profits for the CRR and price-setting CRPs compared to offloading tasks solely to price-setting CRPs. Additionally, the obtained SPE can achieve near-optimal social welfare.

Due to the collaborative machine learning nature of Federated Learning (FL), it enables the training of machine learning models on large-scale distributed datasets in edge computing environments. Nevertheless, the application of FL in edge computing still faces three crucial challenges: resource constraint, privacy leakage, and Byzantine failures. Unfortunately, current approaches lack the ability to effectively balance these three challenges. In this paper, we propose FedEdge, a cost-efficient and secure FL for edge computing. FedEdge contains two main mechanisms: adaptive compression perturbation and dynamic update filtering. The adaptive compression perturbation mechanism reduces the communication overhead, provides different levels of privacy protection for edge nodes, and prevents Byzantine attacks. The dynamic update filtering mechanism is used to further filter Byzantine attacks and limit the impact of adaptive compression perturbation on the global model performance. The experimental results on the MNIST, CIFAR-10, CIFAR-100, and CelebA datasets demonstrate the effectiveness of FedEdge against free-riders, label-flipping, and sign-flipping attacks. Theoretical analysis also demonstrate that FedEdge can still converge even when the majority of edge nodes are malicious.

Federated Learning (FL) exhibits privacy vulnerabilities under gradient inversion attacks (GIAs), which can extract private information from individual gradients. To enhance privacy, FL incorporates Secure Aggregation (SA) to prevent the server from obtaining individual gradients, thus effectively resisting GIAs. In this paper, we propose a stealthy label inference attack to bypass SA and recover individual clients’ private labels. Specifically, we conduct a theoretical analysis of label inference from the aggregated gradients that are exclusively obtained after implementing SA. The analysis results reveal that the inputs (embeddings) and outputs (logits) of the final fully connected layer (FCL) contribute to gradient disaggregation and label restoration. To preset the embeddings and logits of FCL, we craft a fishing model by solely modifying the parameters of a single batch normalization (BN) layer in the original model. Distributing client-specific fishing models, the server can derive the individual gradients regarding the bias of FCL by resolving a linear system with expected embeddings and the aggregated gradients as coefficients. Then the labels of each client can be precisely computed based on preset logits and gradients of FCL’s bias. Extensive experiments show that our attack achieves large-scale label recovery with 100% accuracy on various datasets and model architectures.

Federated learning (FL) is widely used in edge environments as a privacy-preserving collaborative learning paradigm. However, edge devices often have heterogeneous computation capabilities and data distributions, hampering the efficiency of co-training. Existing works develop staleness-aware semi-asynchronous FL that reduces the contribution of slow devices to the global model to mitigate their negative impacts. But this makes data on slow devices unable to be fully leveraged in global model updating, exacerbating the effects of data heterogeneity. In this paper, to cope with both system and data heterogeneity, we propose a clustering and two-stage aggregation-based Efficient Asynchronous Federated Learning (EAFL) framework, which can achieve better learning performance with higher efficiency in heterogeneous edge environments. In EAFL, we first propose a gradient similarity-based dynamic clustering mechanism to cluster devices with similar system and data characteristics together dynamically during the training process. Then, we develop a novel two-stage aggregation strategy consisting of staleness-aware semi-asynchronous intra-cluster aggregation and data size-aware synchronous inter-cluster aggregation to efficiently and comprehensively aggregate training updates across heterogeneous clusters. With that, the negative impacts of slow devices and Non-IID data can be simultaneously alleviated, thus achieving efficient collaborative learning. Extensive experiments demonstrate that EAFL is superior to state-of-the-art methods.

Deep unsupervised domain adaptation (UDA) has significantly boosted the performance of deep models on different domains by transferring knowledge from a source domain to a target domain. However, its robustness against adversarial attacks has not been explored due to the challenges of highly non-convex deep models and different data distribution. In this paper, we give the first attempt to analyze the vulnerability of deep UDA and propose a label-free poisoning attack (LFPA), which injects poisoning data into the training data to mislead adaptation between the two domains without ground truth in target domain. Specifically, we design an unsupervised adversarial loss as the attack goal, in which the pseudo-labels are used to approximate the ground-truth. Since retraining the model will gradually degrade the attack performance, we also add a regularization term to the unsupervised loss, which eliminates negative interactions between the training goal and the attack goal. To accelerate the craft of poisons, we select influential samples as the initial poisons and propose a fast reverse-mode optimization method which updates poisons according to the approximate truncated gradients. Experimental results on multiple state-of-the-art deep UDA methods demonstrate the effectiveness of the proposed LFPA and the high sensitivity of UDA to poisoning attacks.

Federated Learning (FL) is susceptible to the gradient leakage attack (GLA), which can recover local private training data from the shared gradients or model updates. To ensure privacy, differential privacy is applied in FL by clipping and adding noise to local gradients (i.e., Local Differential Privacy (LDP)) or the global model update (i.e., Central Differential Privacy (CDP)). However, the effectiveness of DP in defending GLAs needs to be thoroughly investigated since some works briefly verify that DP can guard FL against GLAs while others question its defense capability. In this paper, we empirically evaluate CDP and LDP on the resistance of GLAs, and pay close attention to the trade-offs between privacy and utility in FL. Our findings reveal that: 1) existing GLAs can be defended by CDP using a per-layer clipping strategy and LDP with a reasonable privacy guarantee and 2) both CDP and LDP ensure the trade-off between privacy and utility in training shallow model, but cannot guarantee this trade-off in deeper model training (e.g., ResNets). Triggered by the crucial role of clipping operation for DP, we propose an improved attack that incorporates the clipping operation into existing GLAs without requiring additional information. The experimental results show our attack can destruct the protection of CDP and weaken the effectiveness of LDP. Overall, our work validates the effectiveness as well as reveals the vulnerability of DP under GLAs. We hope this work can provide guidance on utilizing DP for defending against GLA in FL and inspire the design of future privacy-preserving FL.

In mobile edge computing (MEC), users can offload tasks to nearby MEC servers to reduce computation cost. Considering that the size of offloaded tasks could disclose user location information, several location privacy-preserving task offloading mechanisms have been proposed under the single-server scenario. However, to the best of our knowledge, none of them could provide a strict privacy protection guarantee or be applicable to the multi-server scenario where the user's location can be inferred more accurately if servers collude with each other. In this paper, we propose a novel location privacy-aware task offloading framework (LPA-Offload) for both single-server and multi-server scenarios, which provides strict and provable location privacy protection while achieving efficient task offloading. Specifically, we propose a location perturbation mechanism that allows each user to perturb its real location within a rational perturbation region and provides a differential privacy guarantee. To make a satisfactory offloading strategy, we propose a perturbation region determination mechanism and an offloading strategy generation mechanism that adaptively select a proper perturbation region according to the customized privacy factor, and then generate an optimal offloading strategy based on the perturbed location within the decided region. The determination of the perturbation region could achieve personalized privacy requirements while reducing computation cost. LPA-Offload is proved to satisfy$(\epsilon,\delta)$-differential privacy, and the experiments demonstrate the effectiveness of our framework.

Federated learning (FL) requires frequent uploading and updating of model parameters, which is naturally vulnerable to gradient leakage attacks (GLAs) that reconstruct private training data through gradients. Although some works incorporate differential privacy (DP) into FL to mitigate such privacy issues, their performance is not satisfactory since they did not notice that GLA incurs heterogeneous risks of privacy leakage (RoPL) with respect to gradients from different communication rounds and clients. In this paper, we propose an Adaptive Privacy-Preserving Federated Learning (Adp-PPFL) framework to achieve satisfactory privacy protection against GLA, while ensuring good performance in terms of model accuracy and convergence speed. Specifically, a leakage risk-aware privacy decomposition mechanism is proposed to provide adaptive privacy protection to different communication rounds and clients by dynamically allocating the privacy budget according to the quantified RoPL. In particular, we exploratively design a round-level and a client-level RoPL quantification method to measure the possible risks of GLA breaking privacy from gradients in different communication rounds and clients respectively, which only employ the limited information in general FL settings. Furthermore, to improve the FL model training performance (i.e., convergence speed and global model accuracy), we propose an adaptive privacy-preserving local training mechanism that dynamically clips the gradients and decays the noises added to the clipped gradients during the local training process. Extensive experiments show that our framework outperforms the existing differentially private FL schemes on model accuracy, convergence, and attack resistance.

Face recognition service providers protect face privacy by extracting compact and discriminative facial features (representations) from images, and storing the facial features for real-time recognition. However, such features can still be exploited to recover the appearance of the original face by building a reconstruction network. Although sev-eral privacy-preserving methods have been proposed, the enhancement offace privacy protection is at the expense of accuracy degradation. In this paper, we propose an adver-sarial features-based face privacy protection (AdvFace) approach to generate privacy-preserving adversarial features, which can disrupt the mapping from adversarial features to facial images to defend against reconstruction attacks. To this end, we design a shadow model which simulates the attackers' behavior to capture the mapping function from facial features to images and generate adversarial la-tent noise to disrupt the mapping. The adversarial features rather than the original features are stored in the server's database to prevent leaked features from exposing facial information. Moreover, the AdvFace requires no changes to the face recognition network and can be implemented as a privacy-enhancing plugin in deployed face recognition systems. Extensive experimental results demonstrate that Adv Face outperforms the state-of-the-art face privacy-preserving methods in defending against reconstruction at-tacks while maintaining face recognition accuracy.

Reverse auction-based incentive mechanisms have been commonly proposed to stimulate mobile users to participate in crowdsensing, where users submit bids to the platform to compete for interested tasks. Recent works pointed out that bid is a private information which can reveal sensitive information of users (e.g., location privacy), and proposed bidding-preserving mechanisms with differential privacy against inference attack. However, all these mechanisms rely on a trusted platform, and would fail in bid protection completely when the platform is untrusted. In this paper, we design novel privacy-preserving incentive mechanisms to protect users’ true bid information against the honest-but-curious platform while minimizing the social cost of winner selection. To this end, instead of uploading the true bid to the platform, a differentially private bid obfuscation function is designed with the exponential mechanism, which helps each user to obfuscate bids locally and submit obfuscated bids to the platform. Two solutions are proposed for the platform to solve the winner selection problem with the obfuscated information, which is proved to be NP-hard. Moreover, we further propose a novel task-bid pair protection truthful incentive mechanism to further prevent privacy leakage from the set of interested tasks, where each user encrypts his interested tasks via homomorphic encryption locally, and an encrypted task clustering method is proposed to group users with the same interested tasks into the same cluster for winner selection with users’ encrypted task-bid pairs. Both of theoretical analysis and extensive experiments demonstrate the effectiveness of proposed mechanisms against the untrusted platform.

Worker recruitment in mobile crowdsensing systems aims to recruit the most suitable users to perform tasks with high quality and in real-time. Many worker recruitment or task matching mechanisms have been proposed, especially for crowdsourcing platforms, where content information of tasks from the implicit feedback of workers' attendance is extensively exploited to help workers find preferred tasks efficiently. Different from traditional crowdsourcing systems, tasks in mobile crowdsensing systems are usually time-sensitive and location-dependent which also play a crucial role in worker recruitment. However, these context information have not been effectively explored for user recruitment in mobile crowdsensing systems. In this paper, we propose a novel personalized task-oriented worker recruitment mechanism for mobile crowdsensing systems based on a careful characterization of workers' preference. In particular, we fully exploit the content information (e.g., task category, task description) together with the context information (e.g., task time, task location) from the implicit feedback of workers' attendance to accurately model workers' preference on tasks. Moreover, we regard the task-worker fitness prediction as a binary classification problem and utilize the Logit model to integrate the heterogeneous factors into a single framework to predict the matching probability of each task-worker pair. Finally, the workers with the highest matching probability are recruited proactively for each new task. Extensive experiments on real-world datasets demonstrate that the proposed mechanism achieves better performance than the benchmarks.

Incentive mechanisms have been commonly proposed to encourage people to participate in mobile crowdsensing (MCS). However, most of them set unchangeable rewards for sensing tasks, while the inherent inequality and on-demand feature of sensing tasks have been long ignored, especially for location-dependent sensing tasks (LDSTs). In this paper, we focus on location-dependent MCS systems and propose a demand-driven dynamic incentive mechanism that dynamically changes the rewards of sensing tasks at each sensing round in an on-demand way to balance their popularity. A demand indicator is introduced to characterize the demand of each sensing task by considering its deadline, completing progress, and number of potential participants. At each sensing round, we use the Analytic Hierarchy Process (AHP) to calculate the relative demands of all sensing tasks and then determine their rewards accordingly. Moreover, we consider two task selection problem with participatory users and opportunistic users, respectively, and prove that both of them are NP-hard. We propose an optimal dynamic programming based solution for participatory scenario and an optimal backtracking based solution for opportunistic scenario to help each user select tasks while maximizing its profit. Extensive experiments show that the demand-driven dynamic incentive mechanism outperforms existing incentive mechanisms.

Reverse auction-based incentive mechanisms have been commonly proposed to stimulate mobile users to participate in crowdsensing, where users submit bids to the platform to compete for tasks. Recent works pointed out that bid is a private information which can reveal sensitive information of users (e.g., location privacy), and proposed bid-preserving mechanisms with differential privacy against inference attack. However, all these mechanisms rely on a trusted platform, and would fail in bid protection completely when the platform is untrusted (e.g., honest-but-curious). In this paper, we focus on the bid protection problem in mobile crowdsensing with an untrusted platform, and propose a novel privacy-preserving incentive mechanism to protect users' true bids against the honest-but-curious platform while minimizing the social cost of winner selection. To this end, instead of uploading the true bid to the platform, a differentially private bid obfuscation function is designed with the exponential mechanism, which helps each user to obfuscate bids locally and submit obfuscated task-bid pairs to the platform. The winner selection problem with the obfuscated task-bid pairs is formulated as an integer linear programming problem and proved to be NP-hard. We consider the optimization problem at two different scenarios, and propose a solution based on Hungarian method for single measurement and a greedy solution for multiple measurements, respectively. The proposed incentive mechanism is proved to satisfy ε-differential privacy, individual rationality and γ-truthfulness. The extensive experiments on a real-world data set demonstrate the effectiveness of the proposed mechanism against the untrusted platform.

Location information of workers are usually required for optimal task allocation in mobile crowdsensing, which however raises severe concerns of location privacy leakage. Although many approaches have been proposed to protect the locations of users, the location protection for task allocation in mobile crowdsensing has not been well explored. In addition, to the best of our knowledge, none of existing privacy-preserving task allocation mechanisms can provide personalized location protection considering different protection demands of workers. In this paper, we propose a personalized privacy-preserving task allocation framework for mobile crowdsensing that can allocate tasks effectively while providing personalized location privacy protection. The basic idea is that each worker uploads the obfuscated distances and personal privacy level to the server instead of its true locations or distances to tasks. In particular, we propose a Probabilistic Winner Selection Mechanism (PWSM) to minimize the total travel distance with the obfuscated information from workers, by allocating each task to the worker who has the largest probability of being closest to it. Moreover, we propose a Vickrey Payment Determination Mechanism (VPDM) to determine the appropriate payment to each winner by considering its movement cost and privacy level, which satisfies the truthfulness, profitability, and probabilistic individual rationality. Extensive experiments on the real-world datasets demonstrate the effectiveness of the proposed mechanisms.

With the rich sensing capacity and ubiquitous usage of smartphones, crowdsensing leveraging the power of the crowd of mobile users has become an effective technique to collect data for various sensing applications. Many incentive mechanisms have been proposed to encourage people to participate in crowdsensing. However, most of them set unchangeable rewards for sensing tasks, while the inherent inequality and on-demand feature of sensing tasks have been long ignored, especially for location-dependent sensing tasks. In this paper, we focus on location-dependent crowdsensing systems and propose a demand-based dynamic incentive mechanism that dynamically changes the rewards of sensing tasks at each sensing round in an on-demand way to balance their popularity. A demand indicator is introduced to characterize the demand of each sensing task by considering its deadline, completing progress, and number of potential participants. At each sensing round, we use the Analytic Hierarchy Process to calculate the relative demands of all sensing tasks and then determine their rewards accordingly. Moreover, we prove that the distributed task selection problem with time budget is NP-hard. We propose an optimal dynamic programming based solution and a greedy solution to help each user select tasks while maximizing its profit. Extensive experiments show that the demand-based dynamic incentive mechanism outperforms existing incentive mechanisms.