Deep reinforcement learning (DRL) has been implemented across various critical applications, including smart grids, trac management systems, and autonomous vehicles. To safeguard intellectual property and mitigate security vulnerabilities, access to DRL models is typically restricted to a black-box format. is means specic details like the structure of the policy network and optimization processes are not openly available to users. It is crucial to determine if the hyper-parameters can be inferred from observable states and actions within these models, presenting two primary challenges: 1) limited data available from the black-box model and 2) the intertwined eects of hyperparameters on the model's behavior. Since DRL models exhibit varying behaviors in identical tasks depending on their hyper-parameter congurations, we introduce a novel hyper-parameter inference attack against DRL, named HyperInfer, which allows adversaries to deduce the settings of a black-box DRL model. In order to fully assess the risk of model hyper-parameter leakage, we design two novel state generation methods that provoke divergent responses from DRL models. We also develop an inference framework to elucidate the relationship between model behavior and hyper-parameter settings. rough comprehensive experiments involving multiple DRL models and environments, we demonstrate that model behaviors can indeed reveal hyper-parameter settings, with inference accuracy surpassing 90% in scenarios such as PPO with CartPole. We also discuss keyndings relevant to practical applications and explore how knowledge of hyperparameters can facilitate more sophisticated attacks. Lastly, we propose potential defensive strategies to minimize the risk of hyper-parameter leakage in DRL models.

Machine unlearning is the primary way to fight for the “right to be forgotten” in machine learning field, which is promoted among multiple privacy legislations, such as GDPR and CCPA. However, the latest work has shown that machine unlearning in deep learning cannot be easily verified, making it challenging for the data owners to be convinced that their data has indeed been deleted as claimed. This is especially problematic for federated learning (FL), where a number of participants jointly train a global model while each participant should be free to join and leave the federation as they wish. However,the lack of a reliable approach to verify unlearning in FL will no doubt discourage certain users from joining the federation.In this work, we propose A2E, a black-box watermarking scheme from a leaving participant's perspective to realize verifiable federated unlearning which incurs minimum impact and no security threats to vanilla FL. The key idea is to leverage adversarial training to inject the anti-adversarial example (A2E) characteristic into the uploaded model updates of the last contribution round as the watermark of the leaving participant. Then, we verify whether the server has indeed executed the effective unlearning, with the newly developed probabilistic quantification of unlearning confidence, by checking the unlearned global model's resistance to the specially generated watermark-dependent adversarial examples of the leaver. We conducted large-scale experiments on various popular datasets (including natural images, medical images, and speech) and model structures (including LeNet, ResNet, VGG, and LSTM). The results confirm the effectiveness of A2E in verifying federated unlearning with a high confidence. We also show that A2E is robust against multiple adaptive strategies from the adversarial server and participants.

Previous schemes for designing secure branch prediction unit (SBPU) based on physical isolation can only offer limited security and significantly affect BPU’s prediction capability, leading to prominent performance degradation. Moreover, encryption-based SBPU schemes based on periodic key rerandomization have the risk of being compromised by advanced attack algorithms, and the performance overhead is also considerable. To this end, this paper proposes conflict-invisible SBPU (CIBPU). CIBPU employs redundant storage design, load-aware indexing, and replacement design, as well as an encryption mechanism without requiring periodic key updates, to prevent attackers’ perception of branch conflicts. We provide a thorough security analysis, which shows that CIBPU achieves strong security throughout the BPU’s lifecycle. We implement CIBPU in a RISC-V core model in gem5. The experimental results show that CIBPU causes an average performance overhead of only 2.9%–4.0% with acceptable hardware storage overhead, which is the lowest among the state-of-the-art SBPU schemes. CIBPU has also been implemented in the open-source RISC-V core, SonicBOOM, which is then burned onto an FPGA board. The evaluation based on the board shows an average performance degradation of 2.01%, which is approximately consistent with the result obtained in gem5.

Truth discovery has emerged as an effective tool to mitigate data inconsistency in crowdsensing by prioritizing data from high-quality responders. While local differential privacy (LDP) has emerged as a crucial privacy-preserving paradigm, existing studies under LDP rarely explore a worker's participation in specific tasks for sparse scenarios, which may also reveal sensitive information such as individual preferences and behaviors. Existing LDP mechanisms, when applied to truth discovery in sparse settings, may create undesirable dense distributions, provide insufficient privacy protection, and introduce excessive noise, compromising the efficacy of subsequent non-private truth discovery. Additionally, the interplay between noise injection and truth discovery remains insufficiently explored in the current literature. To address these issues, we propose a lOcally differentially private truth diSCovery approach for spArse cRowdsensing, namely OSCAR. The main idea is to use advanced optimization techniques to reconstruct the sparse data distribution and re-formalize truth discovery by considering the statistical characteristics of injected Laplacian noise while protecting the privacy of both the tasks being completed and the corresponding sensory data. Specifically, to address the data density concerns while alleviating noise, we design a randomized response based Bernoulli matrix factorization method BerRR. To recover the sparse structures from densified, perturbed data, we formalize a 0-1 integer programming problem and develop a sparse recovery solving method SpaIE based on implicit enumeration. We further devise a Laplacian-sensitive truth discovery method LapCRH that leverages maximum likelihood estimation to re-formalize truth discovery by measuring differences between noisy values and truths based on the statistical characteristic of Laplacian noise. Our comprehensive theoretical analysis establishes OSCAR's privacy guarantees, utility bounds, and computational complexity. Experimental results show that OSCAR surpasses the state-of-the-arts by at least 30% in accuracy improvement.

Data inconsistency often arises from multiple observed sensory data due to varying participant reliability for crowdsensing systems. Truth discovery, which includesWeight EstimationandTruth Aggregation, for estimating participant reliability weights and aggregating uploaded values from inconsistent observations respectively, has emerged as an effective solution to address this issue. While local differential privacy (LDP) provides strong privacy guarantees by allowing participants to perturb their data locally before submission, existing LDP-based studies are either designed for static scenarios or compromise on privacy and accuracy trade-off for data streams, satisfying only weaker versions of LDP or mere differential privacy. To effectively and efficiently obtain truths over streams under rigorous LDP, we proposeNANOwhich is locally differeNtially privAte truth discovery via updatiNg time stamp determinatiOn. The main idea lies in its integration of Laplacian noise for privacy protection and inherent Gaussian noise representing natural data variability for effective weight and truth estimations, coupled with the adaptive determination of updating time stamps. InNANO, to obtain theWeight EstimationandTruth Aggregationunder LDP, we design a mixed noise-aware truth discovery methodMixTDby modeling the mixed noise. To capture the dynamic nature of weight and truth evolutions, we develop a changing-aware updating time stamp determination methodCUDto selectively re-conduct truth discovery at specific time stamps. We also introduce a dynamic privacy budget management strategy, which accumulates unused budgets from skipped updates for critical timestamps. In this way,Weight EstimationandTruth Aggregationare limited to critical time stamps, which significantly reduces the privacy budget segmentation and computational costs. We demonstrate thatNANOprovides rigorous LDP guarantees while achieving bounded utility and computational complexity. Extensive experimental results over four real-world datasets and three synthetic datasets showcase thatNANOoutperforms the state-of-the-arts by at least 20% improvement with negligible extra efficiency loss.

In directed graphs, the relationship between users is asymmetric, resulting in two types of triangles: cycle triangles and flow triangles. This paper studies the problem of privacy-preserving triangle counting in directed graphs. Based on different applications, we consider two scenarios, i.e., trusted and untrusted servers. In the literature, privacy-preserving triangle counting in undirected graphs has been widely studied. However, directly applying these algorithms to address our problem suffers from many issues. Concretely, for the trusted server scenario, the differentially private triangle counting algorithms, designed for undirected graphs, exhibit suboptimal performance when applied to directed graphs. Hence, we propose a new centralized differentially private algorithm that adds Laplacian noise to the exact numbers by analyzing global sensitivity. Furthermore, for the untrusted server scenario, the existing techniques cannot be used to count cycle and flow triangles with differential privacy because the local view of each user in directed graphs is limited to out-neighbors rather than all neighbors. Therefore, we design a novel locally differentially private algorithm to provide local unbiased estimation, which implies that after aggregating all the local estimations on the central server side, an unbiased estimation for the numbers of cycle and flow triangles is deduced. Empirical experiments on six real-world graph datasets demonstrate that our proposed algorithms achieve high efficiency and utility.

Label differential privacy (DP) is a framework that protects the privacy of labels in training datasets, while the feature vectors are public. Existing approaches protect the privacy of labels by flipping them randomly, and then train a model to make the output approximate the privatized label. However, as the number of classes K increases, stronger randomization is needed, thus the performances of these methods become significantly worse. In this paper, we propose a vector approximation approach for learning with label local differential privacy, which is easy to implement and introduces little additional computational overhead. Instead of flipping each label into a single scalar, our method converts each label into a random vector with K components, whose expectations reflect class conditional probabilities. Intuitively, vector approximation retains more information than scalar labels. A brief theoretical analysis shows that the performance of our method only decays slightly with K. Finally, we conduct experiments on both synthesized and real datasets, which validate our theoretical analysis as well as the practical performance of our method.

As the implementation of machine learning (ML) systems becomes more widespread, especially with the introduction of larger ML models, we perceive a spring demand for massive data. However, it inevitably causes infringement and misuse problems with the data, such as using unauthorized online artworks or face images to train ML models. To address this problem, many efforts have been made to audit the copyright of the model training dataset. However, existing solutions vary in auditing assumptions and capabilities, making it difficult to compare their strengths and weaknesses. In addition, robustness evaluations usually consider only part of the ML pipeline and hardly reflect the performance of algorithms in real-world ML applications. Thus, it is essential to take a practical deployment perspective on the current dataset copyright auditing tools, examining their effectiveness and limitations. Concretely, we categorize dataset copyright auditing research into two prominent strands: intrusive methods and non-intrusive methods, depending on whether they require modifications to the original dataset. Then, we break down the intrusive methods into different watermark injection options and examine the non-intrusive methods using various finger-prints. To summarize our results, we offer detailed reference tables, highlight key points, and pinpoint unresolved issues in the current literature. By combining the pipeline in ML systems and analyzing previous studies, we highlight several future directions to make auditing tools more suitable for real-world copyright protection requirements.

The significant advances in computing power have led to a surge in model complexity. Training such models today increasingly relies on transfer learning, where models are pre-trained on large datasets and later fine-tuned for different domains, allowing the knowledge in the pre-trained model to be effectively reused and customized for these specific domains. However, such a learning paradigm also opens new attack surfaces on the fine-tuned model. Particularly, a privacy risk never studied before is the threat posed by the adversary affecting the pre-training process to the downstream user's private data for fine-tuning the model: A manipulated pre-trained model can render its fine-tuned version vulnerable to privacy attacks, such as membership inference attacks (MIAs) where the presence of a given sample in the fine-tuning dataset can be determined by querying the vulnerable model. A unique challenge in understanding this privacy risk is how to amplify the membership leakage while ensuring the performance of the fine-tuned model. To address this challenge, we introduce a new technique - active robustness overfitting (ARO). This approach actively induces robustness overfitting during pre-training, which amplifies membership leakage in the downstream task without affecting its accuracy, while also maintaining the stealthiness of the attack. Our extensive evaluations across various datasets and diverse MIA scenarios demonstrate that our methods can effectively amplify membership leakage while preserving satisfactory downstream test accuracy, which contributes to a better understanding of the privacy risk introduced by transfer learning.

Protection of local differential privacy (LDP) proto-cols against manipulation attacks is an important and challenging problem. We hope to design an attack-agnostic framework, which does not rely on any knowledge of attackers. An early work [1] restricts the attacker's capability by converting each sample into a binary signal. However, the compression of signal leads to severe loss of information, and thus results in unnecessary sacrifice of utility, especially when$\epsilon > 1$. In this paper, we propose a general estimation framework RobustLDP for robust estimation under LDP. The general idea is to send carefully crafted pre-defined information to all users, and then aggregate the feedback at the server. We strike a better tradeoff between preserving information and restricting the attacker's capability. We instantiate RobustLDP for frequency estimation and mean estimation in$\ell_{1}$and$\ell_{2}$support, which serve as building blocks for more advanced tasks. We also establish theoretical guarantees for all possible attacks. The result shows that our method significantly outperforms the existing one for$\epsilon > 1$. Extensive experiments on multiple real-world datasets validate the effectiveness of our method.

Text-to-image models based on diffusion processes, such as DALL-E, Stable Diffusion, and Midjourney, are capable of transforming texts into detailed images and have widespread applications in art and design. As such, amateur users can easily imitate professional-level paintings by collecting an artist's work and fine-tuning the model, leading to concerns about artworks' copyright infringement. To tackle these issues, previous studies either add visually imperceptible perturbation to the artwork to change its underlying styles (perturbation-based methods) or embed post-training detectable watermarks in the artwork (watermark-based methods). However, when the artwork or the model has been published online, i.e., modification to the original artwork or model retraining is not feasible, these strategies might not be viable.

Attributed graphs are extensively utilized in marketing, friend recommendations, disease prediction, etc. In attributed graphs, nodes are associated with attributes to enrich the graph representation, while edges indicate relationships between nodes. However, ensuring data privacy when publishing attributed graphs is a significant challenge due to the sensitive nature of both attributes and relationships. Existing methods fail to preserve graph structures effectively and neglect correlations among node attributes, leading to diminished utility for published synthetic graphs. To address these issues, we propose PrivAGS, a framework for publishing attributed graphs with Rényi Differential Privacy (RDP) guarantees. PrivAGS reconstructs graph structures and attributes based on community structures to capture tightly connected features. We propose a bounded Gaussian threshold mechanism to preserve attribute correlations and utilize probabilistic graph models with optimized inference structures to infer distributions and release node attributes. Additionally, PrivAGS introduces a new structural model, MCEG, to capture clustering structures and enable efficient graph reconstruction. Extensive experiments on five real-world datasets show that PrivAGS generates privacy-preserving, high-utility synthetic data.

Large language models(LLMs), represented by ChatGPT, have greatly simplified text generation tasks. However, they have also raised concerns about privacy risks such as data leakage and unauthorized information collection. Existing solutions for privacy-preserving inference face practical challenges related to computational time and communication costs. In this article, we proposeInferDPT, the first practical framework for privacy-preservingInference of closed-box LLMs, implementingDifferentialPrivacy inText generation.InferDPTcomprises two key modules: the “perturbation module” utilizes the differentially private mechanism to generate a perturbed prompt, facilitating privacy-preserving inference with closed-box LLMs; the “extraction module”, inspired by knowledge distillation and phenomenon we observed, extracts coherent and consistent text from the perturbed generation result, ensuring successful text generation completion. To achieve a better balance between utility and privacy protection, we introduce RANTEXT, a novel differentially private mechanism integrated into the perturbation module ofInferDPT, which introduces the concept of “RANdom adjacency list” forTEXTperturbation within the prompt. Experimental results across three datasets demonstrate that the text generation quality ofInferDPTis comparable to that of non-private GPT-4, and RANTEXT surpasses existing state-of-the-art mechanisms, namely, SANTEXT+ and CUSTEXT+ in the trade-off between privacy and utility. Even with a privacy parameter$\varepsilon$value of 6.0, RANTEXT achieves an average privacy protection level of exceeding 0.90 against the embedding inversion attacks, which is 0.58× higher than that of SANTEXT+ and 3.35× higher than that of CUSTEXT+.

Worker recruitment for area coverage maximization, typically requires participants to upload location information, which can deter potential participation without proper protection. While existing studies resort to geo-indistinguishability to address this concern, they primarily focus on either specific task locations (Target Coverage) or operate under pre-defined recruitment quotas for an interested region (Area Coverage). These focuses not only yield suboptimal area coverage when scaled but also fail to leverage valuable prior knowledge in the form of participants’ noisy historical registered locations, to enhance both location obfuscation and worker identification processes. To address these limitations, we presentWILTON, which optimizes area coverage under geo-indistinguishability by recruiting the minimum number of participants through the strategic utilization of noisy prior knowledge. InWILTON, to generate obfuscated locations, we propose a probabilistic and weight-aware input perturbation mechanism, which groups and weights prior locations rather than using only personal prior locations. To privately identify the recruited workers, we design a grid-based worker identification method, which provides a worst-case performance guarantee of ratio 1 − 1/eto the optimum. We provide a theoretical analysis of the privacy, utility, and complexity guarantees ofWILTON. Experimental results over two real-world datasets and one synthetic dataset show thatWILTONsurpasses the state-of-the-arts by at least 8% in area coverage improvement.

Matrix factorization-based recommendations have emerged as a cornerstone for many industrial recommendation algorithms. However, studies employing differential privacy often rely on a trusted server, while those implementing local differential privacy (LDP) frequently encounter substantial accuracy degradation and excessive communication overhead due to noise injection and frequent user interactions. Moreover, the presence of injected noise for privacy protection and inherent Gaussian noise within these perturbed values compounds the accuracy issues and may create a cascading effect under LDP, exacerbating the complexity of the problem at hand. To address these challenges, we proposeMENTOR, which isMixture model-based rEcommeNdations approach with matrix FacTORization under LDP. Its main idea lies in the adoption of a bounded input perturbation mechanism that closely approximates the Laplace distribution while providing rigorous LDP, coupled with accounting for various noise types present in the perturbed data. This approach allows us to reformulate the matrix factorization process with minimal user interaction, requiring only a single round of communication. In particular, to add noise, we design a bidirectional bounded LDP input perturbation mechanismBBVwhile minimizing variance. To generate recommendation results, we devise a matrix factorization techniqueGLMFbased on a Gaussian–Laplacian mixture model. Comprehensive experiments reveal thatMENTORoutperforms the state of the art by at least 15% inRMSEand 12% inF-Measure, showcasing its effectiveness in balancing privacy and utility in recommendation systems.

Recommender systems predict and suggest relevant options to users in various domains, such as e-commerce, streaming services, and social media. Recently, deep reinforcement learning (DRL)-based recommendation systems have become increasingly popular in academics and industry since DRL can characterize the long-term interaction between the system and users to achieve a better recommendation experience, e.g., Netflix, Spotify, Google, and YouTube.

To prevent the mischievous use of synthetic (fake) point clouds produced by generative models, we pioneer the study of detecting point cloud authenticity and attributing them to their sources. We propose an attribution framework FakePCD to attribute (fake) point clouds to their respective generative models (or real-world collections). The main idea of FakePCD is to train an attribution model that learns the point cloud features from different sources and further differentiates these sources using an attribution signal. Depending on the characteristics of the training point clouds, namely, sources and shapes, we formulate four attribution scenarios: close-world, open-world, single-shape, and multiple-shape, and evaluate FakePCD's performance in each scenario. Extensive experimental results demonstrate the effectiveness of FakePCD on source attribution across different scenarios. Take the open-world attribution as an example, FakePCD attributes point clouds to known sources with an accuracy of 0.82-0.98 and to unknown sources with an accuracy of 0.73-1.00. Additionally, we introduce an approach to visualize unique patterns (fingerprints) in point clouds associated with each source. This explains how FakePCD recognizes point clouds from various sources by focusing on distinct areas within them. Overall, we hope our study establishes a baseline for the source attribution of (fake) point clouds.1

The rapid advancements of large language models (LLMs) have raised public concerns about the privacy leakage of personally identifiable information (PII) within their extensive training datasets. Recent studies have demonstrated that an adversary could extract highly sensitive privacy data from the training data of LLMs with carefully designed prompts. However, these attacks suffer from the model's tendency to hallucinate and catastrophic forgetting (CF) in the pre-training stage, rendering the veracity of divulged PIIs negligible. In our research, we propose a novel attack, Janus, which exploits the fine-tuning interface to recover forgotten PIIs from the pre-training data in LLMs. We formalize the privacy leakage problem in LLMs and explain why forgotten PIIs can be recovered through empirical analysis on open-source language models. Based upon these insights, we evaluate the performance of Janus on both open-source language models and two latest LLMs, i.e., GPT-3.5-Turbo and LLaMA-2-7b. Our experiment results show that Janus amplifies the privacy risks by over 10 times in comparison with the baseline and significantly outperforms the state-of-the-art privacy extraction attacks including prefix attacks and in-context learning (ICL). Furthermore, our analysis validates that existing fine-tuning APIs provided by OpenAI and Azure AI Studio are susceptible to our Janus attack, allowing an adversary to conduct such an attack at a low cost.

Neural Radiance Fields (NeRF) have revolutionized 3D computer vision and graphics, facilitating novel view synthesis and influencing sectors like extended reality and e-commerce. However, NeRF's dependence on extensive data collection, including sensitive scene image data, introduces significant privacy risks when users upload this data for model training. To address this concern, we first propose SplitNeRF, a training framework that incorporates split learning (SL) techniques to enable privacy-preserving collaborative model training between clients and servers without sharing local data. Despite its benefits, we identify vulnerabilities in SplitNeRF by developing two attack methods, Surrogate Model Attack and Scene-aided Surrogate Model Attack, which exploit the shared gradient data and a few leaked scene images to reconstruct private scene information. To counter these threats, we introduce S^2NeRF, secure SplitNeRF that integrates effective defense mechanisms. By introducing decaying noise related to the gradient norm into the shared gradient information, S^2NeRF preserves privacy while maintaining a high utility of the NeRF model. Our extensive evaluations across multiple datasets demonstrate the effectiveness of S^2NeRF against privacy breaches, confirming its viability for secure NeRF training in sensitive applications.

Trajectory streams are being generated from location-aware devices, such as smartphones and in-vehicle navigation systems. Due to the sensitive nature of the location data, directly sharing user trajectories suffers from privacy leakage issues. Local differential privacy (LDP), which perturbs sensitive data on the user side before it is shared or analyzed, emerges as a promising solution for private trajectory stream collection and analysis. Unfortunately, existing stream release approaches often neglect the rich spatial-temporal context information within trajectory streams, resulting in suboptimal utility and limited types of downstream applications. To this end, we propose RetraSyn, a novel real-time trajectory synthesis framework, which is able to perform on-the-f1y trajectory synthesis based on the mobility patterns privately extracted from users' trajectory streams. Thus, the downstream trajectory analysis can be performed on the high-utility synthesized data with privacy protection. We also take the genuine behaviors of real-world mobile travelers into consideration, ensuring authenticity and practicality. The key components of RetraSyn include the global mobility model, dynamic mobility update mechanism, real-time synthesis, and adaptive allocation strategy. We conduct extensive experiments on multiple real-world and synthetic trajectory datasets under various location-based utility metrics, encompassing both streaming and historical scenarios. The empirical results demonstrate the superiority and versatility of our proposed framework.

Multiagent reinforcement learning (MARL) promises outstanding performance for multiintersection traffic signal control systems (TSCS), enabling intelligent administration of cities. However, the vulnerability of MARL algorithms to adversarial attacks has raised concerns about the security of TSCS. In this article, we explore the robustness of MARL-based TSCS against adversarial attacks, propose a black-box multiobject attack strategy, and assign an attack budget to ensure stealthiness. We design a dynamic threshold-based selection of critical states to minimize the cumulative reward with a limited number of attacks. In addition, we present a lightweight agnostic dynamic threshold-based defense mechanism by enhancing the worst-case performance of the policy. We formulate it as a min-max optimization problem, i.e., minimizing the quantity of training sample alterations while maximizing the cumulative discount reward of policy against the perturbed states. Extensive experiments on simulation of urban mobility (SUMO) demonstrate that the proposed attack policy can significantly reduce the performance of TSCS.

Recent studies have shown that systems with limited resources like Metadata-private Messenger (MPM) suffer from side-channel attacks under resource allocation (RA). In the case of MPM, which is designed to keep the identities and activities of both callers and callees private from network adversaries, an attacker can compromise a victim’s friends and keep calling the victim to infer whether the victim is busy, which breaks the privacy guarantee of MPM.

Differential privacy (DP), as a rigorous mathematical definition quantifying privacy leakage, has become a well-accepted standard for privacy protection. Combined with powerful machine learning (ML) techniques, differentially private machine learning (DPML) is increasingly important. As the most classic DPML algorithm, DP-SGD incurs a significant loss of utility, which hinders DPML's deployment in practice. Many studies have recently proposed improved algorithms based on DP-SGD to mitigate utility loss. However, these studies are isolated and cannot comprehensively measure the performance of improvements proposed in algorithms. More importantly, there is a lack of comprehensive research to compare improvements in these DPML algorithms across utility, defensive capabilities, and generalizability.

Collecting graph data is costly and well-trained graph neural networks (GNNs) are viewed as intellectual property. To make better use of GNNs, they are used to provide cloud-based services. However, models on cloud-based services may be leaked under model extraction attacks. Adversaries can extract an imitation model by simply querying the GNNs on the cloud-based services. To protect GNNs, watermarks are embedded in the models. However, the watermarks can be removed by the model extraction attacks. To address this issue, we propose adding a watermark that cannot be ignored by queries from the model extraction attacks. Concretely, we add the soft nearest neighbor loss to the loss function of the watermark embedding process to merge the distributions for the normal tasks and watermarks. We also observe that the watermark brings a performance loss to GNNs and propose an optimization method to maintain the model performance. We evaluate our method on multiple real-world datasets to demonstrate the superiority of the method.

Graph generative models become increasingly effective for data distribution approximation and data augmentation. While they have aroused public concerns about their malicious misuses or misinformation broadcasts, just as what Deepfake visual and auditory media has been delivering to society. Hence it is essential to regulate the prevalence of generated graphs. To tackle this problem, we pioneer the formulation of the generated graph detection problem to distinguish generated graphs from real ones. We propose the first framework to systematically investigate a set of sophisticated models and their performance in four classification scenarios. Each scenario switches between seen and unseen datasets/generators during testing to get closer to real-world settings and progressively challenge the classifiers. Extensive experiments evidence that all the models are qualified for generated graph detection, with specific models having advantages in specific scenarios. Resulting from the validated generality and oblivion of the classifiers to unseen datasets/generators, we draw a safe conclusion that our solution can sustain for a decent while to curb generated graph misuses.

Entity Resolution (ER) is a fundamental problem in data preparation. Standard deep ER methods have achieved state-of-the-art effectiveness, assuming that relations from different organizations are centrally stored. However, due to privacy concerns, it can be difficult to centralize data in practice, rendering standard deep ER solutions inapplicable. Despite efforts to develop rule-based privacy-preserving ER methods, they often neglect subtle matching mechanisms and have poor effectiveness as a result. To bridge effectiveness and privacy, in this paper, we propose CampER, an effective framework for privacy-aware deep entity resolution. Specifically, we first design a training pair self-generation strategy to overcome the absence of manually labeled data in privacy-aware scenarios. Based on the self-constructed training pairs, we present a collaborative fine-tuning approach to learn the match-aware and uni-space individual tuple embeddings for accurate matching decisions. During the matching decision-making process, we first introduce a cryptographically secure approach to determine matches. Furthermore, we propose an order-preserving perturbation strategy to significantly accelerate the matching computation while guaranteeing the consistency of ER results. Extensive experiments on eight widely-used benchmark datasets demonstrate that CampER not only is comparable with the state-of-the-art standard deep ER solutions in effectiveness, but also preserves privacy.

Trajectory data has the potential to greatly benefit a wide-range of real-world applications, such as tracking the spread of the disease through people's movement patterns and providing personalized location-based services based on travel preference. However, privacy concerns and data protection regulations have limited the extent to which this data is shared and utilized. To overcome this challenge, local differential privacy provides a solution by allowing people to share a perturbed version of their data, ensuring privacy as only the data owners have access to the original information. Despite its potential, existing point-based perturbation mechanisms are not suitable for real-world scenarios due to poor utility, dependence on external knowledge, high computational overhead, and vulnerability to attacks. To address these limitations, we introduce LDPTrace, a novel locally differentially private trajectory synthesis framework. Our framework takes into account three crucial patterns inferred from users' trajectories in the local setting, allowing us to synthesize trajectories that closely resemble real ones with minimal computational cost. Additionally, we present a new method for selecting a proper grid granularity without compromising privacy. Our extensive experiments using real-world as well as synthetic data, various utility metrics and attacks, demonstrate the efficacy and efficiency of LDPTrace.

Machine unlearning is a process of removing the impact of some training data from the machine learning (ML) models upon receiving removal requests. While straightforward and legitimate, retraining the ML model from scratch incurs a high computational overhead. To address this issue, a number of approximate algorithms have been proposed in the domain of image and text data, among which SISA is the state-of-the-art solution. It randomly partitions the training set into multiple shards and trains a constituent model for each shard. However, directly applying SISA to the graph data can severely damage the graph structural information, and thereby the resulting ML model utility. In this paper, we propose GraphEraser, a novel machine unlearning framework tailored to graph data. Its contributions include two novel graph partition algorithms and a learning-based aggregation method. We conduct extensive experiments on five real-world graph datasets to illustrate the unlearning efficiency and model utility of GraphEraser. It achieves 2.06x (small dataset) to 35.94x (large dataset) unlearning time improvement. On the other hand, GraphEraser achieves up to 62.5% higher F1 score and our proposed learning-based aggregation method achieves up to 112% higher F1 score. https://github.com/MinChen00/Graph-Unlearning.

Existing studies on neural architecture search (NAS) mainly focus on efficiently and effectively searching for network architectures with better performance. Little progress has been made to systematically understand if the NAS-searched architectures are robust to privacy attacks while abundant work has already shown that human-designed architectures are prone to privacy attacks. In this paper, we fill this gap and systematically measure the privacy risks of NAS architectures. Leveraging the insights from our measurement study, we further explore the cell patterns of cell-based NAS architectures and evaluate how the cell patterns affect the privacy risks of NAS-searched architectures. Through extensive experiments, we shed light on how to design robust NAS architectures against privacy attacks, and also offer a general methodology to understand the hidden correlation between the NAS-searched architectures and other privacy risks.

Previous security research efforts orbiting around graphs have been exclusively focusing on either (de-)anonymizing the graphs or understanding the security and privacy issues of graph neural networks. Little attention has been paid to understand the privacy risks of integrating the output from graph embedding models (e.g., node embeddings) with complex downstream machine learning pipelines. In this paper, we fill this gap and propose a novel model-agnostic graph recovery attack that exploits the implicit graph structural information preserved in the embeddings of graph nodes. We show that an adversary can recover edges with decent accuracy by only gaining access to the node embedding matrix of the original graph without interactions with the node embedding models. We demonstrate the effectiveness and applicability of our graph recovery attack through extensive experiments.

We study the problem of publishing a stream of real-valued data satisfying differential privacy (DP). One major challenge is that the maximal possible value in the stream can be quite large, leading to enormous DP noise and bad utility. To reduce the maximal value and noise, one way is to estimate a threshold so that values above it can be truncated. The intuition is that, in many scenarios, only a few values are large; thus truncation does not change the original data much. We develop such a method that finds a suitable threshold with DP. Given the threshold, we then propose an online hierarchical method and several post-processing techniques.

The right to be forgotten states that a data owner has the right to erase their data from an entity storing it. In the context of machine learning (ML), the right to be forgotten requires an ML model owner to remove the data owner's data from the training set used to build the ML model, a process known asmachine unlearning. While originally designed to protect the privacy of the data owner, we argue that machine unlearning may leave some imprint of the data in the ML model and thus create unintended privacy risks. In this paper, we perform the first study on investigating the unintended information leakage caused by machine unlearning. We propose a novel membership inference attack that leverages the different outputs of an ML model's two versions to infer whether a target sample is part of the training set of the original model but out of the training set of the corresponding unlearned model. Our experiments demonstrate that the proposed membership inference attack achieves strong performance. More importantly, we show that our attack in multiple cases outperforms the classical membership inference attack on the original ML model, which indicates that machine unlearning can have counterproductive effects on privacy. We notice that the privacy degradation is especially significant for well-generalized ML models where classical membership inference does not perform well. We further investigate four mechanisms to mitigate the newly discovered privacy risks and show that releasing the predicted label only, temperature scaling, and differential privacy are effective. We believe that our results can help improve privacy protection in practical implementations of machine unlearning. \footnoteOur code is available at \urlhttps://github.com/MinChen00/UnlearningLeaks.

For protecting users' private data, local differential privacy (LDP) has been leveraged to provide the privacy-preserving range query, thus supporting further statistical analysis. However, existing LDP-based range query approaches are limited by their properties, ie, collecting user data according to a pre-defined structure. These static frameworks would incur excessive noise added to the aggregated data especially in the low privacy budget setting. In this work, we propose an Adaptive Hierarchical Decomposition (AHEAD) protocol, which adaptively and dynamically controls the built tree structure, so that the injected noise is well controlled for maintaining high utility. Furthermore, we derive a guideline for properly choosing parameters for AHEAD so that the overall utility can be consistently competitive while rigorously satisfying LDP. Leveraging multiple real and synthetic datasets, we extensively show the effectiveness of AHEAD in both low and high dimensional range query scenarios, as well as its advantages over the state-of-the-art methods. In addition, we provide a series of useful observations for deploying \myahead in practice.

With the huge advantages of iris in authentication and identification, research on protecting iris information in real-world applications has gradually become an important research topic. In this paper, a method to hide the real feature information of the iris is proposed. Such an algorithm aims to prevent iris information from being maliciously utilized otherwise there will be serious consequences. In particular, we focus on accurately manipulating the iris features and ensuring that this method can also prevent iris distortion, thus people could hardly pick out the difference between before and after modification with the naked eye. Furthermore, we verify and evaluate our method and its effect through similarity detection and feature matching. The results show that it has a good performance.

Database-driven cognitive radio has been well recognized as an efficient way to reduce interference between Primary Users (PUs) and Secondary Users (SUs). In database-driven cognitive radio, PUs and SUs must provide their locations to enable dynamic channel allocation, which raises location privacy breach concern. Previous studies only focus on unilateral privacy preservation, i.e., only PUs' or SUs' privacy is preserved. In this paper, we propose to protect bilateral location privacy of PUs and SUs. The main challenge lies in how to coordinate PUs and SUs to maximize their utilities provided that their location privacy is protected. We first introduce a quantitative method to calculate both PUs' and SUs' location privacy, and then design a novel privacy preserving Utility Maximization protocol (UMax). UMax allows for both PUs and SUs to adjust their privacy preserving levels and optimize transmit power iteratively to achieve the maximum utilities. Through extensive evaluations, we demonstrate that our proposed protocol can efficiently increase the utilities of both PUs and SUs while preserving their location privacy.

Marginal tables are the workhorse of capturing the correlations among a set of attributes. We consider the problem of constructing marginal tables given a set of user's multi-dimensional data while satisfying Local Differential Privacy (LDP), a privacy notion that protects individual user's privacy without relying on a trusted third party. Existing works on this problem perform poorly in the high-dimensional setting; even worse, some incur very expensive computational overhead. In this paper, we propose CALM, Consistent Adaptive Local Marginal, that takes advantage of the careful challenge analysis and performs consistently better than existing methods. More importantly, CALM can scale well with large data dimensions and marginal sizes. We conduct extensive experiments on several real world datasets. Experimental results demonstrate the effectiveness and efficiency of CALM over existing methods.

Narrowband Internet of Things (NB-IoT) is one of the most promising technologies for low-power, wide-area, and low-traffic applications. In NB-IoT, random access is implemented in media access control layer to resolve the channel contention conflict among multiple user equipments (TIEs), and is crucial to the throughput performance of NB-IoT. Previous results in long-term evolution cannot be directly applied due to specification differences. In this paper, we take the first attempt to systematically analyze the performance of random access in NB-IoT. First, after extensively studying the backoff mechanism, we characterize the probability that a TIE initiates random access, the probability that a packet is transmitted successfully and the probability that a channel is busy. Then, we define each TIE's buffer as a first-in-first-out queue. We employ Markov chain to model retransmission number caused by collisions and the length of the queue simultaneously. By exploiting the characteristic of the steady-state distribution of the Markov chain, the above three probabilities in steady state can be obtained explicitly. Based on these probabilities, we calculate the system throughput in terms of TIE number, packet generation rate, retransmission number, and the length of the queue. Finally, we investigate the system throughput and conduct extensive simulations under various parameters, which validate our analysis.

Incentive mechanism plays a critical role in privacy-aware crowdsensing. Most previous studies assume a trustworthy fusion center (FC) in their co-design of incentive mechanism and privacy preservation. Very recent work has taken the step to relax the assumption on trustworthy FC and allowed participatory users (PUs) to randomly report their binary sensing data, whereas the focus is to examine PUs' equilibrium behavior. Making a paradigm shift, this paper aims to study the privacy compensation for continuous data sensing while allowing FC to directly control PUs. There are two conflicting objectives in such a scenario: FC desires better quality data in order to achieve higher aggregation accuracy whereas PUs prefer injecting larger noises for higher privacy-preserving levels (PPLs). To strike a good balance therein, we propose an efficient incentive mechanism named REAP to reconcile FC's aggregation accuracy and individual PU's data privacy. Specifically, we adopt the celebrated notion of differential privacy to quantify PUs' PPLs and characterize their impacts on FC's aggregation accuracy. Then, appealing to contract theory, we design an incentive mechanism to maximize FC's aggregation accuracy under a given budget. The proposed incentive mechanism offers different contracts to PUs with different privacy preferences, by which FC can directly control them. It can further overcome the information asymmetry problem, i.e., FC typically does not know each PU's precise privacy preference. We derive closed-form solutions for the optimal contracts in both complete information and incomplete information scenarios. Further, the results are generalized to the continuous case where PUs' privacy preferences take values in a continuous domain. Extensive simulations are provided to validate the feasibility and advantages of our proposed incentive mechanism.

Wearable devices enable users to collect health data and share them with healthcare providers for improved health service. Since health data contain privacy-sensitive information, unprotected data release system may result in privacy leakage problem. Most of the existing work use differential privacy for private data release. However, they have limitations in healthcare scenarios because they do not consider the unique features of health data being collected from wearables, such as continuous real-time collection and pattern preservation. In this paper, we propose Re-DPoctor, a real-time health data releasing scheme with w-day differential privacy where the privacy of health data collected from any consecutive w days is preserved. We improve utility by using a specially-designed partition algorithm to protect the health data patterns. Meanwhile, we improve privacy preservation by applying newly proposed adaptive sampling tech- nique and budget allocation method. We prove that Re-DPoctor satisfies w-day differential privacy. Experiments on real health data demonstrates that our method achieves better utility with strong privacy guarantee than existing state-of-the-art methods.

Database-driven cognitive radio has been well recognized as an efficient way to reduce interference between Primary Users (PUs) and Secondary Users (SUs). In database-driven cognitive radio, PUs and SUs must provide their locations to enable dynamic channel allocation, which raises location privacy breach concern. Previous studies only focus on unilateral privacy preservation, i.e., Only PUs' or SUs' privacy is preserved. In this paper, we propose to protect bilateral location privacy of a PU and an SU. The main challenge lies in how to coordinate the PU and SU to maximize their utility provided that their location privacy is protected. We first introduce a quantitative method to calculate both PU's and SU's location privacy, and then design a novel privacy preserving Utility Maximization protocol (UMax). UMax allows for both PU and SU to adjust their privacy preserving levels and optimize transmit power iteratively to achieve the maximum utility. Through extensive evaluations, we demonstrate that our proposed mechanism can efficiently increase the utility of both PU and SU while preserving their location privacy.