While text embeddings enable efficient semantic processing in LLMs, they remain vulnerable to inversion attacks that reconstruct sensitive original text. However, current defense methods typically treat text embeddings from the feature level independently, ignoring the exploitation of the mutual relation among the embedding construction pipeline. To address this limitation, we propose Eguard, a framework that effectively disrupts chains of relationships between the original semantic space and defended functional space. Our improvements manifest at two levels, i.e., the global-level and local-level mutual information. At the global level, we propose to minimize the statistical dependency between protected embeddings and their original inputs, effectively decoupling sensitive content from the semantic space accessible to adversaries. At the local level, we apply keyword-antonym contrastive learning to enforce semantic discriminability within the space of downstream utility. This synergy of global privacy control and local semantic alignment allows Eguard to achieve a superior privacy-utility trade-off than traditional defenses. Our approach significantly reduces privacy risks, protecting over 95 percent of tokens from inversion while maintaining high performance across downstream tasks consistent with original embeddings.

Large Language Models (LLMs) have exhibited remarkable capabilities but remain vulnerable to jailbreaking attacks, which can elicit harmful content from the models by manipulating the input prompts. Existing black-box jailbreaking techniques primarily rely on static prompts crafted with a single, non-adaptive strategy, or employ rigid combinations of several underperforming attack methods, which limits their adaptability and generalization. To address these limitations, we propose MAJIC, a Markovian adaptive jailbreaking framework that attacks black-box LLMs by iteratively combining diverse innovative disguise strategies. MAJIC first establishes a ''Disguise Strategy Pool'' by refining existing strategies and introducing several innovative approaches. To further improve the attack performance and efficiency, MAJIC formulate the sequential selection and fusion of strategies in the pool as a Markov chain. Under this formulation, MAJIC initializes and employs a Markov matrix to guide the strategy composition, where transition probabilities between strategies are dynamically adapted based on attack outcomes, thereby enabling MAJIC to learn and discover effective attack pathways tailored to the target model. Our empirical results demonstrate that MAJIC significantly outperforms existing jailbreak methods on prominent models such as GPT-4o and Gemini-2.0-flash, achieving over 90\% attack success rate with fewer than 15 queries per attempt on average.

Despite the integration of safety alignment and external filters, text-to-image (T2I) generative systems are still susceptible to producing harmful content, such as sexual or violent imagery. This raises serious concerns about unintended exposure and potential misuse. Red teaming, which aims to proactively identify diverse prompts that can elicit unsafe outputs from the T2I system, is increasingly recognized as an essential method for assessing and improving safety before real-world deployment. However, existing automated red teaming approaches often treat prompt discovery as an isolated, prompt-level optimization task, which limits their scalability, diversity, and overall effectiveness. To bridge this gap, in this paper, we propose DREAM, a scalable red teaming framework to automatically uncover diverse problematic prompts from a given T2I system. Unlike prior work that optimizes prompts individually, DREAM directly models the probabilistic distribution of the target system's problematic prompts, which enables explicit optimization over both effectiveness and diversity, and allows efficient large-scale sampling after training. To achieve this without direct access to representative training samples, we draw inspiration from energy-based models and reformulate the objective into a simple and tractable form. We further introduce GC-SPSA, an efficient optimization algorithm that provides stable gradient estimates through the long and potentially non-differentiable T2I pipeline. During inference, we also propose a diversity-aware sampling strategy to enhance prompt variety. The effectiveness of DREAM is validated through extensive experiments, demonstrating state-of-the-art performance across a wide range of T2I models and safety filters in terms of both prompt success rate and diversity. Our code is available at https://github.com/AntigoneRandy/DREAM

The substantial investment required to develop Large Language Models (LLMs) makes them valuable intellectual property, raising significant concerns about copyright protection. LLM fingerprinting has emerged as a key technique to address this, which aims to verify a model's origin by extracting an intrinsic, unique signature (a ''fingerprint'') and comparing it to that of a source model to identify illicit copies. However, existing black-box fingerprinting methods often fail to generate distinctive LLM fingerprints. This ineffectiveness arises because black-box methods typically rely on model outputs, which lose critical information about the model's unique parameters due to the usage of non-linear functions. To address this, we first leverage Fisher Information Theory to formally demonstrate that the gradient of the model's input is a more informative feature for fingerprinting than the output. Based on this insight, we propose ZeroPrint, a novel method that approximates these information-rich gradients in a black-box setting using zeroth-order estimation. ZeroPrint overcomes the challenge of applying this to discrete text by simulating input perturbations via semantic-preserving word substitutions. This operation allows ZeroPrint to estimate the model's Jacobian matrix as a unique fingerprint. Experiments on the standard benchmark show ZeroPrint achieves a state-of-the-art effectiveness and robustness, significantly outperforming existing black-box methods.

While retrieval-augmented generation systems partially address the hallucination issues in large language models, it also introduces new vulnerabilities to knowledge corruption attacks. Adversaries exploit these vulnerabilities by poisoning documents provided by RAG system to manipulate LLM outputs. To counter this threat, we propose SecureCollaRAG, a Byzantine-tolerant collaborative RAG framework leveraging Multi-source Knowledge Validation Mechanism. Our approach enables agent system to securely verify document provenance through dynamic GNN-based credibility scoring, effectively preventing stealthy knowledge corruption attacks while preserving essential domain knowledge integrity. Through extensive evaluations and formal analysis, we demonstrate that SecureCollaRAG maintains robustness against attackers under non-IID data distributions.

Backdoor attacks intend to inject hidden backdoor into the deep neural networks (DNNs), such that the predictions of infected models will be maliciously changed if the hidden backdoor is activated by the attacker-specified trigger pattern. Since the infected models behave normally on predicting benign samples, the backdoor attack is stealthy and therefore a serious threat to practical applications of DNNs. Currently, most existing backdoor attacks adopted the setting of static trigger, i.e ., triggers across the training and testing images follow the same appearance and are located in the same area. In this paper, we revisit this attack paradigm by analyzing trigger characteristics. We demonstrate that this attack paradigm is vulnerable when the trigger in testing images is not consistent with the one used for training. As such, those attacks are far less effective in the physical world, where the location and appearance of the trigger contained in digitized test samples may be different from that of the one used for training. Besides, we introduce a plug-in attack enhancement module during training, inspired by the expectation over transformation (EOT), to alleviate such inconsistency vulnerability. Based on this plug-in module, we also reveal that the widely adopted data augmentation may exacerbate the security risks of backdoor attacks, although it can enhance model performance. Moreover, we evaluate our methods on multiple benchmark datasets to verify their effectiveness. We hope that our work could inspire more explorations on the properties of backdoor attacks, to facilitate the design of more robust and secure DNNs.

Personalized federated learning (pFL) has emerged as a promising paradigm for mitigating client heterogeneity in distributed machine learning. In cross-device scenarios, however, the continuous generation of sensitive data by clients introduces severe communication bottlenecks and privacy risks, limiting the effectiveness of existing pFL methods. To ad dress these challenges, we propose Fed-EHP, a novel privacy preserving and communication-efficient framework for hetero geneous pFL. The originality of Fed-EHP lies in its task-specific synergistic integration of three customized components within a unified fog-assisted architecture: 1) Data-aware client clustering at the fog layer to alleviate statistical heterogeneity and reduce communication load; 2) MIFE-based secure aggregation to ensure strong privacy protection against inference attacks while preserving model utility; and 3) Cluster-driven personalized knowledge distillation to effectively address model heterogeneity and boost personalization across devices and fog nodes. To demonstrate privacy guarantee and security of the proposed framework, we provide a formal security analysis. We also con duct extensive experiments on MNIST, Fashion-MNIST, CIFAR 10, and CIFAR-100. Fed-EHP consistently delivers notable im provements in both accuracy and communication efficiency over state-of-the-art pFL methods. These results demonstrate that our integrated and customized framework enables capabilities and performance gains that are unattainable using existing techniques in isolation, establishing Fed-EHP as a practical and reliable solution for real-world heterogeneous federated learning.

Model watermarking is a technique that embeds identification information as watermarks to verify model ownership and protect model priority against model stealing (MS) attacks. Watermark is a type of external knowledge which typically make a model sensitive to a specific trigger pattern, causing it to misclassify patterns to a targeted class. However, current fixed form of trigger pattern makes watermarks easy to be recovered by adversaries, thus compromising their secrecy. In this paper, we propose a new approach, named AttMark, which can generate unique patterns for each input via a group of generators. The application of generators adds randomness to trigger patterns by embedding characters into samples in various ways. Therefore, it challenges the convergence of watermark recovering algorithms of adversaries. Nevertheless, random trigger patterns render them more difficult to be embedded, making it even more challenging to transfer watermarks to stolen models. Thus, we design attention-based watermarks that leverage the characteristic of attention transferring in MS attacks. By minimizing the attention deviation caused by random trigger patterns, we enable the stolen model to learn watermarks simultaneously with the primary task. AttMark is evaluated on three major MS attacks and the watermark validation rate is tested against recovering and removal attacks. The results show that our watermark cannot be recovered by adversaries and has a$30\%$stronger transferability compared to prior works. Our code will be available11https://github.com/LiuJingjinga/AttMark.git.

The Private Set Intersection (PSI) functionality is valuable which allows different organizations to collaborate in privacy-sensitive environments, and its intersection is highly valuable in analytics. With similar business interests, they can securely compute intersection and its associated values for insightful trends and patterns, which can be considered as intersection-based analysis. However, in such settings, existing PSI schemes are too costly or inadvertently leak sensitive information. Our proposed scheme, Approx-PSI, employs Oblivious Pseudo-Random Functions (OPRF) to provide two-party efficient PSI in a semi-honest model and introduces “noise” into the intersection through differential privacy (DP) to ensure individual privacy. Moreover, the noised intersection for approximate analysis has more flexibility and adaptability in both associated values and desired functions. Compared to existing schemes, Approx-PSI achieves at least$1.5 - 3 \times$faster than the most efficient prior protocols, and up to$77 - 81 \times$faster in certain settings, while reducing communication by at least 90%. Thus, the above benefits make our scheme more practical in the real-world.

In recent years, RAG has emerged as a key paradigm for enhancing large language models (LLMs). By integrating externally retrieved information, RAG alleviates issues like outdated knowledge and, crucially, insufficient domain expertise. While effective, RAG introduces new risks of external data extraction attacks (EDEAs), where sensitive or copyrighted data in its knowledge base may be extracted verbatim. These risks are particularly acute when RAG is used to customize specialized LLM applications with private knowledge bases. Despite initial studies exploring these risks, they often lack a formalized framework, robust attack performance, and comprehensive evaluation, leaving critical questions about real-world EDEA feasibility unanswered. In this paper, we present the first comprehensive study to formalize EDEAs against retrieval-augmented LLMs. We first formally define EDEAs and propose a unified framework decomposing their design into three components: extraction instruction, jailbreak operator, and retrieval trigger, under which prior attacks can be considered instances within our framework. Guided by this framework, we develop SECRET: a Scalable and EffeCtive exteRnal data Extraction aTtack. Specifically, SECRET incorporates (1) an adaptive optimization process using LLMs as optimizers to generate specialized jailbreak prompts for EDEAs, and (2) cluster-focused triggering, an adaptive strategy that alternates between global exploration and local exploitation to efficiently generate effective retrieval triggers. Extensive evaluations across 4 models reveal that SECRET significantly outperforms previous attacks, and is highly effective against all 16 tested RAG instances. Notably, SECRET successfully extracts 35% of the data from RAG powered by Claude 3.7 Sonnet for the first time, whereas other attacks yield 0% extraction. Our findings call for attention to this emerging threat.

Fine-tuning large language models (LLMs) on down-stream tasks has become a standard approach to adapt their capabilities. However, the process raises privacy concerns when using sensitive datasets, prompting increasing interest in differentially private (DP) fine-tuning methods. While existing approaches build upon the seminal work of DP-SGD, they are constrained by the inherent inefficiency bottlenecks. In this paper, we investigate the potential of DP zeroth-order methods for LLM fine-tuning, which avoids the scalability bottleneck of SGD by approximating gradients with more efficient zeroth-order gradients. We propose the stagewise DP zeroth-order method (DP-ZOSO) that dynamically schedules key hyperparameters to leverage the synergy between DP random perturbation and the gradient approximation error. To further enhance the scalability, we propose DP zeroth-order stagewise pruning method (DP-ZOPO) which reduces the trainable parameters by a data-free pruning technique requiring no extra privacy budget. We provide theoretical analysis for both proposed methods and conduct extensive empirical analysis on both encoder-only masked and decoder-only autoregressive language model, achieving impressive results in terms of scalability and utility across diverse tasks.

Model fingerprinting has emerged as a crucial mechanism for safeguarding the intellectual property of open-source models, offering a non-intrusive approach that requires no modifications to the protected model. However, our analysis reveals that existing fingerprinting techniques are fundamentally vulnerable to false claim attacks, wherein adversaries can fraudulently assert ownership over independent third-party models. We demonstrate that this vulnerability stems from the untargeted nature of current methods, which evaluate model similarity based on arbitrary sample outputs rather than alignment with a specific, predefined reference. To mitigate this vulnerability, we introduce FIT-Print, a targeted fingerprinting paradigm that actively counters false claim attacks. Specifically, FIT-Print leverages optimization to transform the fingerprint into a verifiable, targeted signature. Building upon this foundation, we propose two black-box fingerprinting methods, the bit-wise FIT-ModelDiff and the list-wise FIT-LIME, which utilize output distances and feature attributions as robust model signatures, respectively. Extensive evaluations across benchmark models and datasets show that our framework perfectly neutralizes false claim attacks (100% defense success rate) and eliminates false alarms on independent models (0.0%), all while maintaining a 100% ownership verification rate against diverse model reuse techniques.

Modern Language Models (LMs) have been successfully employed in source code generation and understanding, leading to a significant increase in research focused on learning-based code intelligence, such as automated bug repair and test case generation. Despite their great potential, LMs for code intelligence (LM4Code) are susceptible to potential pitfalls , which hinder realistic performance and further impact their reliability and applicability in real-world deployment . Such challenges drive the need for a comprehensive understanding—not just identifying these issues but delving into their possible implications and existing solutions to build more reliable LMs tailored to code intelligence. Based on a well-defined systematic research approach, we conducted an extensive literature review to uncover the pitfalls inherent in LM4Code. Finally, 121 primary studies from top-tier venues have been identified. After carefully examining these studies, we designed a taxonomy of pitfalls in LM4Code research and conducted a systematic study to summarize the issues, current solutions, implications, and challenges of different pitfalls for LM4Code systems. We developed a comprehensive classification scheme that dissects pitfalls across four crucial aspects: data collection and labeling, system design and learning, performance evaluation, and deployment and maintenance. Through this study, we aim to provide a roadmap for researchers and practitioners, facilitating their understanding and utilization of LM4Code in reliable and trustworthy ways.

Machine learning-based Android malware classifiers struggle with concept drift: the rapid evolution of malware, especially with new families, can depress classification accuracy to near-random levels. Previous research has largely centered on detecting drift samples, with expert-led label revisions on these samples to guide model retraining. However, these methods often lack a comprehensive understanding of malware concepts and provide limited guidance for effective drift adaptation, leading to high human labeling costs.

Controller Area Network (CAN) bus has been the fundamental communication method to interconnect the critical powertrain and body domains in vehicles for decades. Typical in-vehicle CAN buses employ rule-based Intrusion Detection Systems (IDS) to prevent non-compliant CAN messages for security enhancement. However, as the progress of connected autonomous vehicular techniques, more undiscovered vulnerabilities behind CAN IDS are gradually exposed, causing severe threats to vehicle safety, even threatening human life. To address it, fuzzing has become an efficient security testing method to automatically explore unknown vulnerabilities underlying the in-vehicle CAN bus. Existing fuzzing methods either conduct time-consuming random testing that is easily detected by IDS, causing inefficient vulnerability exploration, or rely on precious expert experience for higher efficiency. Instead, this paper turns to seek the power of deep generative models that automatically learn the data communication protocol for compliant fuzzing sample generation. Such a method both releases the experts' knowledge of reverse engineering, and can generate compliant samples to bypass typical rule-based IDS. To further improve the efficiency of exploring exceptions based on the generated samples, we optimize the reward function of SeqGAN with Extreme Value Theory (EVT), to fit the distribution of exception samples. With the trained model, we design a fuzzing system GFuzz4CAN, and conduct extensive experiments for evaluation. Experimental results on open-source datasets and a CAN bus device demonstrate that GFuzz4CAN can achieve$2 \sim 11 \mathrm{x}$higher efficiency compared to byte-level random fuzzing, and is feasible to identify 2 vulnerabilities on the device.

Backdoor attacks on deep neural networks (DNNs) have emerged as a significant security threat, allowing adversaries to implant hidden malicious behaviors during the model training phase. Pre-processing-based defense, which is one of the most important defense paradigms, typically focuses on input transformations or backdoor trigger inversion (BTI) to deactivate or eliminate embedded backdoor triggers during the inference process. However, these methods suffer from inherent limitations: transformation-based defenses often fail to balance model utility and defense performance, while BTI-based defenses struggle to accurately reconstruct trigger patterns without prior knowledge. In this paper, we propose REFINE, an inversion-free backdoor defense method based on model reprogramming. REFINE consists of two key components: \textbf{(1)} an input transformation module that disrupts both benign and backdoor patterns, generating new benign features; and \textbf{(2)} an output remapping module that redefines the model's output domain to guide the input transformations effectively. By further integrating supervised contrastive loss, REFINE enhances the defense capabilities while maintaining model utility. Extensive experiments on various benchmark datasets demonstrate the effectiveness of our REFINE and its resistance to potential adaptive attacks.

Knowledge distillation (KD) is a vital technique for deploying deep neural networks (DNNs) on resource-constrained devices by transferring knowledge from large teacher models to lightweight student models. While teacher models from third-party platforms may undergo security verification (e.g., backdoor detection), we uncover a novel and critical threat: distillation-conditional backdoor attacks (DCBAs). DCBA injects dormant and undetectable backdoors into teacher models, which become activated in student models via the KD process, even with clean distillation datasets. While the direct extension of existing methods is ineffective for DCBA, we implement this attack by formulating it as a bilevel optimization problem and proposing a simple yet effective method (i.e., SCAR). Specifically, the inner optimization simulates the KD process by optimizing a surrogate student model, while the outer optimization leverages outputs from this surrogate to optimize the teacher model for implanting the conditional backdoor. Our SCAR addresses this complex optimization utilizing an implicit differentiation algorithm with a pre-optimized trigger injection function. Extensive experiments across diverse datasets, model architectures, and KD techniques validate the effectiveness of our SCAR and its resistance against existing backdoor detection, highlighting a significant yet previously overlooked vulnerability in the KD process. Our code is available at https://github.com/WhitolfChen/SCAR.

Invisible Image Watermarking is crucial for ensuring content provenance and accountability in generative AI. While Gen-AI providers are increasingly integrating invisible watermarking systems, the robustness of these schemes against forgery attacks remains poorly characterized. This is critical, as forging traceable watermarks onto illicit content leads to false attribution, potentially harming the reputation and legal standing of Gen-AI service providers who are not responsible for the content. In this work, we propose WMCopier, an effective watermark forgery attack that operates without requiring any prior knowledge of or access to the target watermarking algorithm. Our approach first models the target watermark distribution using an unconditional diffusion model, and then seamlessly embeds the target watermark into a non-watermarked image via a shallow inversion process. We also incorporate an iterative optimization procedure that refines the reconstructed image to further trade off the fidelity and forgery efficiency. Experimental results demonstrate that WMCopier effectively deceives both open-source and closed-source watermark systems (e.g., Amazon’s system), achieving a significantly higher success rate than existing methods. Additionally, we evaluate the robustness of forged samples and discuss the potential defense against our attack. Code is available at: https://github.com/holdrain/WMCopier.

Text-to-image (T2I) diffusion models have achieved impressive image generation quality and are increasingly fine-tuned for personalized applications. However, these models often inherit unsafe behaviors from toxic pretraining data, raising growing safety concerns. While recent safety-driven unlearning methods have made promising progress in suppressing model toxicity, they are found to be fragile to downstream fine-tuning, as we reveal that state-of-the-art methods largely fail to retain their effectiveness even when fine-tuned on entirely benign datasets. To mitigate this problem, in this paper, we propose ResAlign, a safety-driven unlearning framework with enhanced resilience against downstream fine-tuning. By modeling downstream fine-tuning as an implicit optimization problem with a Moreau envelope-based reformulation, ResAlign enables efficient gradient estimation to minimize the recovery of harmful behaviors. Additionally, a meta-learning strategy is proposed to simulate a diverse distribution of fine-tuning scenarios to improve generalization. Extensive experiments across a wide range of datasets, fine-tuning methods, and configurations demonstrate that ResAlign consistently outperforms prior unlearning approaches in retaining safety, while effectively preserving benign generation capability. Our code and pretrained models are publicly available at https://github.com/AntigoneRandy/ResAlign.

Continuous release of data streams is frequently used in numerous applications. However, when data is sensitive, this poses privacy risks. To mitigate this risk, efforts have been devoted to devising techniques that satisfy a formal privacy notion called w-event differential privacy. Nevertheless, a recent benchmark reveals that none of the existing works offer a universally effective solution across all types of data streams, making it challenging to select an appropriate scheme for unknown data streams in practical scenarios. We identify that all existing methods are heuristic-based and make data-independent decisions. In this paper, we change this landscape by introducing SPAS which is built on data-dependent strategies. Specifically, SPAS continuously predicts an optimal publishing strategy within each sliding window that minimizes the error of the released results based on the characteristics of the data stream. Additionally, we develop a weighted sparse vector technique to control data sampling and manage privacy budget consumption following that optimal publishing strategy. Comprehensive experimental evaluations demonstrate the efficacy of SPAS in adapting to diverse one-dimensional and multi-dimensional data streams for both data release and range query tasks. Our code is open-sourced.

Semantic caching significantly reduces computational costs and improves efficiency by storing and reusing large language model (LLM) responses. However, existing systems rely primarily on matching individual queries, lacking awareness of multi-turn dialogue contexts, which leads to incorrect cache hits when similar queries appear in different conversational settings. This demonstration introduces ContextCache, a context-aware semantic caching system for multi-turn dialogues. ContextCache employs a two-stage retrieval architecture that first executes vector-based retrieval on the current query to identify potential matches and then integrates current and historical dialogue representations through self-attention mechanisms for precise contextual matching. Evaluation of real-world conversations shows that ContextCache improves precision and recall compared to existing methods. Additionally, cached responses exhibit approximately 10 times lower latency than direct LLM invocation, enabling significant computational cost reductions for LLM conversational applications.

Deep Neural Networks (DNNs) are susceptible to adversarial examples. Conventional attacks generate controlled noise-like perturbations that fail to reflect real-world scenarios and hard to interpretable. In contrast, recent unconstrained attacks mimic natural image transformations occurring in the real world for perceptible but inconspicuous attacks, yet compromise realism due to neglect of image post-processing and uncontrolled attack direction. In this paper, we propose RetouchUAA, an unconstrained attack that exploits a real-life perturbation: image retouching styles, highlighting its potential threat to DNNs. Compared to existing attacks, RetouchUAA offers several notable advantages. Firstly, RetouchUAA excels in generating interpretable and realistic perturbations through two key designs: the image retouching attack framework and the retouching style guidance module. The former custom-designed human-interpretability retouching framework for adversarial attack by linearizing images while modelling the local processing and retouching decision-making in human retouching behaviour, provides an explicit and reasonable pipeline for understanding the robustness of DNNs against retouching. The latter guides the adversarial image towards standard retouching styles, thereby ensuring its realism. Secondly, attributed to the design of the retouching decision regularization and the persistent attack strategy, RetouchUAA also exhibits outstanding attack capability and defense robustness, posing a heavy threat to DNNs. Experiments on ImageNet, Place365 and CUB200 reveal that RetouchUAA achieves nearly 100% white-box attack success against three DNNs, while achieving a better trade-off between image naturalness, transferability and defense robustness than baseline attacks.

Adversarial training has been widely explored for mitigating attacks against deep models. However, a critical limitation of existing works is that robustness enhancement is at the cost of noticeable accuracy degradation. To achieve a better trade-off between robustness and accuracy, we propose the Vanilla Feature Distillation Adversarial Training (VFDAT), which conducts knowledge distillation from a pre-trained model (optimized towards high accuracy) to guide adversarial training model towards generating high-quality and well-separable features by constraining the obtained features of natural and adversarial examples. More specifically, both adversarial examples and their natural counterparts are forced to be aligned in feature space by distilling predictive representations from a pre-trained natural model. In this way, the adversarial training model can be updated towards maximally preserving the accuracy as gaining robustness. A key advantage of our method is that it can be universally adapted to and boost existing works. Exhaustive experiments on various datasets, classification models, and adversarial training algorithms demonstrate the effectiveness of our proposed method.

Knowledge Graph Embedding (KGE) has been widely studied as an important semantic enhancement technique that extracts expressive representation from Knowledge Graph (KG) to facilitate various downstream applications, such as knowledge reasoning, Semantic Web, and question answering. The advent of Federated KGE (FKGE) allows for collaborative training across distributed KGs without revealing clients’ private raw KGs that contain sensitive knowledge graph triples. Despite this, FKGE remains susceptible to privacy threats as demonstrated in previously studied federated learning models. However, utilizing and addressing these vulnerabilities remain uninvestigated for FKGE which exhibits unique characteristics distinct from other models. In this work, we conduct the first comprehensive study of the privacy issues in FKGE from both attack and defense perspectives. On the attack side, we introduce five new inference attacks, highlighting the privacy vulnerabilities by successfully deducing the presence of KG triples from the targeted dataset. On the defense side, we present PDP-Flames, a novel differentially private FKGE scheme that leverages the sparse gradient nature of FKGE for better privacy-utility trade-off by integrating advanced private selection techniques. We further introduce a dynamic defense policy based on the observation that the privacy risk fluctuates throughout the training procedure. Additionally, we incorporate a personalized procedure to provide a customized model tailored to the unique data distributions of individual clients. Joint differential privacy is introduced to guarantee the privacy of the personalized models. Comprehensive experiments demonstrate that PDP-Flames effectively mitigates privacy concerns, notably diminishing the attack success rate while maintaining decent model utility.

Federated learning (FL) is a distributed machine learning paradigm allowing multiple clients to collaboratively train a global model without sharing their local data. However, FL entails exposing the model to various participants. This poses a risk of unauthorized model distribution or resale by the malicious client, compromising the intellectual property rights of the FL group. To deter such misbehavior, it is essential to establish a mechanism for verifying the ownership of the model and as well tracing its origin to the leaker among the FL participants. In this paper, we present FedTracker, the first FL model protection framework that provides both ownership verification and traceability. FedTracker adopts a bi-level protection scheme consisting of global watermark mechanism and local fingerprint mechanism. The former authenticates the ownership of the global model, while the latter identifies which client the model is derived from. FedTracker leverages Continual Learning (CL) principles to embed the watermark in a way that preserves the utility of the FL model on both primitive task and watermark task. FedTracker also devises a novel metric to better discriminate different fingerprints. Experimental results show FedTracker is effective in ownership verification, traceability, and maintains good fidelity and robustness against various watermark removal attacks.

Neural networks are vulnerable to data inference attacks, including the membership inference attack, the model inversion attack, and the attribute inference attack. In this paper, we proposePurifierto defend against membership inference attacks by quantifying the differences between dataset members and non-members in three dimensions: individual shape, statistical distribution, and prediction label.Purifierinvolves transforming the confidence scores produced by the target classifier, resulting in purified confidence scores that are indistinguishable across the dimensions above. We conduct experiments on widely-used datasets and models. The results show thatPurifieroffers robust defense against membership inference attacks with superior efficacy compared to prior defense techniques while maintaining minimal utility degradation (e.g., less than 0.7% classification accuracy drop of most datasets). Additionally, our extended experiments explore the effectiveness ofPurifierin defending against the model inversion attack and the attribute inference attack.

Machine Learning as a Service (MLaaS) platforms have gained popularity due to their accessibility, cost-efficiency, scalability, and rapid development capabilities. However, recent research has highlighted the vulnerability of cloud-based models in MLaaS to model extraction attacks. In this paper, we introduce FDINet, a novel defense mechanism that leverages the feature distribution of deep neural network (DNN) models. Concretely, by analyzing the feature distribution from the adversary's queries, we reveal that the feature distribution of these queries deviates from that of the model's problem domain. Based on this key observation, we propose Feature Distortion Index (FDI), a metric designed to quantitatively measure the feature distribution deviation of received queries. The proposed FDINet utilizes FDI to train a binary detector and exploits FDI similarity to identify colluding adversaries from distributed extraction attacks. We conduct extensive experiments to evaluate FDINet against six state-of-the-art extraction attacks on four benchmark datasets and four popular model architectures. Empirical results demonstrate the following findings: (1) FDINet proves to be highly effective in detecting model extraction, achieving a100% detection accuracyon DFME and DaST. (2) FDINet is highly efficient, using just 50 queries to raise an extraction alarm with anaverage confidence of 96.08%for GTSRB. (3) FDINet exhibits the capability to identify colluding adversaries with an accuracyexceeding 91%. Additionally, it demonstrates the ability to detect two types of adaptive attacks.

Currently, sample-specific backdoor attacks (SSBAs) are the most advanced and malicious methods since they can easily circumvent most of the current backdoor defenses. In this paper, we reveal that SSBAs are not sufficiently stealthy due to their poisoned-label nature, where users can discover anomalies if they check the image-label relationship. In particular, we demonstrate that it is ineffective to directly generalize existing SSBAs to their clean-label variants by poisoning samples solely from the target class. We reveal that it is primarily due to two reasons, including(1)the ‘antagonistic effects’ of ground-truth features and(2)the learning difficulty of sample-specific features. Accordingly, trigger-related features of existing SSBAs cannot be effectively learned under the clean-label setting due to their mild trigger intensity required for ensuring stealthiness. We argue that the intensity constraint of existing SSBAs is mostly because their trigger patterns are ‘content-irrelevant’ and therefore act as ‘noises’ for both humans and DNNs. Motivated by this understanding, we propose to exploit content-relevant features,$a.k.a.$(human-relied) attributes, as the trigger patterns to design clean-label SSBAs. This new attack paradigm is dubbed backdoor attack with attribute trigger (BAAT). Extensive experiments are conducted on benchmark datasets, which verify the effectiveness of our BAAT and its resistance to existing defenses.

FL is vulnerable to model poisoning attacks due to the invisibility of local data and the decentralized nature of FL training. The adversary attempts to maliciously manipulate local model gradients to compromise the global model (i.e., victim model). Commonly-studied model poisoning attacks heavily depend on accessing additional knowledge, such as local data and the aggregation algorithm from the victim model, which easily encounter practical obstacles due to limited adversarial knowledge. In this paper, we first reveal that aggregated gradients in FL can serve as an attack carrier, exposing the latent knowledge of the victim model. In particular, we propose a data-free model poisoning attack named FedGhost, which aims to redirect the training objective of FL towards the adversary’s objective without any auxiliary information. In FedGhost, we design a black-box adaptive optimization algorithm to dynamically adjust the perturbation factor for malicious gradients, maximizing the poisoning impact of FL. Experimental results on five datasets in IID and Non-IID FL settings demonstrate that FedGhost achieves the highest attack success rate, outperforming other state-of-the-art model poisoning attacks by more than$10\%-60\%$.

Recently, point clouds have been widely used in computer vision, whereas their collection is time-consuming and expensive. As such, point cloud datasets are the valuable intellectual property of their owners and deserve protection. To detect and prevent unauthorized use of these datasets, especially for commercial or open-sourced ones that cannot be sold again or used commercially without permission, we intend to identify whether a suspicious third-party model is trained on our protected dataset under the black-box setting. We achieve this goal by designing ascalableclean-label backdoor-based dataset watermark for point clouds that ensures both effectiveness and stealthiness. Unlike existing clean-label watermark schemes, which were susceptible to the number of categories, our method can watermark samples from all classes instead of only from the target one. Accordingly, it can still preserve high effectiveness even on large-scale datasets with many classes. Specifically, we perturb selected point clouds with non-target categories in both shape-wise and point-wise manners before inserting trigger patterns without changing their labels. The features of perturbed samples are similar to those of benign samples from the target class. As such, models trained on the watermarked dataset will have a distinctive yet stealthy backdoor behavior,$i.e$., misclassifying samples from the target class whenever triggers appear, since the trained DNNs will treat the inserted trigger pattern as a signal to deny predicting the target label. We also design a hypothesis-test-guided dataset ownership verification based on the proposed watermark. Extensive experiments on benchmark datasets are conducted, verifying the effectiveness of our method and its resistance to potential removal methods. The codes are available athttps://github.com/weic0810/PointNCBW.

Large Language Models (LLMs) have shown impressive performance in natural language tasks, but their outputs can exhibit undesirable attributes or biases. Existing methods for steering LLMs toward desired attributes often assume unbiased representations and rely solely on steering prompts. However, the representations learned from pre-training can introduce semantic biases that influence the steering process, leading to suboptimal results. We propose LLMGuardrail, a novel framework that incorporates causal analysis and adversarial learning to obtain unbiased steering representations in LLMs. LLMGuardrail systematically identifies and blocks the confounding effects of biases, enabling the extraction of unbiased steering representations. Experiments demonstrate LLMGuardrail's effectiveness in steering LLMs toward desired attributes while mitigating biases. Our work contributes to developing safe and reliable LLMs that align with desired attributes.

Over the past years, Machine Learning-as-a-Service (MLaaS) has received a surging demand for supporting Machine Learning-driven services to offer revolutionized user experience across diverse application areas. MLaaS provides inference service with low inference latency based on an ML model trained using a dataset collected from numerous individual data owners. Recently, for the sake of data owners' privacy and to comply with the "right to be forgotten (RTBF)" as enacted by data protection legislation, many machine unlearning methods have been proposed to remove data owners' data from trained models upon their unlearning requests. However, despite their promising efficiency, almost all existing machine unlearning methods handle unlearning requests independently from inference requests, which unfortunately introduces a new security issue of inference service obsolescence and a privacy vulnerability of undesirable exposure for machine unlearning in MLaaS.

Prompts have significantly improved the performance of pre-trained Large Language Models (LLMs) on various downstream tasks recently, making them increasingly indispensable for a diverse range of LLM application scenarios. However, the backdoor vulnerability, a serious security threat that can maliciously alter the victim model’s normal predictions, has not been sufficiently explored for prompt-based LLMs. In this paper, we present PoisonPrompt, a novel backdoor attack capable of successfully compromising both hard and soft prompt-based LLMs. We evaluate the effectiveness, fidelity, and robustness of PoisonPrompt through extensive experiments on three popular prompt methods, using six datasets and three widely used LLMs. Our findings highlight the potential security threats posed by backdoor attacks on prompt-based LLMs and emphasize the need for further research in this area.

Recent studies revealed that using third-party models may lead to backdoor threats, where adversaries can maliciously manipulate model predictions based on backdoors implanted during model training. Arguably, backdoor trigger inversion (BTI), which generates trigger patterns of given benign samples for a backdoored model, is the most critical module for backdoor defenses used in these scenarios. With BTI, defenders can remove backdoors by fine-tuning based on generated poisoned samples with ground-truth labels or deactivate backdoors by removing trigger patterns during the inference process. However, we find that existing BTI methods suffer from relatively poor performance, $i.e.$, their generated triggers are significantly different from the ones used by the adversaries even in the feature space. We argue that it is mostly because existing methods require to 'extract' backdoor features at first, while this task is very difficult since defenders have no information ($e.g.$, trigger pattern or target label) about poisoned samples. In this paper, we explore BTI from another perspective where we decouple benign features instead of decoupling backdoor features directly. Specifically, our method consists of two main steps, including \textbf{(1)} decoupling benign features and \textbf{(2)} trigger inversion by minimizing the differences between benign samples and their generated poisoned version in decoupled benign features while maximizing the differences in remaining backdoor features. In particular, our method is more efficient since it doesn't need to `scan' all classes to speculate the target label, as required by existing BTI. We also exploit our BTI module to further design backdoor-removal and pre-processing-based defenses. Extensive experiments on benchmark datasets demonstrate that our defenses can reach state-of-the-art performances.

Streaming data, crucial for applications like crowd-sourcing analytics, behavior studies, and real-time monitoring, faces significant privacy risks due to the large and diverse data linked to individuals. In particular, recent efforts to release data streams, using the rigorous privacy notion of differential privacy (DP), have encountered issues with unbounded privacy leakage. This challenge limits their applicability to only a finite number of time slots ("finite data stream") or relaxation to protecting the events ("event or w-event DP") rather than all the records of users. A persistent challenge is managing the sensitivity of outputs to inputs in situations where users contribute many activities and data distributions evolve over time. In this paper, we present a novel technique for Differentially Private data streaming over Infinite disclosure (DPI) that effectively bounds the total privacy leakage of each user in infinite data streams while enabling accurate data collection and analysis. Furthermore, we also maximize the accuracy of DPI via a novel boosting mechanism. Finally, extensive experiments across various streaming applications and real datasets (e.g., COVID-19, Network Traffic, and USDA Production), show that DPI maintains high utility for infinite data streams in diverse settings. Code for DPI is available at https://github.com/ShuyaFeng/DPI.

Modern deep neural network models (DNNs) require extensive data for optimal performance, prompting reliance on multiple entities for the acquisition of training datasets. One prominent security threat is backdoor attacks where the adversary party poisons a small subset of training datasets to implant a backdoor into the model, leading to misclassifications during runtime for triggered samples. To mitigate the attack, many defense methods have been proposed, such as detecting and removing poisoned samples or rectifying trojaned model weights in victim DNNs. However, existing approaches suffer from notable inefficiency as they are faced with large-scale training datasets, consequently rendering these defenses impractical in the real world. In this paper, we propose a lightweight backdoor identification and removal scheme, called ReBack. In this scheme, ReBack first extracts a subset of suspicious and benign samples, and then, proceeds with a "averaging and differencing" based method to identify target label(s). Next, leveraging the identification results, ReBack invokes a novel reverse engineering method to recover the exact trigger using only basic arithmetic atoms. Our experiments demonstrate that, for ImageNet with 750 labels, ReBack can defend against backdoor attacks in around 2 hours, showcasing a speed improvement of 18.5× to 214× compared to existing methods. For backdoor removal, the attack success rate can be decreased to 0.05% owing to 99% cosine similarity of the reversed triggers. The code is online available.

Large language models (LLMs) have witnessed a meteoric rise in popularity among the general public users over the past few months, facilitating diverse downstream tasks with human-level accuracy and proficiency. Prompts play an essential role in this success, which efficiently adapt pre-trained LLMs to task-specific applications by simply prepending a sequence of tokens to the query texts. However, designing and selecting an optimal prompt can be both expensive and demanding, leading to the emergence of Prompt-as-a-Service providers who profit by providing well-designed prompts for authorized use. With the growing popularity of prompts and their indispensable role in LLM-based services, there is an urgent need to protect the copyright of prompts against unauthorized use.In this paper, we propose PromptCARE, the first framework for prompt copyright protection through watermark injection and verification. Prompt watermarking presents unique challenges that render existing watermarking techniques developed for model and dataset copyright verification ineffective. PromptCARE overcomes these hurdles by proposing watermark injection and verification schemes tailor-made for characteristics pertinent to prompts and the natural language domain. Extensive experiments on six well-known benchmark datasets, using three prevalent pre-trained LLMs (BERT, RoBERTa, and Facebook OPT-1.3b), demonstrate the effectiveness, harmlessness, robustness, and stealthiness of PromptCARE.

Top-k frequent items detection is a fundamental task in data stream mining. Many promising solutions are proposed to improve memory efficiency while still maintaining high accuracy for detecting the Top-k items. Despite the memory efficiency concern, the users could suffer from privacy loss if participating in the task without proper protection, since their contributed local data streams may continually leak sensitive individual information. However, most existing works solely focus on addressing either the memory-efficiency problem or the privacy concerns but seldom jointly, which cannot achieve a satisfactory tradeoff between memory efficiency, privacy protection, and detection accuracy. In this paper, we present a novel framework HG-LDP to achieve accurate Top-k item detection at bounded memory expense, while providing rigorous local differential privacy (LDP) protection. Specifically, we identify two key challenges naturally arising in the task, which reveal that directly applying existing LDP techniques will lead to an inferior "accuracy-privacy-memory efficiency" tradeoff. Therefore, we instantiate three advanced schemes under the framework by designing novel LDP randomization methods, which address the hurdles caused by the large size of the item domain and by the limited space of the memory. We conduct comprehensive experiments on both synthetic and real-world datasets to show that the proposed advanced schemes achieve a superior "accuracy-privacy-memory efficiency" tradeoff, saving 2300× memory over baseline methods when the item domain size is 41,270. Our code is anonymously open-sourced via the link.

Numerous studies have underscored the significant privacy risks associated with various leakage patterns in encrypted data stores. While many solutions have been proposed to mitigate these leakages, they either (1) incur substantial overheads, (2) focus on specific subsets of leakage patterns, or (3) apply the same security notion across various workloads, thereby impeding the attainment of fine-tuned privacy-efficiency trade-offs. In light of various detrimental leakage patterns, this paper starts with an investigation into which specific leakage patterns require our focus in the contexts of key-value, range-query, and dynamic workloads, respectively. Subsequently, we introduce new security notions tailored to the specific privacy requirements of these workloads. Accordingly, we propose and instantiate Swat, an efficient construction that progressively enables these workloads, while provably mitigating system-wide leakage via a suite of algorithms with tunable privacy-efficiency trade-offs. We conducted extensive experiments and compiled a detailed result analysis, showing the efficiency of our solution. Swat is about an order of magnitude slower than an encryption-only data store that reveals various leakage patterns and is two orders of magnitude faster than a trivial zero-leakage solution. Meanwhile, the performance of Swat remains highly competitive compared to other designs that mitigate specific types of leakage.

With AI systems widely deployed in societal applications, the fairness of these models is of increasing concern, for instance, hiring systems should recommend applicants impartially from different demographic groups, and risk assessment systems must eliminate racial inequity in the criminal justice system. Therefore, ensuring fairness in these models is crucial. In this paper, we propose Task-Free Fairness-Aware Adversarial Perturbation (TF-FAAP), a flexible approach for improving the fairness of black-box deployed models by adding perturbations on input samples that blind their fairness-related attribute information without modifying the model's parameters or structures. The proposed TF-FAAP consists of a discriminator and a generator to create universal fairness-aware perturbations for a variety of tasks. The former aims to distinguish fairnessrelated attributes, and the latter generates perturbations to make the discriminator's prediction distribution of fairness-related attributes uniform. To preserve the utility of perturbed samples, we maximize the mutual information between their representations and corresponding original samples, retaining more original samples' information. In addition, the perturbation generated by TF-FAAP has a high transferability, i.e., the perturbations learned on one dataset can also alleviate the unfairness of a model trained on a different dataset. The extensive experimental evaluation demonstrated the effectiveness and superior performance of our method.

The shuffle model is recently proposed to address the issue of severe utility loss in Local Differential Privacy (LDP) due to distributed data randomization. In the shuffle model, a shuffler is utilized to break the link between the user identity and the message uploaded to the data analyst. Since less noise needs to be introduced to achieve the same privacy guarantee, following this paradigm, the utility of privacy-preserving data collection is improved. We propose DUMP (DUMmy-Point-based), a framework for privacy-preserving histogram estimation in the shuffle model. The core of DUMP is a new concept ofdummy blanket, which enables enhancing privacy by just introducing dummy points on the user side and further improving the utility of the shuffle model. We instantiate DUMP by proposing two protocols: pureDUMP and mixDUMP, and conduct a comprehensive experimental evaluation to compare them with existing protocols. The experimental results show that, under the same privacy guarantee, (1) the proposed protocols have significant improvements in communication efficiency over all existing multi-message protocols, by at least 3 orders of magnitude; (2) they achieve competitive utility, while the only known protocol (Ghaziet al., PMLR 2020) having better utility than ours employs hard-to-exactly-sample distributions which are vulnerable to floating-point attacks (CCS 2012).

Deep unsupervised domain adaptation (UDA) has significantly boosted the performance of deep models on different domains by transferring knowledge from a source domain to a target domain. However, its robustness against adversarial attacks has not been explored due to the challenges of highly non-convex deep models and different data distribution. In this paper, we give the first attempt to analyze the vulnerability of deep UDA and propose a label-free poisoning attack (LFPA), which injects poisoning data into the training data to mislead adaptation between the two domains without ground truth in target domain. Specifically, we design an unsupervised adversarial loss as the attack goal, in which the pseudo-labels are used to approximate the ground-truth. Since retraining the model will gradually degrade the attack performance, we also add a regularization term to the unsupervised loss, which eliminates negative interactions between the training goal and the attack goal. To accelerate the craft of poisons, we select influential samples as the initial poisons and propose a fast reverse-mode optimization method which updates poisons according to the approximate truncated gradients. Experimental results on multiple state-of-the-art deep UDA methods demonstrate the effectiveness of the proposed LFPA and the high sensitivity of UDA to poisoning attacks.

Recent substitute training methods have utilized the concept of Generative Adversarial Networks (GANs) to implement data-free black-box attacks. Specifically, in designing the generators, the substitute training methods use a similar structure to the generators in GANs. However, this design approach ignores the potential situation that the generators in GANs operate under real data supervision, while the generators in substitute training methods lack such supervision. This difference in data-supervised conditions constrain the diversity of data generated by the substitute training methods, resulting in inadequate data to support effective training of the substitute model. This impacts the substitute model's ability to attack the target model further. Consequently, to solve the above issues, we propose three strategies to improve the attack success rates. For the generator, we first propose a dense projection space that projects the input noise into various latent feature spaces to diversify feature information. Then, we introduce a novel disguised natural color mode. This mode improves information exchange between the generator's output layer and previous layers, allowing for more diverse generated data. Besides, we present a regularization method for the substitute model, called noise-based balanced learning, to prevent the potential risk of overfitting due to the lack of diversity of the generated data. In the experimental analysis, extensive experiments are conducted to validate the effectiveness of these proposed strategies.

With the performance of deep neural networks (DNNs) remarkably improving, DNNs have been widely used in many areas. Consequently, the DNN model has become a valuable asset, and its intellectual property is safeguarded by ownership verification techniques (e.g., DNN fingerprinting). However, the feasibility of the DNN fingerprint removal attack and its potential influence remains an open problem. In this paper, we perform the first comprehensive investigation of DNN fingerprint removal attacks. Generally, the knowledge contained in a DNN model can be categorized into general semantic and fingerprint-specific knowledge. To this end, we propose a min-max bilevel optimization-based DNN fingerprint removal attack namedRemovalNet, to evade model ownership verification. The lower-level optimization is designed to remove fingerprint-specific knowledge. While in the upper-level optimization, we distill the victim model's general semantic knowledge to maintain the surrogate model's performance. We conduct extensive experiments to evaluate thefidelity,effectiveness, andefficiencyof theRemovalNetagainst four advanced defense methods on six metrics. The empirical results demonstrate that (1) theRemovalNetiseffective. After our DNN fingerprint removal attack, the model distance between the target and surrogate models is ×100 times higher than that of the baseline attacks, (2) theRemovalNetisefficient. It uses only 0.2% (400 samples) of the substitute dataset and 1,000 iterations to conduct our attack. Besides, compared with advanced model stealing attacks, theRemovalNetsaves nearly 85% of computational resources at most, (3) theRemovalNetachieves highfidelitythat the created surrogate model maintains high accuracy after the DNN fingerprint removal process.

Swift advancements in large language model (LLM) technologies lead to widespread research and applications, particularly in integrating LLMs with auxiliary tools, known as tool-using LLM agents. However, amid user interactions, the transmission of private information to both LLMs and tools poses considerable privacy risks to users. In this paper, we delve into current privacy-preserving solutions for LLMs and outline three pivotal challenges for tool-using LLM agents: generalization to both open-source and closed-source LLMs and tools, compliance with privacy requirements, and applicability to unrestricted tasks. To tackle these challenges, we present PrivacyAsst, the first privacy-preserving framework tailored for tool-using LLM agents, encompassing two solutions for different application scenarios. First, we incorporate a homomorphic encryption scheme to ensure computational security guarantees for users as a safeguard against both open-source and closed-source LLMs and tools. Moreover, we propose a shuffling-based solution to broaden the framework's applicability to unrestricted tasks. This solution employs an attribute-based forgery generative model and an attribute shuffling mechanism to craft privacy-preserving requests, effectively concealing individual inputs. Additionally, we introduce an innovative privacy concept,$t$-closeness in image data, for privacy compliance within this solution. Finally, we implement PrivacyAsst, accompanied by two case studies, demonstrating its effectiveness in advancing privacy-preserving artificial intelligence.

Deep learning classifiers achieve state-of-the-art performance in various risk detection applications. They explore rich semantic representations and are supposed to automatically discover risk behaviors. However, due to the lack of transparency, the behavioral semantics cannot be conveyed to downstream security experts to reduce their heavy workload in security analysis. Although feature attribution (FA) methods can be used to explain deep learning, the underlying classifier is still blind to what behavior is suspicious, and the generated explanation cannot adapt to downstream tasks, incurring poor explanation fidelity and intelligibility.

Phishing attacks have always been a security issue that has attracted great attention in the cyber security community. Recently, the famous pre-trained models is being used as an anti-phishing solution. However, existing studies either simply transfer models pre-trained on text to phishing detection task, or pre-train models using only extremely small phishing samples. In this paper, we propose PhishBERT, a veritable pretrained deep transformer network model for phishing URL detection. Using a tailor pre-training objective, PhishBERT obtained a general understanding of various URLs by being pretrained on a corpus of more than 3 billion unlabeled URL data. It is then transferred to the detection task of benign and malicious URL data, with supervised fine-tuning using adversarial methods. Extensive and rigorous benchmark studies verify that PhishBERT is significantly superior to the current state-of-the-art methods in terms of efficiency, robustness and accuracy on the task of phishing website detection.

Machine unlearning is an emerging task of removing the influence of selected training datapoints from a trained model upon data deletion requests, which echoes the widely enforced data regulations mandating the Right to be Forgotten. Many unlearning methods have been proposed recently, achieving significant efficiency gains over the naive baseline of retraining from scratch. However, existing methods focus exclusively on unlearning from standard training models and do not apply to adversarial training models (ATMs) despite their popularity as effective defenses against adversarial examples. During adversarial training, the training data are involved in not only an outer loop for minimizing the training loss, but also an inner loop for generating the adversarial perturbation. Such bi-level optimization greatly complicates the influence measure for the data to be deleted and renders the unlearning more challenging than standard model training with single-level optimization. This paper proposes a new approach called MUter for unlearning from ATMs. We derive a closed-form unlearning step underpinned by a total Hessian-related data influence measure, while existing methods can mis-capture the data influence associated with the indirect Hessian part. We further alleviate the computational cost by introducing a series of approximations and conversions to avoid the most computationally demanding parts of Hessian inversions. The efficiency and effectiveness of MUter have been validated through experiments on four datasets using both linear and neural network models.

In recent years, the underground economy is proliferating in the mobile system. These underground economy apps (UEware for short) make profits from providing non-compliant services, especially in sensitive areas (e.g., gambling, porn, loan). Unlike traditional malware, most of them (over 80%) do not have malicious payloads. Due to their unique characteristics, existing detection approaches cannot effectively and efficiently mitigate this emerging threat. To address this problem, we propose a novel approach to effectively and efficiently detect UEware by considering their UI transition graphs (UTGs). Based on the proposed approach, we design and implement a system, named DeUEDroid, to perform the detection. To evaluate DeUEDroid, we collect 25, 717 apps and build up the first large-scale ground-truth dataset (1, 700 apps) of UEware. The evaluation result based on the ground-truth dataset shows that DeUEDroid can cover new UI features and statically construct precise UTG. It achieves 98.22% detection F1-score and 98.97% classification accuracy, a significantly better performance than the traditional approaches. The evaluation result involving 24, 017 apps demonstrates the effectiveness and efficiency of UEware detection in real-world scenarios. Furthermore, the result also reveals that UEware are prevalent, i.e., 54% apps in the wild and 11% apps in the app stores are UEware. Our work sheds light on the future work of analyzing and detecting UEware. To engage the community, we have made our prototype system and the dataset available online.

Federated Learning (FL) allows clients to form a consortium to train a global model under the orchestration of a central server while keeping data on the local client without sharing it, thus mitigating data privacy issues. However, training a robust global model is challenging since the local data is invisible to the server. The local data of clients are naturally heterogeneous, while some clients can use corrupted data or send malicious updates to interfere with the training process artificially. Meanwhile, communication and computation costs are inevitable challenges in designing a practical FL algorithm. In this paper, to improve the robustness of FL, we propose a Shapley value-inspired adaptive weighting mechanism, which regards the FL training as sequential cooperative games and adjusts clients' weights according to their contributions. We also develop a client sampling strategy based on importance sampling, which can reduce the communication cost by optimizing the variance of the global updates according to the weights of clients. Furthermore, to diminish the computation cost of the server, we propose a weight calculation method by estimating differences between the Shapley value of clients. Our experimental results on several real data sets demonstrate the effectiveness of our approaches.

We study the problem of $(\epsilon,\delta)$-certified machine unlearning for minimax models. Most of the existing works focus on unlearning from standard statistical learning models that have a single variable and their unlearning steps hinge on the direct Hessian-based conventional Newton update. We develop a new $(\epsilon,\delta)$-certified machine unlearning algorithm for minimax models. It proposes a minimax unlearning step consisting of a total Hessian-based complete Newton update and the Gaussian mechanism borrowed from differential privacy. To obtain the unlearning certification, our method injects calibrated Gaussian noises by carefully analyzing the ''sensitivity'' of the minimax unlearning step (i.e., the closeness between the minimax unlearning variables and the retraining-from-scratch variables). We derive the generalization rates in terms of population strong and weak primal-dual risk for three different cases of loss functions, i.e., (strongly-)convex-(strongly-)concave losses. We also provide the deletion capacity to guarantee that a desired population risk can be maintained as long as the number of deleted samples does not exceed the derived amount. With training samples $n$ and model dimension $d$, it yields the order $\mathcal O(n/d^{1/4})$, which shows a strict gap over the baseline method of differentially private minimax learning that has $\mathcal O(n/d^{1/2})$. In addition, our rates of generalization and deletion capacity match the state-of-the-art rates derived previously for standard statistical learning models.

Knowledge Graph Embedding (KGE) is a fundamental technique that extracts expressive representation from knowledge graph (KG) to facilitate diverse downstream tasks. The emerging federated KGE (FKGE) collaboratively trains from distributed KGs held among clients while avoiding exchanging clients’ sensitive raw KGs, which can still suffer from privacy threats as evidenced in other federated model trainings (e.g., neural networks). However, quantifying and defending against such privacy threats remain unexplored for FKGE which possesses unique properties not shared by previously studied models. In this paper, we conduct the first holistic study of the privacy threat on FKGE from both attack and defense perspectives. For the attack, we quantify the privacy threat by proposing three new inference attacks, which reveal substantial privacy risk by successfully inferring the existence of the KG triple from victim clients. For the defense, we propose DP-Flames, a novel differentially private FKGE with private selection, which offers a better privacy-utility tradeoff by exploiting the entity-binding sparse gradient property of FKGE and comes with a tight privacy accountant by incorporating the state-of-the-art private selection technique. We further propose an adaptive privacy budget allocation policy to dynamically adjust defense magnitude across the training procedure. Comprehensive evaluations demonstrate that the proposed defense can successfully mitigate the privacy threat by effectively reducing the success rate of inference attacks from to on average with only a modest utility decrease.

The increasing demand for data-driven machine learning (ML) models has led to the emergence of model markets, where a broker collects personal data from data owners to produce high-usability ML models. To incentivize data owners to share their data, the broker needs to price data appropriately while protecting their privacy. Forequitable data valuation, which is crucial in data pricing,Shapley valuehas become the most prevalent technique because it satisfies all four desirable properties in fairness: balance, symmetry, zero element, and additivity. Forthe right to be forgotten, which is stipulated by many data privacy protection laws to allow data owners to unlearn their data from trained models, thesharded structurein ML model training has become a de facto standard to reduce the cost of future unlearning by avoiding retraining the entire model from scratch. In this paper, we explore how the sharded structure for the right to be forgotten affects Shapley value for equitable data valuation in model markets. To adapt Shapley value for the sharded structure, we propose S-Shapley value, a sharded structure-based Shapley value, which satisfies four desirable properties for data valuation. Since we prove that computing S-Shapley value is #P-complete, two sampling-based methods are developed to approximate S-Shapley value. Furthermore, to efficiently update valuation results after data owners unlearn their data, we present two delta-based algorithms that estimate the change of data value instead of the data value itself. Experimental results demonstrate the efficiency and effectiveness of the proposed algorithms.

Machine learning has shown promise for improving the accuracy of Android malware detection in the literature. However, it is challenging to (1) stay robust towards real-world scenarios and (2) provide interpretable explanations for experts to analyse. In this article, we proposeMsDroid, an Androidmalware detection system that makes decisions by identifyingmalicioussnippets with interpretable explanations. We mimic a common practice of security analysts, i.e., filtering APIs before looking through each method, to focus on local snippets around sensitive APIs instead of the whole program. Each snippet is represented with a graph encoding both code attributes and domain knowledge and then classified by Graph Neural Network (GNN). The local perspective helps the GNN classifier to concentrate on code highly correlated with malicious behaviors, and the information contained in graphs benefit in better understanding of the behaviors. Hence,MsDroidis more robust and interpretable in nature. To identify malicious snippets, we present a semi-supervised learning approach that only requires app labeling. The key insight is that malicious snippets only exist in malwares and appear at least once in a malware. To make malicious snippets less opaque, we design an explanation mechanism to show the importance of control flows and to retrieve similarly implemented snippets from known malwares. A comprehensive comparison with 5 baseline methods is conducted on a dataset of more than 81K apps in 3 real-world scenarios, includingzero-day,evolution, andobfuscation. The experimental results show thatMsDroidis more robust than state-of-the-art systems in all cases, with 5.37% to 49.52% advantage in F1-score. Besides, we demonstrate that the provided explanations are effective and illustrate how the explanations facilitate malware analysis.

Recently, model stealing attacks are widely studied but most of them are focused on stealing a single non-discrete model, e.g., neural networks. For ensemble models, these attacks are either non-executable or suffer from intolerant performance degradation due to the complex model structure (multiple sub-models) and the discreteness possessed by the sub-model (e.g., decision trees). To overcome the bottleneck, this paper proposes a divide-and-conquer strategy called DivTheft to formulate the model stealing attack to common ensemble models by combining active learning (AL). Specifically, based on the boosting learning concept, we divide a hard ensemble model stealing task into multiple simpler ones about single sub-model stealing. Then, we adopt AL to conquer the data-free sub-model stealing task. During the process, the current AL algorithm easily causes the stolen model to be biased because of ignoring the past useful memories. Thus, DivTheft involves a newly designed uncertainty sampling scheme to filter reusable samples from the previously used ones. Experiments show that compared with the prior work, DivTheft can save almost 50% queries while ensuring a competitive agreement rate to the victim model.

Location-based services (LBS) have been significantly developed and widely deployed in mobile devices. It is also well-known that LBS applications may result in severe privacy concerns by collecting sensitive locations. A strong privacy model ''local differential privacy'' (LDP) has been recently deployed in many different applications (e.g., Google RAPPOR, iOS, and Microsoft Telemetry) but not effective for LBS applications due to the low utility of existing LDP mechanisms. To address such deficiency, we propose the first LDP framework for a variety of location-based services (namely ''L-SRR''), which privately collects and analyzes user locations with high utility. Specifically, we design a novel randomization mechanism ''Staircase Randomized Response'' (SRR) and extend the empirical estimation to significantly boost the utility for SRR in different LBS applications (e.g., traffic density estimation, and k-nearest neighbors). We have conducted extensive experiments on four real LBS datasets by benchmarking with other LDP schemes in practical applications. The experimental results demonstrate that L-SRR significantly outperforms them.

Vertical Federated Learning (FL) is a new paradigm that enables users with non-overlapping attributes of the same data samples to jointly train a model without directly sharing the raw data. Nevertheless, recent works show that it's still not sufficient to prevent privacy leakage from the training process or the trained model. This paper focuses on studying the privacy-preserving tree boosting algorithms under the vertical FL. The existing solutions based on cryptography involve heavy computation and communication overhead and are vulnerable to inference attacks. Although the solution based on Local Differential Privacy (LDP) addresses the above problems, it leads to the low accuracy of the trained model. This paper explores to improve the accuracy of the widely deployed tree boosting algorithms satisfying differential privacy under vertical FL. Specifically, we introduce a framework called OpBoost. Three order-preserving desensitization algorithms satisfying a variant of LDP called distance-based LDP (dLDP) are designed to desensitize the training data. In particular, we optimize the dLDP definition and study efficient sampling distributions to further improve the accuracy and efficiency of the proposed algorithms. The proposed algorithms provide a trade-off between the privacy of pairs with large distance and the utility of desensitized values. Comprehensive evaluations show that OpBoost has a better performance on prediction accuracy of trained models compared with existing LDP approaches on reasonable settings. Our code is open source.

The website fingerprinting (WF) attack enables a local eavesdropper to identify which website a client is visiting under encrypted network connections. By leveraging deep neural networks, the state-of-the-art WF attacks achieve high accuracy in classic experimental scenes. However, due to the high variance of neural networks, those attacks are sensitive to the specific information in the data and would result in less-than-ideal performance on data outside the training set or on data that is impacted by the effect of concept drift. In this paper, we present snWF, a novelWF attack, which leverages an out-of-the ordinary ensemble to reduce the variance of neural networks and improve the robustness of the attack. In a large open-world setting with 400,000 websites, snWF manages to determine whether a user is visiting a monitored website, with a true positive rate of 98.1% and a false positive rate of 5.7%. We also evaluated snWF in a more realistic attack scenario, termed aswide-world, to examine whether snWF can correctly classify websites that even an adversary has not seen before, and we found that snWF achieves a higher classification accuracy than the state-of-the-art attacks in this new setting. In addition, in the face of concept drift, snWF is found to be more resilient than any other attacks. Moreover, we are the first to reveal that under concept drift WF attacks suffer more severe performance degradation in aopen-worldsetting than in aclosed-worldsetting.

Transferability of adversarial examples is of central importance for attacking an unknown model, which facilitates adversarial attacks in more practical scenarios, e.g., black-box attacks. Existing transferable attacks tend to craft adversarial examples by indiscriminately distorting features to degrade prediction accuracy in a source model without aware of intrinsic features of objects in the images. We argue that such brute-force degradation would introduce model-specific local optimum into adversarial examples, thus limiting the transferability. By contrast, we propose the Feature Importance-aware Attack (FIA), which disrupts important object-aware features that dominate model decisions consistently. More specifically, we obtain feature importance by introducing the aggregate gradient, which averages the gradients with respect to feature maps of the source model, computed on a batch of random transforms of the original clean image. The gradients will be highly correlated to objects of interest, and such correlation presents invariance across different models. Besides, the random transforms will preserve intrinsic features of objects and suppress model-specific information. Finally, the feature importance guides to search for adversarial examples to-wards disrupting critical features, achieving stronger transferability. Extensive experimental evaluation demonstrates the effectiveness and superior performance of the proposed FIA, i.e., improving the success rate by 9.5% against normally trained models and 12.8% against defense models as compared to the state-of-the-art transferable attacks. Code is available at: https://github.com/hcguoO0/FIA

Norm emergence is a process where agents in a multi-agent system establish self-enforcing conformity through repeated interactions. When such interactions are confined to a social topology, several self-reinforcing substructures (SRS) may emerge within the population. This prevents a formation of a global norm. We propose incremental social instruments (ISI) to dissolve these SRSs by creating ties between agents. Establishing ties requires some effort and cost. Hence, it is worth to design methods that build a small number of ties yet dissolve the SRSs. By using the notion of information entropy, we propose an indicator called the BA-ratio that measures the current SRSs. We find that by building ties with minimal BA-ratio, our ISI is effective in facilitating the global norm emergence. We explain this through our experiments and theoretical results. Furthermore, we propose the small-degree principle in minimising the BA-ratio that helps us to design efficient ISI algorithms for finding the optimal ties. Experiments on both synthetic and real-world network topologies demonstrate that our adaptive ISI is efficient at dissolving SRS.

Deep neural networks (DNNs) have seen extensive studies on image recognition and classification, image segmentation, and related topics. However, recent studies show that DNNs are vulnerable in defending adversarial examples. The classification network can be deceived by adding a small amount of perturbation to clean samples. There are challenges when researchers want to design a general approach to defend against a wide variety of adversarial examples. To solve this problem, we introduce a defensive method to prevent adversarial examples from generating. Instead of designing a stronger classifier, we built a more robust classification system that can be viewed as a structural black box. After adding a buffer to the classification system, attackers can be efficiently deceived. The real evaluation results of the generated adversarial examples are often contrary to what the attacker thinks. Additionally, we do not assume a specific attack method premise. This incognizance to underlying attacks demonstrates the generalizability of the buffer to potential adversarial attacks. Extensive experiments indicate that the defense method greatly improves the security performance of DNNs.

Public blockchains have emerged as a promising direction in revolutionizing existing data-driven systems relying on centralized service providers. Among others, one kind of such systems is the popular crowdsensing systems which promise convenient data collection and aggregation. Although promising, leveraging public blockchains to build crowdsensing systems is non-trivial and has to overcome several barriers. First, public blockchains are transparent and lack support for data privacy. Second, participants from the open blockchain environment may misbehave in serving crowdsensing applications, like providing invalid data or doing aggregation incorrectly. Further, on-chain processing incurs monetary cost, so simply putting all workload on-chain is highly uneconomical and a delicate joint on-chain and off-chain design is required. In this paper, we take the first research attempt and explore a new design point to bridge public blockchains with crowdsensing systems. We propose a framework for building private, robust, and verifiable blockchain-empowered crowdsensing systems. It features an open service paradigm where blockchain nodes can rent out their computing resources to serve crowdsensing applications, with custom and full-fledged mechanisms to foster a healthy and economical ecosystem and to simultaneously tackle the challenges of data privacy, robustness against misbehaving participants, and service correctness assurance. Extensive experiments demonstrate our designs practicality.

Verifiable Searchable Symmetric Encryption, as an important cloud security technique, allows users to retrieve the encrypted data from the cloud through keywords and verify the validity of the returned results. Dynamic update for cloud data is one of the most common and fundamental requirements for data owners in such schemes. To the best of our knowledge, the existing verifiable SSE schemes supporting data dynamic update are all based on asymmetric-key cryptography verification, which involves time-consuming operations. The overhead of verification may become a significant burden due to the sheer amount of cloud data. Therefore, how to achieve keyword search over dynamic encrypted cloud data with efficient verification is a critical unsolved problem. To address this problem, we explore achieving keyword search over dynamic encrypted cloud data with symmetric-key based verification and propose a practical scheme in this paper. In order to support the efficient verification of dynamic data, we design a novel Accumulative Authentication Tag (AAT) based on the symmetric-key cryptography to generate an authentication tag for each keyword. Benefiting from the accumulation property of our designed AAT, the authentication tag can be conveniently updated when dynamic operations on cloud data occur. In order to achieve efficient data update, we design a new secure index composed by a search table ST based on the orthogonal list and a verification list VL containing AATs. Owing to the connectivity and the flexibility of ST, the update efficiency can be significantly improved. The security analysis and the performance evaluation results show that the proposed scheme is secure and efficient.

Cross-media retrieval (CMR) is an attractive networked application where a server responds to queries with retrieval results of different modalities. Different from traditional information retrieval, CMR relies on a more enriched set of machine learning techniques to produce semantic models projecting multimodal data into a common space. A larger training dataset usually gives more accurate models, leading to a better retrieval result. Despite very promising with potential underpinnings in network analytics and multimedia applications, applying CMR in such contexts also faces severe privacy challenges, due to the fact that various data scattering among multiple parties may be sensitive and not allowed to be shared publicly. Studies jointly considering cross-media analytics, privacy protection, collaborative learning, and distributed networking contexts, are relatively sparse. In this work, we propose the first practical system for privacy-preserving cross-media retrieval by utilizing trusted processors. Our scheme enables secure aggregation of the data from distinct parties, and secure canonical correlation analysis (CCA) over collaborated data to obtain semantic models. Verification mechanisms are designed to defend against active attacks from a malicious adversary. Furthermore, to deal with large data sets, we provide a set of optimization methods to accomodate to limited trusted memory and improve the efficiency of training process in CMR. We consider issues such as data block splitting to manage memory overhead, ordering of operations as well as parameters reuse and release to simplify I/O, and parallel computation to speed up dual operations. Our experiments over both synthetic and real datasets show that our solution is very efficient in practice, outperforms the existing solutions, and performs comparably with the original CMR system.

Blockchain applications that rely on the Proof-of-Work (PoW) have increasingly become energy inefficient with a staggering carbon footprint. In contrast, energy efficient alternative consensus protocols such as Proof-of-Stake (PoS) may cause centralization and unfairness in the blockchain system. To address these challenges, we propose a modular version of PoS-based blockchain systems called e-PoS that resists the centralization of network resources by extending mining opportunities to a wider set of stakeholders. Moreover, e-PoS leverages the in-built system operations to promote fair mining practices by penalizing malicious entities. We validate e-PoS 's achievable objectives through theoretical analysis and simulations. Our results show that e-PoS ensures fairness and decentralization, and can be applied to existing blockchain applications.

In this paper, we raise practical situations in profiling based power analysis when profiling and target devices are quite different at several levels. "Crossed devices" are newly termed, including homogeneous and heterogeneous devices, which have not been carefully investigated. We identify such device variations and take a further step towards leveraging the deep learning based power analysis. Traditional template attacks and straight-forward deep learning based power analysis will fail, when the gap across devices is significantly enlarged. In this paper, we propose a noval frequency and learning based power analysis machanism, which is able to explore new attacking power of deep learning and address challenges caused by device variations. For the first time, power traces collected from our own PIC devices can be utilized to successfully attack the public dataset in DPAContest v4 which is based on a totally different AVR microcontroller.

Nowadays, crowd sensing becomes increasingly more popular due to the ubiquitous usage of mobile devices. However, the quality of such human-generated sensory data varies significantly among different users. To better utilize sensory data, the problem of truth discovery, whose goal is to estimate user quality and infer reliable aggregated results through quality-aware data aggregation, has emerged as a hot topic. Although the existing truth discovery approaches can provide reliable aggregated results, they fail to protect the private information of individual users. Moreover, crowd sensing systems typically involve a large number of participants, making encryption or secure multi-party computation based solutions difficult to deploy. To address these challenges, in this paper, we propose an efficient privacy-preserving truth discovery mechanism with theoretical guarantees of both utility and privacy. The key idea of the proposed mechanism is to perturb data from each user independently and then conduct weighted aggregation among users’ perturbed data. The proposed approach is able to assign user weights based on information quality, and thus the aggregated results will not deviate much from the true results even when large noise is added. We adapt local differential privacy definition to this privacy-preserving task and demonstrate the proposed mechanism can satisfy local differential privacy while preserving high aggregation accuracy. We formally quantify utility and privacy trade-off and further verify the claim by experiments on both synthetic data and a real-world crowd sensing system.

In this demonstration, we show that audio signals emitted by a smartphone speaker can be captured by the accelerometer on the same smartphone, and accelerometers on recently released smartphones can cover most of the fundamental frequency band of adult speech. Based on these pivotal observations, we present AccelEve, a new side channel attack that allows smartphone applications to eavesdrop on the smartphone speaker without the requirement of sensitive system permissions. Through analyzing the accelerometer measurements of a smartphone, AccelEve is able to: 1) recognize the speech information (text) carried by the acceleration signal; 2) reconstruct the audio signal played by the smartphone speaker. This demo will present experimental validations for our observations and the proposed system.

Autonomous vehicles (AVs) have promised to drastically improve the convenience of driving by releasing the burden of drivers and reducing traffic accidents with more precise control. With the fast development of artificial intelligence and significant advancements of the Internet of Things technologies, we have witnessed the steady progress of autonomous driving over the recent years. As promising as it is, the march of autonomous driving technologies also faces new challenges, among which security is the top concern. In this article, we give a systematic study on the security threats surrounding autonomous driving, from the angles of perception, navigation, and control. In addition to the in-depth overview of these threats, we also summarize the corresponding defense strategies. Furthermore, we discuss future research directions about the new security threats, especially those related to deep-learning-based self-driving vehicles. By providing the security guidelines at this early stage, we aim to promote new techniques and designs related to AVs from both academia and industry and boost the development of secure autonomous driving.

Cruising for a vacant and economical parking spot causes not only time-consuming and frustrating driving experiences, but fuel waste and air pollution. Public parking spots in crowded cities are scarce and expensive. On the contrary, private parking spots usually have low utilization rates, and the spot suppliers are willing to provide their extra parking resources due to a maintenance cost by charging parking fees. Given this situation, it is imperative to call for a smart parking system that collects and provides private parking spots (e.g., around home or workplace) to ease public parking concerns. However, when the suppliers (drivers) are providing (querying for) parking spots, their privacy (e.g., location, identity) is inevitable to be disclosed and existing parking schemes cannot achieve anonymous authentication and anonymous payment simultaneously. To tackle these problems, we propose an anonymous smart-parking and payment (ASAP) scheme in vehicular networks. Specifically, we use short randomizable signature to provide anonymity and conditional privacy. We achieve quick result matching with hashmap and anonymous payment with E-cash. Security analysis and experimental results show that ASAP can protect privacy in a conditional way and has low computational costs and communication overhead.

Finding min-cut is a fundamental operation in graph theory. It has been widely used in many applications such as image segmentation and network partition. However, solving the min-cut problem is time-consuming for resource-constrained devices, especially when the graph is large and dense. In this paper, we explore how to securely solve the min-cut problem in the cloud environment, and propose two secure outsourcing schemes for the min-cut of undirected edge-weighted graphs. The first scheme is based on two non-colluded untrusted cloud servers, and the second one is under the single untrusted cloud server model. In the designed schemes, we develop a new technique that contains merging vertices and edges, inserting fake vertices and edges, shuffling vertices and randomizing weights of edges to protect the privacy of graphs. In order to realize the checkability of result from a single untrusted cloud server, we design a novel verification mechanism by outsourcing the min-cut of two related graphs. Besides, we also provide security analysis and the experimental evaluation. To the best of our knowledge, it is the first research on secure outsourcing of graph algorithm.

Location-based service (LBS) in vehicular ad hoc networks (VANETs) has significantly benefited information acquisition from geographically-based social networking. Authentication guarantees the unforgeability and the effectiveness of the LBS information. Unfortunately, owing to a large quantity of redundant or useless LBS messages disseminated in VANETs, the heavy authentication overhead of the existing work adopting a periodically released authentication key, filtering with message identifiers or exploiting public key (fully) homomorphic encryption (FHE), is either intolerable by resource-constrained on-board units (OBUs) or inappropriate to the realtime controlling requirement for VANETs. In this paper, an efficient multi-key secure outsourced computation scheme MSOC without exploiting public key FHE is first proposed, in the setting of two non-colluding servers, namely the cloud and the cryptographic service provider (CSP). Then, based on MSOC, an efficient and secure comparison protocol LSCP is devised, without the interaction between the server and the users. Furthermore, a lightweight privacy-preserving authentication protocol LPPA for LBS in VANETs is proposed, by eliminating duplicate and useless encrypted LBS messages before authentication is executed, through a newly devised efficient privacy-preserving information filtering system. Both user’s location privacy and interest privacy are well protected against even the collusion between the roadside units (RSUs) serving as the cloud (or CSP) and malicious users. Especially, the property of ciphertext re-encryption of our proposed MSOC also guarantees the interest pattern privacy whether two users accept the same LBS information. Finally, formal security proof and extensive simulation results verify the effectiveness and practicability of our proposed LPPA.

Many real-world social networks are decentralized in nature, and the only way to analyze such a network is to collect local views of the social graph from individual participants. Since local views may contain sensitive information, it is often desirable to apply differential privacy in the data collection process, which provides strong and rigorous privacy guarantees. In many practical situations, the local view of a participant contains not only her own connections, but also those of her neighbors, which are private and sensitive for the neighbors, but not directly so for the participant herself. We call such information beyond direct connections an extended local view (ELV)</>, and study two fundamental problems related to ELVs: first, how do we correctly enforce differential privacy for all participants in the presence of ELVs? Second, how can the data collector utilize ELVs to obtain accurate estimates of global graph properties?

It has been discovered that every photo carries an unique hardware fingerprint of the photographing digital camera. This camera fingerprint is remarkably effective in image-to-camera matching and has been applied in a wide variety of beneficial forensic tasks such as copyright protection and integrity validation. However, the fingerprints carried by the images can also be utilized for malicious purposes. An adversary can launch identity linking attack, which re-identifies the anonymous social network accounts, through exploiting the digital cameras' fingerprints that are carried by the posted images. Moreover, the adversary can easily frame an innocent victim or bypass camera-based smartphone authentication systems by launching identity forgery attacks, i.e., fabricating unoriginal fingerprints onto images. Currently, no effective counter measures against such attacks have been proposed yet. To solve this problem, in this paper, we first evaluate the effectiveness of the attacks in the current image sharing practices. We then propose CFP, an intermediary between smartphone users and image sharing platforms that conceals the camera fingerprint of the photographing device. Instead of removing the camera fingerprint from the image of interest, our system protects user privacy through obfuscating the camera fingerprint with a specially designed random perturbation component. With such design, the proposed system is enabled to prevent malicious utilizations of camera fingerprint while preserving the beneficial applications. Extensive experiments are conducted to demonstrate the effectiveness and efficiency of the CFP system on various social platforms. Using the CFP obfuscated images, the True Positive Rate of identity linking attacks is reduced by around 85%. For identity forgery attacks, our system enables an effective detection mechanism that could achieve 100% detection rate.

The emerging hardware-assisted security technologies facilitate the deployment of secure and trustworthy applications in today's cloud computing infrastructure. Despite promising, the advantages appear to diminish due to limited resources of trusted execution environments and ever-increasing workload to be processed inside. Different from existing task-specific and system-level optimizations, our key observation is that those redundant computations occur commonly among several applications when handling the same input data. In light of this, we propose SPEED, a secure and generic computation deduplication system in the context of Intel SGX. It allows SGX-enabled applications to identify redundant computations and reuse computation results, while protecting the confidentiality and integrity of code, inputs, and results. To maximize the benefit of computation deduplication, we design a cross-application deduplication scheme, empowering multiple applications to securely utilize the shared results as long as they perform identical computations. To ease the use of SPEED, we implement a fully functional prototype and provide a concise and expressive API for developers to deduplicate rich computations with minimal effort, as few as 2 lines of code per function call. Extensive evaluations of four popular applications demonstrate that SPEED improves performance by up to 400 times. The source code is available on GitHub for public use.

With the proliferation of the smart Internet of Things (IoT) devices based on Android system, malicious Android applications targeting for IoT devices have received more and more attention due to the concern of privacy leakage and property loss. However, existing malware detection approaches based on static or dynamic analysis are not scalable to the evolvement of malware and cannot extract enough valid semantics in application programming interface (API) level, failing to detect new malware. In this paper, we propose EveDroid, a scalable and event-aware Android malware detection system, which exploits the behavioral patterns in different events to effectively detect new malware based on the insight that events can reflect apps' possible running activities. Unlike existing approaches using API calls as features directly, we propose to use event group to describe apps' behaviors in event level, which can capture higher level of semantics than in API level. In event group, we adopt function clusters to represent behaviors in each event so that behaviors hidden in events can still be captured as time goes on, which enables EveDroid to detect new malware in the event level. The function clusters can generalize API calls into vectors based on their API composition to capture new API calls, which makes EveDroid scalable to malware evolving. Moreover, a neural network is specifically designed to aggregate the multiple events and automatically mine the semantic relationship among them. We train the system and evaluate its F1-measure on a dataset of 14 956 benign and 28 848 malicious Android apps released in different years. The experimental results show that EveDroid outperforms other malware detection systems.

Deep neural networks have been widely applied in various machine learning applications for mobile data analytics in cloud. However, this approach introduces significant data challenges, because the cloud operator can perform deep inferences on the available data. Recent advances in edge computing have paved the way to more efficient and private data processing at the edge of the network for simple tasks and lightweight models, but challenges still remain in building efficient complex models (e.g., deep learning) for edge computing. To tackle these issues, we propose EdgeSanitizer, a deep inference framework-based edge computing with local differential privacy for mobile data analytics. EdgeSanitizer leverages deep learning model to conduct data minimization and obfuscates the learned features by adaptively injecting noise, thereby forming a new protection layer against sensitive inference. We evaluate its performance in terms of data privacy and utility through theoretical analysis and experimental evaluation. The theoretical analysis proves that EdgeSanitizer can provide provable privacy guarantees with a large improvement in utility. And the experimental results demonstrate the robustness of our approach against sensitive inference, as well as its applicability on resource-constrained edge devices.

The photo response non-uniformity of a smartphone camera is a reliable hardware fingerprint that can be used to authenticate the smartphone owner. This type of camera-based authentication is convenient and of low cost since it requires only pictures taken by a smartphone. However, as shown in this paper, existing camera-based authentication systems are either impractical or subject to fingerprint forgery attacks, in which an adversary intentionally submits forged pictures with fake fingerprints. We propose Camera in Motion (CIM), a practical and reliable camera-based smartphone authentication system. In CIM, a user is asked to move his/her smartphone along a specific route, take pictures of QR codes displayed on the verifier's interface in burst mode, and submit particular burst pictures to the verifier for authentication. We find that, because burst images are captured in rapid succession, the random noise components of a captured image can be partially preserved across multiple images that are captured in a row. The preserved noise forms a forgery-sensitive noisechain embedded in burst images. We also find that there exists various correlations between the movement of the camera and the noise components of the captured images. The noisechain and these correlations are then explored for forgery detection. We performed extensive experiments with 22 smartphones of 5 different models. Our experiment results show that CIM can achieve 100% true acceptance rate at 0% false acceptance rate in both fingerprint matching and forgery detection.

By learning generative models of semantic-rich data distributions from samples, generative adversarial network (GAN) has recently attracted intensive research interests due to its excellent empirical performance as a generative model. The model is used to estimate the underlying distribution of a dataset and randomly generate realistic samples according to their estimated distribution. However, GANs can easily remember training samples due to the high model complexity of deep networks. When GANs are applied to private or sensitive data, the concentration of distribution may divulge some critical information. It consequently requires new technological advances to mitigate the information leakage under GANs. To address this issue, we propose GANobfuscator, a differentially private GAN, which can achieve differential privacy under GANs by adding carefully designed noise to gradients during the learning procedure. With GANobfuscator, analysts are able to generate an unlimited amount of synthetic data for arbitrary analysis tasks without disclosing the privacy of training data. Moreover, we theoretically prove that GANobfuscator can provide strict privacy guarantee with differential privacy. In addition, we develop a gradient-pruning strategy for GANobfuscator to improve the scalability and stability of data training. Through extensive experimental evaluation on benchmark datasets, we demonstrate that GANobfuscator can produce high-quality generated data and retain desirable utility under practical privacy budgets.

The recent proliferation of human-carried mobile devices has given rise to the crowd sensing systems. However, the sensory data provided by individual participants are usually not reliable. To better utilize such sensory data, the topic of truth discovery, whose goal is to estimate user quality and infer reliable aggregated results through quality-aware data aggregation, has drawn significant attention. Though able to improve aggregation accuracy, existing truth discovery approaches fail to address the privacy concerns of individual users. In this article, we propose a novel privacy-preserving truth discovery (PPTD) framework, which can protect not only users’ sensory data but also their reliability scores derived by the truth discovery approaches. The key idea of the proposed framework is to perform weighted aggregation on users’ encrypted data using a homomorphic cryptosystem, which can guarantee both high accuracy and strong privacy protection. In order to deal with large-scale data, we also propose to parallelize PPTD with MapReduce framework. Additionally, we design an incremental PPTD scheme for the scenarios where the sensory data are collected in a streaming manner. Extensive experiments based on two real-world crowd sensing systems demonstrate that the proposed framework can generate accurate aggregated results while protecting users’ private information.

Crowdsourcing is a new problem-solving paradigm for tasks that are difficult for computers but easy for humans. Since the answers collected from the recruited participants (workers) may contain sensitive information, crowdsourcing raises serious privacy concerns. In this paper, we investigate the problem of protecting user privacy under local differential privacy (LDP), where individual workers randomize their answers independently and send the perturbed answers to the task requester. The utility goal is to ensure high accuracy of the inferred true answers (i.e., truth) from the perturbed data. One of the challenges of LDP perturbation is the sparsity of worker answers (i.e., each worker only answers a small number of tasks). Simple extension of existing approaches (e.g., Laplace perturbation and randomized response) may incur large errors in truth inference on sparse data. Thus we design a new matrix factorization (MF) algorithm under LDP that addresses the trade-off between privacy and utility (i.e., accuracy of truth inference). We prove that our MF algorithm can provide both LDP guarantee and small error of truth inference, regardless of the sparsity of worker answers. We perform extensive experiments on real-world and synthetic datasets and demonstrate that the MF algorithm performs better than the existing LDP algorithms on sparse crowdsourcing data.

Magnetic Resonance Imaging (MRI) is a widely used technique to help form images of internal body structures for medical diagnosis. Recently, the Simultaneous Auto-calibrating and K-space Estimation (SAKE) becomes one of the most popular rapid imaging reconstruction methods to restore key information from scanned MRI data. This technique intrinsically requires a fair amount of high-resolution MRI data to accommodate the need of accurate diagnosis and imposes vast computational overhead onto resource-constrained clinics. To solve this problem, the practitioners start seeking the help of cloud computing platform to utilize its robust and economical computation power. However, the privacy concerns with outsourcing patients' private data to public cloud servers are ignited and hinder those practitioners from enjoying the benefits of cloud computing

Soliciting answers from online users is an efficient and effective solution to many challenging tasks. Due to the variety in the quality of users, it is important to infer their ability to provide correct answers during aggregation. Therefore, truth discovery methods can be used to automatically capture the user quality and aggregate user-contributed answers via a weighted combination. Despite the fact that truth discovery is an effective tool for answer aggregation, existing work falls short of the protection towards the privacy of participating users. To fill this gap, we propose perturbation-based mechanisms that provide users with privacy guarantees and maintain the accuracy of aggregated answers. We first present a one-layer mechanism, in which all the users adopt the same probability to perturb their answers. Aggregation is then conducted on perturbed answers but the aggregation accuracy could drop accordingly. To improve the utility, a two-layer mechanism is proposed where users are allowed to sample their own probabilities from a hyper distribution. We theoretically compare the one-layer and two-layer mechanisms, and prove that they provide the same privacy guarantee while the two-layer mechanism delivers better utility. This advantage is brought by the fact that the two-layer mechanism can utilize the estimated user quality information from truth discovery to reduce the accuracy loss caused by perturbation, which is confirmed by experimental results on real-world datasets. Experimental results also demonstrate the effectiveness of the proposed two-layer mechanism in privacy protection with tolerable accuracy loss in aggregation.

Content-based image retrieval (CBIR) applications have been rapidly developed along with the increase in the quantity, availability and importance of images in our daily life. However, the wide deployment of CBIR scheme has been limited by its the severe computation and storage requirement. In this paper, we propose a privacy-preserving content-based image retrieval scheme, which allows the data owner to outsource the image database and CBIR service to the cloud, without revealing the actual content of the database to the cloud server. Local features are utilized to represent the images, and earth mover's distance (EMD) is employed to evaluate the similarity of images. The EMD computation is essentially a linear programming (LP) problem. The proposed scheme transforms the EMD problem in such a way that the cloud server can solve it without learning the sensitive information. In addition, local sensitive hash (LSH) is utilized to improve the search efficiency. The security analysis and experiments show the security and efficiency of the proposed scheme.

Nowadays gigantic crowd-sourced data from mobile devices have become widely available in social networks, enabling the possibility of many important data mining applications to improve the quality of our daily lives. While providing tremendous benefits, the release of crowd-sourced social network data to the public will pose considerable threats to mobile users' privacy. In this paper, we investigate the problem of real-time spatio-temporal data publishing in social networks with privacy preservation. Specifically, we consider continuous publication of population statistics and design RescueDP-an online aggregate monitoring framework over infinite streams with w-event privacy guarantee. Its key components including adaptive sampling, adaptive budget allocation, dynamic grouping, perturbation and filtering, are seamlessly integrated as a whole to provide privacy-preserving statistics publishing on infinite time stamps. Moreover, we further propose an enhanced RescueDP with neural networks to accurately predict the values of statistics and improve the utility of released data. Both RescueDP and the enhanced RescueDP are proved satisfying w-event privacy. We evaluate the proposed schemes with real-world as well as synthetic datasets and compare them with two w-event privacy-assured representative methods. Experimental results show that the proposed schemes outperform the existing methods and improve the utility of real-time data sharing with strong privacy guarantee.

A large amount of valuable information resides in decentralized social graphs, where no entity has access to the complete graph structure. Instead, each user maintains locally a limited view of the graph. For example, in a phone network, each user keeps a contact list locally in her phone, and does not have access to other users' contacts. The contact lists of all users form an implicit social graph that could be very useful to study the interaction patterns among different populations. However, due to privacy concerns, one could not simply collect the unfettered local views from users and reconstruct a decentralized social network.

Millions of the smart meters, as essential components, are being deployed ubiquitously in the next generation power system. However, the public privacy concerns over the users' power consumption leakage raise, since the smart meters' unintermittent readings contain customers' behavior patterns. To alleviate this problem, the state-of-the-art techniques are common to use a rechargeable battery to hide the actual power consumption. Unfortunately, none of the existing works completely provide a rigorous privacy protection with reasonable cost under real-world battery settings, i.e., achieving the well-known differential privacy guarantee economically using batteries with limited charge/discharge rate and capacity. To attain this goal, this paper proposes a differentially private meter reading report mechanism. The main idea is to first narrow down the domain of the noise distribution parameter, in order to decrease the possibility of violating the battery limits. It also combines a multi-armed bandit algorithm to further reduce the cost as much as possible. In addition, a novel switch mechanism is proposed to prevent the meter from reporting its reading when the battery limitations might be violated. The theoretical analysis provides a formal proof of the privacy guarantee of the proposed scheme. Besides, experimental results show that the privacy protection of the proposed scheme is at least nine times stronger than that of the existing solutions with acceptable extra cost.

Releasing representative data sets without compromising the data privacy has attracted increasing attention from the database community in recent years. Differential privacy is an influential privacy framework for data mining and data release without revealing sensitive information. However, existing solutions using differential privacy cannot effectively handle the release of high-dimensional data due to the increasing perturbation errors and computation complexity. To address the deficiency of existing solutions, we propose DPPro, a differentially private algorithm for high-dimensional data release via random projection to maximize utility while guaranteeing privacy. We theoretically prove that DPPro can generate synthetic data set with the similar squared Euclidean distance between high-dimensional vectors while achieving (ϵ, δ)-differential privacy. Based on the theoretical analysis, we observed that the utility guarantees of released data depend on the projection dimension and the variance of the noise. Extensive experimental results demonstrate that DPPro substantially outperforms several state-of-the-art solutions in terms of perturbation error and privacy budget on high-dimensional data sets.

In local differential privacy (LDP), each user perturbs her data locally before sending the noisy data to a data collector. The latter then analyzes the data to obtain useful statistics. Unlike the setting of centralized differential privacy, in LDP the data collector never gains access to the exact values of sensitive data, which protects not only the privacy of data contributors but also the collector itself against the risk of potential data leakage. Existing LDP solutions in the literature are mostly limited to the case that each user possesses a tuple of numeric or categorical values, and the data collector computes basic statistics such as counts or mean values. To the best of our knowledge, no existing work tackles more complex data mining tasks such as heavy hitter discovery over set-valued data. In this paper, we present a systematic study of heavy hitter mining under LDP. We first review existing solutions, extend them to the heavy hitter estimation, and explain why their effectiveness is limited. We then propose LDPMiner, a two-phase mechanism for obtaining accurate heavy hitters with LDP. The main idea is to first gather a candidate set of heavy hitters using a portion of the privacy budget, and focus the remaining budget on refining the candidate set in a second phase, which is much more efficient budget-wise than obtaining the heavy hitters directly from the whole dataset. We provide both in-depth theoretical analysis and extensive experiments to compare LDPMiner against adaptations of previous solutions. The results show that LDPMiner significantly improves over existing methods. More importantly, LDPMiner successfully identifies the majority true heavy hitters in practical settings.

Nowadays gigantic crowd-sourced data collected from mobile phone users have become widely available, which enables the possibility of many important data mining applications to improve the quality of our daily lives. While providing tremendous benefits, the release of these data to the public will pose a considerable threat to mobile users' privacy. To solve this problem, the notion of differential privacy has been proposed to provide privacy with theoretical guarantee, and recently it has been applied in streaming data publishing. However, most of the existing literature focus on either event-level privacy on infinite streams or user-level privacy on finite streams. In this paper, we investigate the problem of real-time spatiotemporal crowd-sourced data publishing with privacy preservation. Specifically, we consider continuous publication of population statistics for monitoring purposes and design RescueDP-an online aggregate monitoring scheme over infinite streams with privacy guarantee. RescueDP's key components include adaptive sampling, adaptive budget allocation, dynamic grouping, perturbation and filtering, which are seamlessly integrated as a whole to provide privacy-preserving statistics publishing on infinite time stamps. We show that RescueDP can achieve w-event privacy over data generated and published periodically by crowd users. We evaluate our scheme with real-world as well as synthetic datasets and compare it with two w-event privacy-assured representative benchmarks. Experimental results show that our solution outperforms the existing methods and improves the utility with strong privacy guarantee.

With the increasing importance of images in people's daily life, content-based image retrieval (CBIR) has been widely studied. Compared with text documents, images consume much more storage space. Hence, its maintenance is considered to be a typical example for cloud storage outsourcing. For privacy-preserving purposes, sensitive images, such as medical and personal images, need to be encrypted before outsourcing, which makes the CBIR technologies in plaintext domain to be unusable. In this paper, we propose a scheme that supports CBIR over encrypted images without leaking the sensitive information to the cloud server. First, feature vectors are extracted to represent the corresponding images. After that, the pre-filter tables are constructed by locality-sensitive hashing to increase search efficiency. Moreover, the feature vectors are protected by the secure kNN algorithm, and image pixels are encrypted by a standard stream cipher. In addition, considering the case that the authorized query users may illegally copy and distribute the retrieved images to someone unauthorized, we propose a watermark-based protocol to deter such illegal distributions. In our watermark-based protocol, a unique watermark is directly embedded into the encrypted images by the cloud server before images are sent to the query user. Hence, when image copy is found, the unlawful query user who distributed the image can be traced by the watermark extraction. The security analysis and the experiments show the security and efficiency of the proposed scheme.

Advances in cloud computing have greatly motivated data owners to outsource their huge amount of personal multimedia data and/or computationally expensive tasks onto the cloud by leveraging its abundant resources for cost saving and flexibility. Despite the tremendous benefits, the outsourced multimedia data and its originated applications may reveal the data owner's private information, such as the personal identity, locations, or even financial profiles. This observation has recently aroused new research interest on privacy-preserving computations over outsourced multimedia data. In this paper, we propose an effective and practical privacy-preserving computation outsourcing protocol for the prevailing scale-invariant feature transform (SIFT) over massive encrypted image data. We first show that the previous solutions to this problem have either efficiency/security or practicality issues, and none can well preserve the important characteristics of the original SIFT in terms of distinctiveness and robustness. We then present a new scheme design that achieves efficiency and security requirements simultaneously with the preservation of its key characteristics, by randomly splitting the original image data, designing two novel efficient protocols for secure multiplication and comparison, and carefully distributing the feature extraction computations onto two independent cloud servers. We both carefully analyze and extensively evaluate the security and effectiveness of our design. The results show that our solution is practically secure, outperforms the state-of-the-art, and performs comparably with the original SIFT in terms of various characteristics, including rotation invariance, image scale invariance, robust matching across affine distortion, and addition of noise and change in 3D viewpoint and illumination.

Nowadays, cloud has become a promising multimedia data processing and sharing platform. Many institutes and companies plan to outsource and share their large-scale video and image datasets on cloud for scientific research and public interest. Among various video applications, the discovery of frequent visual patterns over graphical data is an exploratory and important technique. However, the privacy concerns over the leakage of sensitive information contained in the videos/images impedes the further implementation. Although the frequent visual patterns mining (FVPM) algorithm aggregates summary over individual frames and seems not to pose privacy threat, the private information contained in individual frames still may be leaked from the statistical result. In this paper, we study the problem of privacy-preserving publishing of graphical data FVPM on cloud. We propose the first differentially private frequent visual patterns mining algorithm for graphical data, named DPcode. We propose a novel mechanism that integrates the privacy-preserving visual word conversion with the differentially private mechanism under the noise allocation strategy of the sparse vector technique. The optimized algorithms properly allocate the privacy budgets among different phases in FPM algorithm over images and reduce the corresponding data distortion. Extensive experiments are conducted based on datasets commonly used in visual mining algorithms. The results show that our approach achieves high utility while satisfying a practical privacy requirement.

The image and multimedia data produced by individuals and enterprises is increasing every day. Motivated by the advances in cloud computing, there is a growing need to outsource such computational intensive image feature detection tasks to cloud for its economic computing resources and on-demand ubiquitous access. However, the concerns over the effective protection of private image and multimedia data when outsourcing it to cloud platform become the major barrier that impedes the further implementation of cloud computing techniques over massive amount of image and multimedia data. To address this fundamental challenge, we study the state-of-the-art image feature detection algorithms and focus on Scalar Invariant Feature Transform (SIFT), which is one of the most important local feature detection algorithms and has been broadly employed in different areas, including object recognition, image matching, robotic mapping, and so on. We analyze and model the privacy requirements in outsourcing SIFT computation and propose Secure Scalar Invariant Feature Transform (SecSIFT), a high-performance privacy-preserving SIFT feature detection system. In contrast to previous works, the proposed design is not restricted by the efficiency limitations of current homomorphic encryption scheme. In our design, we decompose and distribute the computation procedures of the original SIFT algorithm to a set of independent, co-operative cloud servers and keep the outsourced computation procedures as simple as possible to avoid utilizing a computationally expensive homomorphic encryption scheme. The proposed SecSIFT enables implementation with practical computation and communication complexity. Extensive experimental results demonstrate that SecSIFT performs comparably to original SIFT on image benchmarks while capable of preserving the privacy in an efficient way.

The recent proliferation of human-carried mobile devices has given rise to the crowd sensing systems. However, the sensory data provided by individual participants are usually not reliable. To identify truthful values from the crowd sensing data, the topic of truth discovery, whose goal is to estimate user quality and infer truths through quality-aware data aggregation, has drawn significant attention. Though able to improve aggregation accuracy, existing truth discovery approaches fail to take into consideration an important issue in their design, i.e., the protection of individual users' private information. In this paper, we propose a novel cloud-enabled privacy-preserving truth discovery (PPTD) framework for crowd sensing systems, which can achieve the protection of not only users' sensory data but also their reliability scores derived by the truth discovery approaches. The key idea of the proposed framework is to perform weighted aggregation on users' encrypted data using homomorphic cryptosystem. In order to deal with large-scale data, we also propose to parallelize PPTD with MapReduce framework. Through extensive experiments on not only synthetic data but also real world crowd sensing systems, we justify the guarantee of strong privacy and high accuracy of our proposed framework.

The amount and availability of user-contributed image data have been dramatically increased during the past ten years. Popular multimedia social networks, e.g. Flicker, commonly utilize user image data to construct user behavior models, social preferences, etc., for the purpose of effective advertisement, better user retention and attraction, and many others. Existing practices of data utilization, however, seriously deteriorate users' personal privacy and have led to increasing criticisms and legislation pressures. In this paper, we aim to construct a privacy-preserving feature detection scheme over encrypted image data. The proposed system enables an interested party to perform a variety of image feature detection tasks, including visual descriptors in MPEG-7 standard, while protecting user privacy relating to image contents. We implement a prototype system based on somewhat homomorphic encryption scheme and the benchmark Caltech256 database. The experimental results show that our system can guarantee effective image feature detection without sacrificing user privacy.

As the image data produced by individuals and enterprises is rapidly increasing, Scalar Invariant Feature Transform (SIFT), as a local feature detection algorithm, has been heavily employed in various areas, including object recognition, robotic mapping, etc. In this context, there is a growing need to outsource such image computation with high complexity to cloud for its economic computing resources and on-demand ubiquitous access. However, how to protect the private image data while enabling image computation becomes a major concern. To address this fundamental challenge, we study the privacy requirements in outsourcing SIFT computation and propose SecSIFT, a high performance privacy-preserving SIFT feature detection system. In previous private image computation works, one common approach is to encrypt the private image in a public key based homomorphic scheme that enables the original processing algorithms designed for plaintext domain to be performed over ciphertext domain. In contrast to these works, our system is not restricted by the efficiency limitations of homomorphic encryption scheme. The proposed system distributes the computation procedures of SIFT to a set of independent, co-operative cloud servers, and keeps the outsourced computation procedures as simple as possible to avoid utilizing homomorphic encryption scheme. Thus, it enables implementation with practical computation and communication complexity. Extensive experimental results demonstrate that SecSIFT performs comparably to original SIFT on image benchmarks while capable of preserving the privacy in an efficient way.

Short-range wireless communication technologies have been used in many security-sensitive smartphone applications and services such as contactless micro payment and device pairing. Typically, the data confidentiality of the existing short-range communication systems relies on so-called “key-exchange then encryption” mechanism, which is inefficient, especially for short communication sessions. In this work, we present${\ssb{PriWhisper}}$—a keyless secure acoustic short-range communication system for smartphones. It is designed to provide a software-based solution to secure smartphone communication without the key agreement phase.${\ssb{PriWhisper}}$adopts the emerging friendly jamming technique from radio communication for data confidentiality. The system prototype is implemented and evaluated on several Android smartphone platforms for efficiency and usability. We theoretically and experimentally analyze the security of our proposed acoustic communication system against eavesdropping. In particular, we study the (in)separability of the data signal and jamming signal against blind signal segmentation (BSS) attacks such as independent component analysis (ICA). The result shows that${\ssb{PriWhisper}}$provides sufficient security guarantees for commercial smartphone applications and yet strong compatibilities with most legacy smartphone platforms. As an application, we also develop${\ssb{AcousAuth}}$—a novel smartphone-empowered system for personal authentication.