Local Differential Privacy (LDP) enables massive data collection and analysis while protecting end users' privacy against untrusted aggregators. It has been applied to various data types (e.g., categorical, numerical, and graph data) and application settings (e.g., static and streaming). Recent findings indicate that LDP protocols can be easily disrupted by poisoning or manipulation attacks, where an attacker can leverage injected/corrupted fake users to send crafted data to the aggregator in order to manipulate the final estimate of the aggregator. However, current attacks primarily target static protocols, neglecting the security of LDP protocols in the streaming settings. Our research fills the gap by developing novel fine-grained manipulation attacks to LDP protocols for data streams. By reviewing the attack surfaces in existing algorithms, we introduce a unified attack framework with composable modules, which can manipulate the LDP estimated stream toward a target stream. Our attack framework can adapt to state-of-the-art streaming LDP algorithms with different analytic tasks (e.g., frequency and mean) and LDP models (event-level, user-level, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$w$</tex-math></inline-formula>-event level). We verify our attacks theoretically and validate them through extensive experiments on real-world datasets. Finally, we explore a possible defense mechanism for mitigating our attacks.

Ensuring the security of AI models is becoming increasingly important as AI systems are integrated into various products and services. In the future, many AI models may undergo security evaluations at testing centers to obtain certifications that validate their security and reliability. However, due to concerns about trade secrets, companies may only provide black-box access to their models during testing. This necessitates the development of black-box model-level backdoor detection methods, since evaluation is limited to the model itself, without access to its internal structure or training data.In this work, we propose BProm, a novel black-box model-level backdoor detection method based on visual prompting (VP). VP is a technique that adapts well-trained frozen models for source domain tasks to target domain tasks by mapping class subspaces between the source and target domains. We identify a critical misalignment, termed class subspace inconsistency, between clean and poisoned datasets. Using this insight, BProm detects backdoors by observing the low classification accuracy of prompted models when backdoors are present. Extensive experiments demonstrate the effectiveness of BProm in identifying backdoored models under black-box settings, paving the way for secure and reliable certification of AI models.

Visual State Space Models (VSSM) have shown remarkable performance in various computer vision tasks. However, backdoor attacks pose significant security challenges, causing compromised models to predict target labels when specific triggers are present while maintaining normal behavior on benign samples. In this paper, we investigate the robustness of VSSMs against backdoor attacks. Specifically, we delicately design a novel framework for VSSMs, dubbed BadVim, which utilizes low-rank perturbations on state-wise to uncover their impact on state transitions during training. By poisoning only 0.3% of the training data, our attacks cause any trigger-embedded input to be misclassified to the targeted class with a high attack success rate (over 97%) at inference time. Our findings suggest that the state-space representation property of VSSMs, which enhances model capability, may also contribute to its vulnerability to backdoor attacks. Our attack exhibits effectiveness across three datasets, even bypassing state-of-the-art defenses against such attacks. Extensive experiments show that the backdoor robustness of VSSMs is comparable to that of Transformers (ViTs) and superior to that of Convolutional Neural Networks (CNNs). We believe our findings will prompt the community to reconsider the trade-offs between performance and robustness in model design.

Differentially private (DP) synthetic data has become the de facto standard for releasing sensitive data. However, many DP generative models suffer from the low utility of synthetic data, especially for high-resolution images. On the other hand, one of the emerging techniques in parameter efficient fine-tuning (PEFT) is visual prompting (VP), which allows well-trained existing models to be reused for the purpose of adapting to subsequent downstream tasks. In this work, we explore such a phenomenon in constructing captivating generative models with DP constraints. We show that VP in conjunction with DP-NTK, a DP generator that exploits the power of the neural tangent kernel (NTK) in training DP generative models, achieves a significant performance boost, particularly for high-resolution image datasets, with accuracy improving from 0.644±0.044 to 0.769. Lastly, we perform ablation studies on the effect of different parameters that influence the overall performance of VP-NTK. Our work demonstrates a promising step forward in improving the utility of DP synthetic data, particularly for high-resolution images.

The integration of Differential Privacy (DP) with diffusion models (DMs) presents a promising yet challenging frontier, particularly due to the substantial memorization capabilities of DMs that pose significant privacy risks. Differential privacy offers a rigorous framework for safeguarding individual data points during model training, with Differential Privacy Stochastic Gradient Descent (DP-SGD) being a prominent implementation. Diffusion method decomposes image generation into iterative steps, theoretically aligning well with DP's incremental noise addition. Despite the natural fit, the unique architecture of DMs necessitates tailored approaches to effectively balance privacy-utility trade-off. Recent developments in this field have highlighted the potential for generating high-quality synthetic data by pre-training on public data (i.e., ImageNet) and fine-tuning on private data, however, there is a pronounced gap in research on optimizing the trade-offs involved in DP settings, particularly concerning parameter efficiency and model scalability. Our work addresses this by proposing a parameter-efficient fine-tuning strategy optimized for private diffusion models, which minimizes the number of trainable parameters to enhance the privacy-utility trade-off. We empirically demonstrate that our method achieves state-of-the-art performance in DP synthesis, significantly surpassing previous benchmarks on widely studied datasets (e.g., with only 0.47M trainable parameters, achieving a more than 35% improvement over the previous state-of-the-art with a small privacy budget on the CelebA-64 dataset). Anonymous codes available at https://anonymous.4open.science/r/DP-LORA-F02F.

Large Language Models (LLMs) are increasingly adopted in high-stakes scenarios, yet their safety mechanisms often remain fragile. Simple jailbreak prompts or even benign fine-tuning can bypass internal safeguards, underscoring the need to understand the failure modes of current safety strategies. Recent findings suggest that vulnerabilities emerge when alignment is confined to only the initial output tokens. To address this, we introduce the notion of safety depth, a designated output position where the model refuses to generate harmful content. While deeper alignment appears promising, identifying the optimal safety depth remains an open and underexplored challenge. We leverage the equivalence between autoregressive language models and Markov chains to derive the first theoretical result on identifying the optimal safety depth. To reach this safety depth effectively, we propose a cyclic group augmentation strategy that improves safety scores across six LLMs. In addition, we uncover a critical interaction between safety depth and ensemble width, demonstrating that larger ensembles can offset shallower alignments. These results suggest that test-time computation, often overlooked in safety alignment, can play a key role. Our approach provides actionable insights for building safer LLMs.

Semi-supervised learning (SSL) has achieved remarkable performance with a small fraction of labeled data by leveraging vast amounts of unlabeled data from the Internet. However, this large pool of untrusted data is extremely vulnerable to data poisoning, leading to potential backdoor attacks. Current backdoor defenses are not yet effective against such a vulnerability in SSL. In this study, we propose a novel method, Unlabeled Data Purification (UPure), to disrupt the association between trigger patterns and target classes by introducing perturbations in the frequency domain. By leveraging the Rate-Distortion-Perception (RDP) trade-off, we further identify the frequency band, where the perturbations are added, and Justify this selection. Notably, UPure purifies poisoned unlabeled data without the need of extra clean labeled data. Extensive experiments on four benchmark datasets and five SSL algorithms demonstrate that UPure effectively reduces the attack success rate from 99.78% to 0% while maintaining model accuracy. Code is available here: https://github.com/chengyi-chris/UPure.

In the realm of subject-driven text-to-image (T2I) generative models, recent developments like DreamBooth and BLIP-Diffusion have led to impressive results yet encounter limitations due to their intensive fine-tuning demands and substantial parameter requirements. While the low-rank adaptation (LoRA) module within DreamBooth offers a reduction in trainable parameters, it introduces a pronounced sensitivity to hyperparameters, leading to a compromise between parameter efficiency and the quality of T2I personalized image synthesis. Addressing these constraints, we introduce DiffuseKronA, a novel Kronecker product-based adaptation module that not only significantly reduces the parameter count by 35 % and 99.947 % compared to LoRADreamBooth and the original DreamBooth, respectively, but also enhances the quality of image synthesis. Crucially, DiffuseKronA mitigates the issue of hyperparameter sensitivity, delivering consistent high-quality generations across a wide range of hyperparameters, thereby diminishing the necessity for extensive fine-tuning. Furthermore, a more controllable decomposition makes DiffuseKronA more interpretable and can even achieve up to a 50 % reduction with results compa-rable to LoRA-Dreambooth. Evaluated against diverse and complex input images and text prompts, DiffuseKronA consistently outperforms existing low-rank models, producing diverse images of higher quality with improved fidelity and a more accurate color distribution of objects, all the while upholding exceptional parameter efficiency, thus presenting a substantial advancement in the field of T2I generative modeling.

Differentially private synthetic data is a promising alternative for sensitive data release. Many differentially private generative models have been proposed in the literature. Unfortunately, they all suffer from the low utility of the synthetic data, especially for high resolution images. Here, we propose differentially private aggregation in forward phase (DPAF), an effective differentially private generative model for high-dimensional image synthesis. Unlike previous private stochastic gradient descent-based methods, which add the Gaussian noise in the backward phase during model training, DPAF adds differentially private feature aggregation in the forward phase, which brings advantages, such as reducing information loss in gradient clipping and low sensitivity to aggregation. Since an inappropriate batch size has a negative impact on the utility of synthetic data, DPAF also addresses the problem of setting an appropriate batch size by proposing a novel training strategy that asymmetrically trains different parts of the discriminator. We extensively evaluate different methods on multiple image datasets (up to images of$128\times 128$resolution) to demonstrate the performance of DPAF.

The development of artificial intelligence has led to the rapid growth of various industries. In addition to driving innovative applications, it has also brought significant changes to daily life. This transformation has accelerated the evolution of digital finance, enabling financial services to integrate AI and develop new applications. Among these services, smart investment has become one of the most popular. Due to the uncertainty of financial markets, achieving stable profits remains a challenging task. In this article, we propose a novel trading strategy that uses Search Economy Algorithms to analyze stock market trends. Our approach does not focus on a single stock; instead, it enables industry-wide investment decisions through our stock selection mechanism and fitness function, combined with the trading strategies provided by the Search Economy Algorithms. Compared with existing methodologies, our experimental environment is more complex. We benchmark our approach against ETF-0050 and demonstrate its effectiveness by evaluating capital growth rates and risk under specific parameters.

Deep neural networks (DNNs) are known to be vulnerable to backdoor attacks. Specifically, the attacker endeavors to implant backdoors in the DNN model by injecting a set of poisoning samples such that the malicious model predicts target labels once the backdoor is triggered. The clean-image attack has recently emerged as a threat in multi-label classification, where an attacker is able to poison training labels without tampering with image contents. In this paper, we propose a simple but effective method to alleviate clean-image backdoor attacks. Considering the difference in weight convergence between the benign model and backdoor model, our method relies on partial weight initialization and fine-tuning to mitigate the backdoor behaviors of a suspicious model. The fine-tuned model sustains its clean accuracy through knowledge distillation over a few iterations. Importantly, our approach does not require extra clean images for purification. Extensive experiments demonstrate the effectiveness of our defenses against clean-image attacks for multi-label classifications across two benchmark datasets.

Dataset distillation offers a potential means to enhance data efficiency in deep learning. Recent studies have shown its ability to counteract backdoor risks present in original training samples. In this study, we delve into the theoretical aspects of backdoor attacks and dataset distillation based on kernel methods. We introduce two new theory-driven trigger pattern generation methods specialized for dataset distillation. Following a comprehensive set of analyses and experiments, we show that our optimization-based trigger design framework informs effective backdoor attacks on dataset distillation. Notably, datasets poisoned by our designed trigger prove resilient against conventional backdoor attack detection and mitigation methods. Our empirical results validate that the triggers developed using our approaches are proficient at executing resilient backdoor attacks.

Diffusion models for text-to-image (T2I) synthesis, such as Stable Diffusion (SD), have recently demonstrated exceptional capabilities for generating high-quality content. However, this progress has raised several concerns of potential misuse, particularly in creating copyrighted, prohibited, and restricted content, or NSFW (not safe for work) images. While efforts have been made to mitigate such problems, either by implementing a safety filter at the evaluation stage or by fine-tuning models to eliminate undesirable concepts or styles, the effectiveness of these safety measures in dealing with a wide range of prompts remains largely unexplored. In this work, we aim to investigate these safety mechanisms by proposing one novel concept retrieval algorithm for evaluation. We introduce Ring-A-Bell, a model-agnostic red-teaming scheme for T2I diffusion models, where the whole evaluation can be prepared in advance without prior knowledge of the target model. Specifically, Ring-A-Bell first performs concept extraction to obtain holistic representations for sensitive and inappropriate concepts. Subsequently, by leveraging the extracted concept, Ring-A-Bell automatically identifies problematic prompts for diffusion models with the corresponding generation of inappropriate content, allowing the user to assess the reliability of deployed safety mechanisms. Finally, we empirically validate our method by testing online services such as Midjourney and various methods of concept removal. Our results show that Ring-A-Bell, by manipulating safe prompting benchmarks, can transform prompts that were originally regarded as safe to evade existing safety mechanisms, thus revealing the defects of the so-called safety mechanisms which could practically lead to the generation of harmful contents. In essence, Ring-A-Bell could serve as a red-teaming tool to understand the limitations of deployed safety mechanisms and to explore the risk under plausible attacks. Our codes are available at https://github.com/chiayi-hsu/Ring-A-Bell.

Backdoor attacks are a significant concern in deep learning, especially in applications where models are trained on data from untrusted sources. Plenty of approaches use latent representations of a backdoor model to separate trigger samples from clean ones. However, these defenses rely on some clean data to train a classifier. Recently, researchers have designed adaptive attacks that are latently inseparable, making it even harder for the defender to prevent backdoor attacks. For these reasons, we propose a novel defense, Higher Moment Disparity (HMD), based on the higher moment inspired by latent statistics. HMD uses no clean data and all intermediate representations to avoid previous concerns. Extensive experiments show that our defense against various attacks is promising.

While large language models (LLMs) such as Llama-2 or GPT-4 have shown impressive zero-shot performance, fine-tuning is still necessary to enhance their performance for customized datasets, domain-specific tasks, or other private needs. However, fine-tuning all parameters of LLMs requires significant hardware resources, which can be impractical for typical users. Therefore, parameter-efficient fine-tuning such as LoRA have emerged, allowing users to fine-tune LLMs without the need for considerable computing resources, with little performance degradation compared to fine-tuning all parameters. Unfortunately, recent studies indicate that fine-tuning can increase the risk to the safety of LLMs, even when data does not contain malicious content. To address this challenge, we propose $\textsf{Safe LoRA}$, a simple one-liner patch to the original LoRA implementation by introducing the projection of LoRA weights from selected layers to the safety-aligned subspace, effectively reducing the safety risks in LLM fine-tuning while maintaining utility. It is worth noting that $\textsf{Safe LoRA}$ is a training-free and data-free approach, as it only requires the knowledge of the weights from the base and aligned LLMs. Our extensive experiments demonstrate that when fine-tuning on purely malicious data, $\textsf{Safe LoRA}$ retains similar safety performance as the original aligned model. Moreover, when the fine-tuning dataset contains a mixture of both benign and malicious data, $\textsf{Safe LoRA}$ mitigates the negative effect made by malicious data while preserving performance on downstream tasks. Our codes are available at https://github.com/IBM/SafeLoRA.

From gimmicky IoT devices to self-driving cars, cyber-physical systems have become increasingly accessible to the masses. As these systems interact intimately with the physical world, failures in the systems can lead to severe, potentially life-threatening damage. Classical cyber-physical systems, such as aircraft flight control, employ dedicated redundancies for fault tolerance. However, a more flexible and cost-effective approach to redundancy is needed as cyber-physical systems become more versatile and expand to the consumer market. In this work, we explore the synthesis of redundancies for program functionalities with neural network models. The models are trained with the data from normal program executions and will be deployed to supplant the original program functionalities when failures occur. We have tested the prototype on representative cyber-physical systems, including ArduPilot and OpenPilot. The evaluation results indicate that the approach can synthesize redundancy models for both numerical and logical programs. We also demonstrate that the redundancy models can effectively avoid bugs and security vulnerabilities in the original programs.

Traffic sign recognition plays a crucial role in self-driving cars, but unfortunately, it is vulnerable to adversarial patches (AP). Although AP can efficiently fool DNN-based models in previous studies, the connection between image forensics and AP detection still needs to be explored. From a high-level point of view, their goals are the same. That is to find tampered regions and prevent false positives in the meantime. A natural question arises: "Is achieving application-agnostic anomaly detection possible?" In this paper, we propose Image Forensics Defense Against Adversarial Patch (IDAP), a framework to defend against adversarial patches via generalizable features learned from tampered images. In addition, we incorporate the Hausdorff erosion loss into our network model for joint training to complete the shape of a predicted mask. Extensive experimental comparisons on three datasets, including COCO, DFG, and APRICOT demonstrate that IDAP outperforms state-of-the-art AP detection methods.

Recently, quantum classifiers have been known to be vulnerable to adversarial attacks, where quantum classifiers are fooled by imperceptible noises to have misclassification. In this paper, we propose one first theoretical study that utilizing the added quantum random rotation noise can improve the robustness of quantum classifiers against adversarial attacks. We connect the definition of differential privacy and demonstrate the quantum classifier trained with the natural presence of additive noise is differentially private. Lastly, we derive a certified robustness bound to enable quantum classifiers to defend against adversarial examples supported by experimental results.

Visual Prompting (VP) is an emerging and powerful technique that allows sample-efficient adaptation to downstream tasks by engineering a well-trained frozen source model. In this work, we explore the benefits of VP in constructing compelling neural network classifiers with differential privacy (DP). We explore and integrate VP into canonical DP training methods and demonstrate its simplicity and efficiency. In particular, we discover that VP in tandem with PATE, a state-of-the-art DP training method that leverages the knowledge transfer from an ensemble of teachers, achieves the state-of-the-art privacy-utility tradeoff with minimum expenditure of privacy budget. Moreover, we conduct additional experiments on cross-domain image classification with a sufficient domain gap to further unveil the advantage of VP in DP. Lastly, we also conduct extensive ablation studies to validate the effectiveness and contribution of VP under DP consideration. Our code is available at https://github.com/EzzzLi/Prompt-PATE.

Client-side data deduplication enables cloud storage services to reduce storage space and bandwidth consumption, resulting in reduced operating cost and a high level of user satisfaction. However, duplicate checks (i.e., the corresponding message exchange) generate a side channel, exposing the file existence status to attackers. In particular, the binary response from a duplicate check reveals file existence information. This can be exploited to launch further attacks, such as learning sensitive file content and establishing a covert channel. As current solutions provide only weaker privacy or rely on unreasonable assumptions, we propose random response (RARE) to achieve stronger privacy. The underlying principle is that the uploading user simultaneously sends a duplication request for two chunks. The cloud receiving the request returns a carefully designed randomized duplication response to preserve the deduplication gain and minimize privacy leakage. By proposing multiple chunk uploading and chunk rearrangement strategies, we optimize the design of RARE to significantly reduce the communication burden without compromising privacy. We also study the impact of different implementations of multiple chunk uploading on the deduplication gain, further reducing communication cost. The analytical results confirm that formal privacy is ensured, whereas experiment results demonstrate that RARE preserves both privacy and the deduplication benefit.

Due to the data-driven intelligence from the recent deep learning based approaches, the huge amount of data collected from various kinds of sensors from industrial devices have the potential to revolutionize the current technologies used in the industry. To improve the efficiency and quality of machines, the machine manufacturer needs to acquire the history of the machine operation process. However, due to the business secrecy, the factories are not willing to do so. One promising solution to the abovementioned difficulty is the synthetic dataset and an informatic network structure, both through deep generative models such as differentially private generative adversarial networks. Hence, this article initiates the study of the utility difference between the abovementioned two kinds. We carry out an empirical study and find that the classifier generated by private informatic network structure is more accurate than the classifier generated by private synthetic data, with approximately 0.31–7.66%.

Adversarial examples causing evasive predictions are widely used to evaluate and improve the robustness of machine learning models. However, current studies focus on supervised learning tasks, relying on the ground truth data label, a targeted objective, or supervision from a trained classifier. In this paper, we propose a framework of generating adversarial examples for unsupervised models and demonstrate novel applications to data augmentation. Our framework exploits a mutual information neural estimator as an information theoretic similarity measure to generate adversarial examples without supervision. We propose a new MinMax algorithm with provable convergence guarantees for the efficient generation of unsupervised adversarial examples. Our framework can also be extended to supervised adversarial examples. When using unsupervised adversarial examples as a simple plugin data augmentation tool for model retraining, significant improvements are consistently observed across different unsupervised tasks and datasets, including data reconstruction, representation learning, and contrastive learning. Our results show novel methods and considerable advantages in studying and improving unsupervised machine learning via adversarial examples.

Despite an increased demand for valuable data, the privacy concerns associated with sensitive datasets present a barrier to data sharing. One may use differentially private generative models to generate synthetic data. Unfortunately, generators are typically restricted to generating images of low-resolutions due to the limitation of noisy gradients. Here, we propose DPGEN, a network model designed to synthesize high-resolution natural images while satisfying differential privacy. In particular, we propose an energy-guided network trained on sanitized data to indicate the direction of the true data distribution via Langevin Markov chain Monte Carlo (MCMC) sampling method. In contrast to the state-of-the-art methods that can process only low-resolution images (e.g., MNIST and Fashion-MNIST), DPGEN can generate differentially private synthetic images with resolutions up to$128\times 128$with superior visual quality and data utility. Our code is available at https://github.com/chiamuyu/DPGEN

Deep neural networks have developed rapidly and have achieved out-standing performance in several tasks, such as image classification and natural language processing. However, recent studies have indicated that both digital and physical adversarial examples can fool neural networks. Face-recognition systems are used in various applications that involve security threats from physical adversarial examples. Herein, we propose a physical adversarial attack with the use of full-face makeup. The presence of makeup on the human face is a reasonable possibility, which possibly increases the imperceptibility of attacks. In our attack framework, we combine the cycle-adversarial generative network (cycle-GAN) and a victimized classifier. The Cycle-GAN is used to generate adversarial makeup, and the architecture of the victimized classifier is VGG 16. Our experimental results show that our attack can effectively overcome manual errors in makeup application, such as color and position-related errors. We also demonstrate that the approaches used to train the models can influence physical attacks; the adversarial perturbations crafted from the pre-trained model are affected by the corresponding training data.

Abstract Intelligent reflecting surface (IRS) composed of a large number of low‐cost, phase‐adjustable passive reflecting elements, which is an attractive solution of overcoming the signal attenuation and interference. IRS can be used as a low‐power passive system with signal enhancement and interference suppression. However, the planning problem of a large‐scale network contains a great numbers of base stations (BSs) and IRSs is challenging. The network planning problem considers limited deployment cost and signal enhancement at the same time is vital. Here, the 6G wireless signal coverage (WSC) problem with the consideration of both BS and IRS is considered. More specifically, the 6G WSC problem is first formulated through the integer linear programming. Due to its obvious NP‐hardness, two efficient heuristic algorithms that can reach a near‐optimal solution, that is, the WSC algorithm and the tree‐based WSC (TBWSC) algorithm is then proposed. Simulation‐based results showed that the proposed WSC algorithm achieves lower total cost than that of the TBWSC algorithm but also results in lower signal quality. The proposed TBWSC algorithm obtains the planning result with the highest signal quality and slightly higher total cost. Both proposed algorithms can find better planning results of multiple BSs and IRSs for 6G wireless communications.

The use of differentially private synthetic data has been adopted as a common security measure for the public release of sensitive data. However, the existing solutions either suffer from serious privacy budget splitting or fail to fully automate the generation procedures. In this study, we propose an automated system for synthesizing differentially private synthetic tabular data, calledDPView. Our key insight is that high-dimensional data synthesis can be accomplished by utilizing the domain sizes of attributes, which are public information, whereas identifying the correlation among attributes is necessary but leads to severe privacy budget splitting. In addition, we analytically optimize both the privacy budget allocation and consistency procedures of the proposed method through mathematical programming. We further propose two novel methods, including iterative non-negativity and consistency-aware normalization, to postprocess the synthetic data. An extensive set of experimental results demonstrates the superior utility ofDPView.

With the growing use of camera devices, the industry has many image datasets that provide more opportunities for collaboration between the machine learning community and industry. However, the sensitive information in the datasets discourages data owners from releasing these datasets. Despite recent research devoted to removing sensitive information from images, they provide neither meaningful privacy-utility trade-off nor provable privacy guarantees. In this study, with the consideration of the perceptual similarity, we propose perceptual indistinguishability (PI) as a formal privacy notion particularly for images. We also propose PI-Net, a privacy-preserving mechanism that achieves image obfuscation with PI guarantee. Our study shows that PI-Net achieves significantly better privacy utility trade-off through public image data.

Adversarial robustness has become an emerging challenge for neural network owing to its over-sensitivity to small input perturbations. While being critical, we argue that solving this singular issue alone fails to provide a comprehensive robustness assessment. Even worse, the conclusions drawn from singular robustness may give a false sense of overall model robustness. Specifically, our findings show that adversarially trained models that are robust to input perturbations are still (or even more) vulnerable to weight perturbations when compared to standard models. In this paper, we formalize the notion of non-singular adversarial robustness for neural networks through the lens of joint perturbations to data inputs as well as model weights. To our best knowledge, this study is the first work considering simultaneous input-weight adversarial perturbations. Based on a multi-layer feed-forward neural network model with ReLU activation functions and standard classification loss, we establish error analysis for quantifying the loss sensitivity subject to ℓ∞-norm bounded perturbations on data inputs and model weights. Based on the error analysis, we propose novel regularization functions for robust training and demonstrate improved non-singular robustness against joint input-weight adversarial perturbations.

Studying the sensitivity of weight perturbation in neural networks and its impacts on model performance, including generalization and robustness, is an active research topic due to its implications on a wide range of machine learning tasks such as model compression, generalization gap assessment, and adversarial attacks. In this paper, we provide the first integral study and analysis for feed-forward neural networks in terms of the robustness in pairwise class margin and its generalization behavior under weight perturbation. We further design a new theory-driven loss function for training generalizable and robust neural networks against weight perturbations. Empirical experiments are conducted to validate our theoretical analysis. Our results offer fundamental insights for characterizing the generalization and robustness of neural networks against weight perturbations.

In Internet-of-Things (IoT)-driven smart-world systems, real-time crowdsourced databases from multiple distributed servers can be aggregated to extract dynamic statistics from a larger population, thus providing more reliable knowledge for our society. Particularly, multiple distributed servers in a decentralized network can realize real-time collaborative statistical estimation by disseminating statistics from their separate databases. Despite no raw data sharing, the real-time statistics could still expose the data privacy of crowdsourcing participants. For mitigating the privacy concern, while the traditional differential privacy (DP) mechanism can be simply implemented to perturb the statistics in each timestamp and independently for each dimension, this may suffer a great utility loss from the real-time and multidimensional crowdsourced data. Also, the real-time broadcasting would bring significant overheads in the whole network. To tackle the issues, we propose a novel privacy preserving and communication-efficient decentralized statistical estimation algorithm (DPCrowd), which only requires intermittently sharing the DP protected parameters with one-hop neighbors by exploiting the temporal correlations in real-time crowdsourced data. Then, with further consideration of spatial correlations, we develop an enhanced algorithm, DPCrowd+, to deal with multidimensional infinite crowd-data streams. Extensive experiments on several data sets demonstrate that our proposed schemes DPCrowd and DPCrowd+ can significantly outperform existing schemes in providing accurate and consensus estimation with rigorous privacy protection and great communication efficiency.

No abstract available.

Cloud storage services enable individuals and organizations to outsource data storage to remote servers. Cloud storage providers generally adopt data deduplication, a technique for eliminating redundant data by keeping only a single copy of a file, thus saving a considerable amount of storage and bandwidth. However, an attacker can abuse deduplication protocols to steal information. For example, an attacker can perform the duplicate check to verify whether a file (e.g., a pay slip, with a specific name and salary amount) is already stored (by someone else), hence breaching the user privacy. In this paper, we propose ZEUS (zero-knowledge deduplication response) framework. We develop ZEUS and ZEUS+, two privacy-aware deduplication protocols: ZEUS provides weaker privacy guarantees while being more efficient in the communication cost, while ZEUSþ guarantees stronger privacy properties, at an increased communication cost. To the best of our knowledge, ZEUS is the first solution which addresses two-side privacy by neither using any extra hardware nor depending on heuristically chosen parameters used by the existing solutions, thus reducing both cost and complexity of the cloud storage. In summary, through the evaluation on real datasets and comparison to existing solutions, our proposed framework demonstrates its capability of eliminating data deduplication-based side channel and at the same time keeping the deduplication benefits.

In modern information systems different information features, about the same individual, are often collected and managed by autonomous data collection services that may have different privacy policies. Answering many end-users' legitimate queries requires the integration of data from multiple such services. However, data integration is often hindered by the lack of a trusted entity, often called a mediator, with which the services can share their data and delegate the enforcement of their privacy policies. In this article, we propose a flexible privacy-preserving data integration approach for answering data integration queries without the need for a trusted mediator. In our approach, services are allowed to enforce their privacy policies locally. The mediator is considered to be untrusted, and only has access to encrypted information to allow it to link data subjects across the different services. Services, by virtue of a new privacy requirement, dubbed k-Protection, limiting privacy leaks, cannot infer information about the data held by each other. End-users, in turn, have access to privacy-sanitized data only. We evaluated our approach using an example and a real dataset from the healthcare application domain. The results are promising from both the privacy preservation and the performance perspectives.

Deep learning is well-known for its great performances on images classification, object detection, and natural language processing. However, the recent research has demonstrated that visually indistinguishable images called adversarial examples can successfully fool neural networks by carefully crafting. In this paper, we design a detector named MID, calculating mutual information to characterize adversarial subspaces. Meanwhile, we use the defense framework called MagNet and mount the detector MID on it. Experimental results show that projected gradient descent (PGD), basic iterative method (BIM), Carlini and Wanger's attack (C&W attack) and elastic-net attack to deep neural network (elastic-net and L1 rules) can be effectively defended by our method.

The Internet of things (IoT) is a distributed, networked system composed of many embedded sensor devices. Unfortunately, these devices are resource constrained and susceptible to malicious data-integrity attacks and failures, leading to unreliability and sometimes to major failure of parts of the entire system. Intrusion detection and failure handling are essential requirements for IoT security. Nevertheless, as far as we know, the area of data-integrity detection for IoT has yet to receive much attention. Most previous intrusion-detection methods proposed for IoT, particularly for wireless sensor networks (WSNs), focus only on specific types of network attacks. Moreover, these approaches usually rely on using precise values to specify abnormality thresholds. However, sensor readings are often imprecise and crisp threshold values are inappropriate. To guarantee a lightweight, dependable monitoring system, we propose a novel hierarchical framework for detecting abnormal nodes in WSNs. The proposed approach uses fuzzy logic in event-condition-action (ECA) rule-based WSNs to detect malicious nodes, while also considering failed nodes. The spatiotemporal semantics of heterogeneous sensor readings are considered in the decision process to distinguish malicious data from other anomalies. Following our experiments with the proposed framework, we stress the significance of considering the sensor correlations to achieve detection accuracy, which has been neglected in previous studies. Our experiments using real-world sensor data demonstrate that our approach can provide high detection accuracy with low false-alarm rates. We also show that our approach performs well when compared to two well-known classification algorithms.

Cloning is a very serious threat in the Internet of Things (IoT), owing to the simplicity for an attacker to gather configuration and authentication credentials from a non-tamper-proof node, and replicate it in the network. In this paper, we propose MDSClone, a novel clone detection method based on multidimensional scaling (MDS). MDSClone appears to be very well suited to IoT scenarios, as it: 1) detects clones without the need to know the geographical positions of nodes; 2) unlike prior methods, it can be applied to hybrid networks that comprise both static and mobile nodes, for which no mobility pattern may be assumed a priori. Moreover, a further advantage of MDSClone is that 3) the core part of the detection algorithm can be parallelized, resulting in an acceleration of the whole detection mechanism. Our thorough analytical and experimental evaluations demonstrate that MDSClone can achieve a 100% clone detection probability. Moreover, we propose several modifications to the original MDS calculation, which lead to over a 75% speed up in large scale scenarios. The demonstrated efficiency of MDSClone proves that it is a promising method towards a practical clone detection design in IoT.

High-dimensional crowdsourced data collected from numerous users produces rich knowledge about our society; however, it also brings unprecedented privacy threats to the participants. Local differential privacy (LDP), a variant of differential privacy, is recently proposed as a state-of-the-art privacy notion. Unfortunately, achieving LDP on high-dimensional crowdsourced data publication raises great challenges in terms of both computational efficiency and data utility. To this end, based on the expectation maximization (EM) algorithm and Lasso regression, we first propose efficient multi-dimensional joint distribution estimation algorithms with LDP. Then, we develop a local differentially private high-dimensional data publication algorithm (LoPub) by taking advantage of our distribution estimation techniques. In particular, correlations among multiple attributes are identified to reduce the dimensionality of crowdsourced data, thus speeding up the distribution learning process and achieving high data utility. Extensive experiments on real-world datasets demonstrate that our multivariate distribution estimation scheme significantly outperforms existing estimation schemes in terms of both communication overhead and estimation speed. Moreover, LoPub can keep, on average, 80% and 60% accuracy over the released datasets in terms of support vector machine and random forest classification, respectively.

Differential privacy (DP) is a popular technique for protecting individual privacy and at the same for releasing data for public use. However, very few research efforts are devoted to the balance between the corresponding risk of data disclosure (RoD) and data utility. In this paper, we propose data-driven approaches for differentially private data release to evaluate RoD, and offer algorithms to evaluate whether the differentially private synthetic dataset has sufficient privacy. In addition to the privacy, the utility of the synthetic dataset is an important metric for differentially private data release. Thus, we also propose the data-driven algorithm via curve fitting to measure and predict the error of the statistical result incurred by random noise added to the original dataset. Finally, we present an algorithm for choosing appropriate privacy budget ∈ with the balance between the privacy and utility.

In this paper we present our ongoing work to build an approach to empower users of IoT-based cyber physical systems to protect their privacy by themselves. Our approach allows users to identify the privacy risks involved in sharing private data with a data consumer, assess the value of their private data based on identified risks and take a pragmatic data sharing decision balancing the risks with the benefits generated by the sharing. Our approach features a knowledgebase, called the Privacy Oracle, that exploits the power of the Semantic Web to determine how raw metadata can be combined by data consumers to infer privacy-sensitive information as well as the privacy risks associated with the disclosure of inferred information.

Many differentially private data release solutions have been proposed for different types of data with the sacrifice of inherent correlation structure. Here, we propose a unified framework of releasing differentially private data. In particular, our proposed generative adversarial network (GAN)-based framework learns the input distribution, irrespective of tabular data and graphs, and generates synthetic data in a differentially private manner. Our preliminary results show the acceptable utility of the synthetic dataset.

Differential privacy is a recent notion of data privacy protection, which does not matter even when an attacker has arbitrary background knowledge in advance. Consequently, it is viewed as a reliable protection mechanism for sensitive information. Differential privacy introduces Laplace noise to hide the true value in a dataset while preserving statistic properties. However, the large amount of Laplace noise added into a dataset is typically defined by the discursive scale parameter of the Laplace distribution. The privacy parameter ε in differential privacy is with theoretical interpretation, but the implication on the risk of data disclosure (called RoD for short) in practice has not yet been studied. Moreover, choosing appropriate value for ε is not an easy task since it impacts the level of privacy in a dataset significantly. In this paper, we define and evaluate the RoD in a dataset with either numerical or binary attributes for numerical or counting queries with multiple attributes based on the noise estimation. Through confidence probability of noise estimation, we give a simple way to choose the privacy parameter ε. Finally, we show the relation of the RoD and privacy parameter ε in experimental results. To the best of our knowledge, this is the first research work in using noise estimation to practically evaluate the RoD for multiple attributes (both numerical and binary data).

As the number of everyday objects connected to the Internet grows rapidly, securing these connected devices is a big security challenge. Key establishment in Internet of Things (IoT) becomes a challenging problem when considering the resource constrained sensor nodes. In spite of the fact that many clever solutions have been proposed, no practical and suitable scheme has emerged, especially for the extremely large amount of sensor nodes in the wireless sensor network (WSNs) in the future. In this paper, we propose a new key establishment scheme for IoT. The scheme is achieved by Kronecker product and satisfies the following conditions. 1) Substantially decreases the amount of data needs to be stored in a sensor node, 2) efficiently compute the pairwise key, 3) no communication is needed during the computation of the keys. The security evaluation is performed and we also present an in depth analysis of our scheme in terms of computation cost, communication cost and storage cost.

Data deduplication has been widely used in cloud storage to reduce the amount of storage space and save bandwidth. Unfortunately, as an increasing number of sensitive data are stored remotely, the encryption, the simplest way for data privacy, is not compatible with data deduplication. Here, we propose an encrypted deduplication scheme, XDedup, based on Merkle puzzle. To the best of our knowledge, XDedup is the first brute-force resilient encrypted deduplication with only symmetrically cryptographic two-party interactions. XDedup also achieves perfect deduplication.

Clone detection, aimed at detecting illegal copies with all of the credentials of legitimate sensor nodes, is of great importance for sensor networks because of the severe impact of clones on network operations, like routing, data collection, and key distribution. Various detection methods have been proposed, but most of them are communication-inefficient due to the common use of the witness-finding strategy. In view of the sparse characteristic of replicated nodes, we propose a novel clone detection framework, called CSI, based on a state-of-the-art signal processing technology, compressed sensing. Specifically, CSI bases its detection effectiveness on the compressed aggregation of sensor readings. Due to its consideration of data aggregation, CSI not only achieves the asymptotically lowest communication cost but also makes the network traffic evenly distributed over sensor nodes. In particular, this is achieved by exploiting the sparse property of the clones within the sensor network caused by the clone attack. The performance and security of CSI will be demonstrated by numerical simulations, analyses, and prototype implementation.

We develop two novel techniques, FHMT and HWMT, for streaming authenticated data structures to achieve the streaming verifiable computation. By leveraging the computing capability of fully homomorphic encryption, FHMT shifts almost all of the computation tasks to the server, reaching nearly no overhead for the client. HWMT strikes the performance balance between the client and server via the proposed tree decomposition technique over the Merkle tree. We also report our research attempt, SAMT, to construct a more efficient data structure for extremely resource-limited clients without the heavy computation burden on the server.

Storage nodes are expected to be placed as an intermediate tier of large scale sensor networks for caching the collected sensor readings and responding to queries with benefits of power and storage saving for ordinary sensors. Nevertheless, an important issue is that the compromised storage node may not only cause the privacy problem, but also return fake/incomplete query results. We propose a simple yet effective dummy reading-based anonymization framework, under which the query result integrity can be guaranteed by our proposed verifiable top-$k$query (VQ) schemes. Compared with existing works, the VQ schemes have a fundamentally different design philosophy and achieve the lower communication complexity at the cost of slight detection capability degradation. Analytical studies, numerical simulations, and prototype implementations are conducted to demonstrate the practicality of our proposed methods.

We deal with the challenging problem of node replication detection. Although defending against node replication attacks demands immediate attention, compared to the extensive exploration on the defense against node replication attacks in static networks, only a few solutions in mobile networks have been presented. Moreover, while most of the existing schemes in static networks rely on the witness-finding strategy, which cannot be applied to mobile networks, the velocity-exceeding strategy used in existing schemes in mobile networks incurs efficiency and security problems. Therefore, based on our devised challenge-and-response and encounter-number approaches, localized algorithms are proposed to resist node replication attacks in mobile sensor networks. The advantages of our proposed algorithms include 1) localized detection; 2) efficiency and effectiveness; 3) network-wide synchronization avoidance; and 4) network-wide revocation avoidance. Performance comparisons with known methods are provided to demonstrate the efficiency of our proposed algorithms. Prototype implementation on TelosB mote demonstrates the practicality of our proposed methods.

In this paper, we investigate resource block assignment in femtocell networks. A resource block assignment algorithm is designed to avoid co-channel intercell interference and ensure service quality for femtocell networks with dense and random femto deployments. We first formulate the optimization problem as the integer linear programming (ILP) on resource block assignment. The goal of the optimization formulation is to maximize the overall utilization of resource blocks with quality of service (QoS) constraints. We propose an efficient and simple algorithm termed interference-aware resource block assignment (IARBA). By considering conditions of resource blocks, the proposed approach achieves better resource block efficiency and assignment within QoS requirements. Our analytical and simulation results show that IARBA not only provides interference-free resource block assignment but also outperforms existing schemes in terms of average throughput with comparable complexities.

Compressive sensing (CS) has recently attracted much attention due to its unique feature of directly and simultaneously acquiring compressed and encrypted data based on their sparse or compressible properties. To securely transmit compressively sensed multimedia data over networks, it is required to support transcoder to securely convert compressed multimedia into several different types for diverse receivers. In this paper, a secure transcoding scheme for compressive multimedia sensing is proposed. We focus on securely converting compressively sensed multimedia data (not data compressed via standard codec) with a certain number of measurements into other different numbers of measurements without resorting to reconstruct the original data. We show that the security can be achieved via transforming multimedia re-sensing process into another secure domain at the transcoder. We also show that the computational security can be achieved while transmitting compressively sensed data between the sender (or each receiver) and the transcoder over networks.

The two-tier architecture consisting of a small number of resource-abundant storage nodes in the upper tier and a large number of sensors in the lower tier could be promising for large-scale sensor networks in terms of resource efficiency, network capacity, network management complexity, etc. In this architecture, each sensor having multiple sensing capabilities periodically forwards the multidimensional sensed data to the storage node, which responds to the queries, such as range query, top-kquery, and skyline query. Unfortunately, node compromises pose the great challenge of securing the data collection; the sensed data could be leaked to or could be manipulated by the compromised nodes. Furthermore, chunks of the sensed data could be dropped maliciously, resulting in an incomplete query result, which is the most difficult security breach. Here, we propose a simple yet effective hash tree-based framework, under which data confidentiality, query result authenticity, and query result completeness can be guaranteed simultaneously. In addition, the subtree sampling technique, which could be of independent interest to the other applications, is proposed to efficiently identify the compromised nodes. Last, analytical and extensive simulation studies are conducted to evaluate the performance and security of our methods. Prototype implementation on TelosB mote demonstrates the practicality of our proposed methods.

Sensor networks are vulnerable to false data injection attack and path-based denial of service (PDoS) attack. While conventional authentication schemes are insufficient for solving these security conflicts, an en-route filtering scheme, enabling each forwarding node to check the authenticity of the received message, acts as a defense against these two attacks. To construct an efficient en-route filtering scheme, this paper first presents a Constrained Function-based message Authentication (CFA) scheme, which can be thought of as a hash function directly supporting the en-route filtering functionality. Obviously, the crux of the scheme lies on the design of guaranteeing each sensor to have en-route filtering capability. Together with the redundancy property of sensor networks, which means that an event can be simultaneously observed by multiple sensor nodes, the devised CFA scheme is used to construct a CFA-based en-route filtering (CFAEF) scheme. In addition to the resilience against false data injection and PDoS attacks, CFAEF is inherently resilient against false endorsement-based DoS attack. In contrast to most of the existing methods, which rely on complicated security associations among sensor nodes, our design, which directly exploits an en-route filtering hash function, appears to be novel. We examine the CFA and CFAEF schemes from both the theoretical and numerical aspects to demonstrate their efficiency and effectiveness. Moreover, prototype implementation on TelosB mote demonstrates the practicality of our proposed method.

As a security primitive, key establishment plays the most crucial role in the design of the security mechanisms. Unfortunately, the resource limitation of sensor nodes poses a great challenge for designing an efficient and effective key establishment scheme for wireless sensor networks (WSNs). In spite of the fact that many elegant and clever solutions have been proposed, no practical key establishment scheme has emerged. In this paper, a ConstrAined Random Perturbation-based pairwise keY establishment (CARPY) scheme and its variant, a CARPY+ scheme, for WSNs, are presented. Compared to all existing schemes which satisfy only some requirements in so-called sensor-key criteria, including (1) resilience to the adversary's intervention, (2) directed and guaranteed key establishment, (3) resilience to network configurations, (4) efficiency, and (5) resilience to dynamic node deployment, the proposed CARPY+ scheme meets all requirements. In particular, to the best of our knowledge, CARPY+ is the first noninteractive key establishment scheme with great resilience to a large number of node compromises designed for WSNs. We examine the CARPY and CARPY+ schemes from both the theoretical and experimental aspects. Our schemes have also been practically implemented on the TelosB compatible mote to evaluate the corresponding performance and overhead.

The major contribution of this paper is to propose a robust en-route filtering scheme for data authentication in sensor networks without relying on unrealistic assumptions.

In this paper, a constrained random perturbation based pairwise keY establishment (CARPY) scheme and its variant, a CARPY+ scheme, for Wireless Sensor Networks (WSNs), are presented. Compared to all existing schemes which satisfy only some requirements in so-called sensor-key criteria, including: 1) resilience to the adversary's intervention, 2) directed and guaranteed key establishment, 3) resilience to network configurations, 4) efficiency, and 5) resilience to dynamic node deployment, the proposed CARPY+ scheme meets all requirements. In particular, to the best of our knowledge, CARPY+ is the first non-interactive key establishment scheme with great resilience to a large number of node compromises designed for WSNs. We examine the CARPY and CARPY+ schemes from both the theoretical and experimental aspects. Our schemes have also been practically implemented on the TelosB compatible mote to evaluate the corresponding performance and overhead.

In this paper, we study the challenging problem of node replication detection. Although defending against node replication attacks demands immediate attention, only a few solutions were proposed. In this paper, an Efficient and Distributed Detection (EDD) scheme and its variant, SEDD, are proposed to resist against node replication attacks in mobile sensor networks. The characteristics possessed by EDD and SEDD include (1) Distributed Detection; (2) Efficiency and Effectiveness; (3) Individual Detection; (4) Network-Wide Revocation Avoidance. Performance comparison with known methods are provided to demonstrate the efficiency of the EDD and SEDD schemes.

This paper presents a constrained function based message authentication (CFA) scheme for wireless sensor networks, which meets all the requirements of the so-called sensor authentication criteria, while most of the existing schemes only achieve partial requirements. In particular, to the best of our knowledge, CFA is the first authentication scheme supporting en-route filtering with only a single packet overhead. We examine the CFA scheme from both the theoretical and experimental aspects. Our method has also been practically implemented on the TelosB compatible mote for performance evaluation.

This paper presents a Constrained Random Perturbation Vector-based (CRPV) pairwise key establishment scheme and its variant, CRPV+ scheme, for wireless sensor networks (WSNs). Compared to all existing schemes which satisfy only some requirements in a so-called versatileness criteria, the CRPV+ scheme meets all requirements. In particular, the performance improvement of our schemes does not rely on tradeoffs among different requirements, but comes from the use of our constrained random vector strategy.

By launching the node replication attack, the adversary can place the replicas of captured sensor nodes back into the sensor networks in order to eavesdrop the transmitted messages or compromise the functionality of the network. Although defending against node replication attacks demands immediate attention, only a few solutions were proposed. Most of the existing distributed protocols adopt the witness finding strategy, which selects a set of sensor nodes somewhere as the witnesses, to detect the replicas. However, the energy consumption of the witness finding strategy is remarkably high and even gets worse in mobile networks. In addition, the location information is necessary for each node if the witness finding strategy is applied. In this paper, a novel protocol, called extremely Efficient Detection (XED), is proposed to resist against node replication attacks in mobile sensor networks. The advantages of XED include (1) only constant communication cost is required for replica detection; (2) the location information of sensor nodes is not required. Performance analyses and comparison with known methods are provided to demonstrate the effectiveness of our protocol.

A common way of resisting protocol attacks is to employ cryptographic techniques so that provable security can be retained. However, some desired requirements of watermarking such as blind detection and robustness are lost. This paper studies the issue of security against protocol attacks based on a mesh-based media hash-dependent image watermarking approach while maintaining the aforementioned requirements. Our main contributions include (1) media hashing instead of cryptographic hashing is used so that blind detection is still satisfied; (2) robustness against signal processing attacks is retained; (3) the difficulty of resisting ambiguity attack is derived to be equivalent to that of resisting challenging geometric attacks including cropping with larger parts discarded and rotation with larger degrees so that an acceptable trade-off between false positive and false negative can be achieved.