Recent studies reveal that adversaries can manipulate the internal knowledge of large language models (LLMs) on selected topics through model editing, causing attacker-specified harmful or biased outputs when queried about the edited content. Once such tampered LLMs are distributed, they can mislead users on the targeted topics, thereby potentially propagating misinformation or reinforcing stereotypes. However, existing knowledge manipulation attacks rely on the ability to redistribute compromised models, which is infeasible in constrained settings like Federated Instruction Tuning (FedIT), where a central server controls LLM's training and distribution. In this work, we introduce ShadeEdit, the first attack framework that leverages strengthened model editing to enable knowledge manipulation in FedIT scenarios. ShadeEdit introduces two key components to address two challenges posed by the training process of FedIT: (1) a paraphrase-based editing dataset selection strategy to mitigate the dilution from benign updates on malicious ones by constructing a high-quality editing dataset, and (2) an adaptive manipulation mechanism to evade aggregation-based defenses via an adaptive clipping strategy. ShadeEdit achieves an average 99.5% attack success rate over eight robust aggregation algorithms while preserving instruction-following accuracy, demonstrating its strong attack effectiveness and model-utility preservation.

Natural Language Generation (NLG) applications have gained immense popularity due to the utilization of powerful deep learning techniques and large training corpora. However, the increasing prevalence of NLG models also poses a significant risk of unauthorized access or theft of intellectual property (IP). To safeguard NLG models, watermarking has emerged as a promising tool, but existing watermarking techniques based on pre-processing are prone to attacker detection and can potentially harm NLG applications. This paper proposes a novel, semantic, and stealthy watermarking scheme for IP protection of NLG models. Our approach embeds a semantic combination water mark, which is generated through a multi-stage process designed to be semantic and stealthy. This scheme endows an NLG model with a verifiable preference for specific semantic combinations, which are initiated by a foundational pattern but holistically constructed to preserve model functionality. To enhance the robustness, data embedding is systematically performed through a masked location injection. Consequently, the watermark is seamlessly integrated into NLG models without misleading their original attention mechanism. Comprehensive experiments are conducted to demonstrate that the proposed scheme is highly effective and robust in protecting the IP of NLG models while remaining stealthy to potential attackers.

Vision-Language Models (VLMs) have become a cornerstone in multi-modal artificial intelligence, enabling seamless integration of visual and textual information for tasks such as image captioning, visual question answering, and cross-modal retrieval. Despite their impressive capabilities, these models often exhibit inherent vulnerabilities that can lead to safety failures in critical applications. Red-teaming is an important approach to identify and test system’s vulnerabilities, but how to conduct red-teaming for contemporary VLMs is an unexplored area. In this paper, we propose a novel multi-modal red-teaming approach, TRUST-VLM, to enhance both the attack success rate and the diversity of successful test cases for VLMs. Specifically, TRUST-VLM is built upon the in-context learning to adversarially test a VLM on both image and text inputs. Furthermore, we involve feedback from the target VLM to improve the efficiency of test case generation. Extensive experiments show that TRUST-VLM not only outperforms traditional red-teaming techniques in generating diverse and effective adversarial cases but also provides actionable insights for model improvement. These findings highlight the importance of advanced red-teaming strategies in ensuring the reliability of VLMs.

Semi-supervised Federated Learning (SSFL) is a promising approach that allows clients to collaboratively train a global model in the absence of their local data labels. The key step of SSFL is the re-labeling where each client adopts two types of available models, namely global and local models, to re-label the local data. While various technologies such as using the global model or the average of two models have been proposed to conduct the re-labeling step, little literature delves deeply into the performance dominance and limitations of the two models. In this paper, we first theoretically and empirically demonstrate that the local model achieves higher re-labeling accuracy over local data while the global model can progressively improve the re-labeling performance by introducing the extra data knowledge of other clients. Based on these findings, we propose BSemiFL which re-labels the local data through the collaboration between the local and global model in a Bayesian approach. Specifically, to re-label any given local sample, BSemiFL first uses Bayesian inference to assess the closeness of the local/global model to the sample. Then, it applies a weighted combination of their pseudo labels, using the closeness as the weights. Theoretical analysis shows that the labeling error of our method is smaller than that of simply using the global model, the local model, or their simple average. Experimental results show that BSemiFL improves the performance by up to $9.8\%$ as compared to state-of-the-art methods.

Modern Text-to-Image (T2I) models deploy multi-layered defenses to block Not-Safe-For-Work (NSFW) content generation. These defenses typically include sequential layers such as prompt filters, concept erasers and image filters. While existing adversarial attacks have demonstrated vulnerabilities in isolated defense layers, they prove largely ineffective against multi-layered defenses deployed in real-world T2I systems. In this paper, we demonstrate that exploiting overlapping vulnerabilities across these distinct defense layers enables adversaries to systematically bypass the entire safeguard of T2I systems. We propose Transstratal Adversarial Attack (TAA), a novel black-box framework to compromise T2I models with multi-layered protection. It generates transstratal adversarial prompts to evade all defense layers simultaneously. This is accomplished through transstratal adversarial candidate generation using LLMs to fulfill implicit and subjective adversarial requirements against different defense layers, combined with adversarial genetic optimization for efficient black-box search to maximize the bypass rates and generated image harmfulness. Evaluated across 14 T2I models (e.g., Stable Diffusion, DALL·E, and Midjourney) and 17 safety modules, our attack achieves an average attack success rate of 85.6\%, surpassing state-of-the-art methods by 73.5\%. Our findings challenge the isolated design of safety mechanisms and establish the first benchmark for holistic robustness evaluation in multi-layered safeguarded T2I models. The code can be found in https://github.com/Bluedask/TAA-T2I.

Pre-trained models (PTMs) are widely adopted across various downstream tasks in the machine learning supply chain. Adopting untrustworthy PTMs introduces significant security risks, where adversaries can poison the model supply chain by embedding hidden malicious behaviors (backdoors) into PTMs. However, existing backdoor attacks to PTMs can only achieve partially task-agnostic and the embedded backdoors are easily erased during the fine-tuning process. This makes it challenging for the backdoors to persist and propagate through the supply chain. In this paper, we propose a novel and severer backdoor attack, TransTroj, which enables the backdoors embedded in PTMs to efficiently transfer in the model supply chain. In particular, we first formalize this attack as an indistinguishability problem between poisoned and clean samples in the embedding space. We decompose embedding indistinguishability into pre- and post-indistinguishability, representing the similarity of the poisoned and reference embeddings before and after the attack. Then, we propose a two-stage optimization that separately optimizes triggers and victim PTMs to achieve embedding indistinguishability. We evaluate TransTroj on four PTMs and six downstream tasks. Experimental results show that our method significantly outperforms SOTA task-agnostic backdoor attacks -- achieving nearly 100% attack success rate on most downstream tasks -- and demonstrates robustness under various system settings. Our findings underscore the urgent need to secure the model supply chain against such transferable backdoor attacks. The code is available at https://github.com/haowang-cqu/TransTroj

Single-cell proteomics has advanced our understanding of cellular complexity by enabling detailed analysis of protein expression at the single-cell level. However, challenges such as data sparsity, variability, and noise require sophisticated computational solutions. SCPline addresses these by offering a comprehensive data preprocessing and analysis platform specifically for single-cell proteomics. It supports mass spectrometry-based, antibody-based, and multi-omics approaches, performing quality screening, normalization, dimensionality reduction, and clustering for each data type (https://bioinform.nefu.edu.cn/ScPline/). Each module includes tailored functions and visualizations for easy quality checks, allowing researchers with limited programming experience to efficiently preprocess data. By streamlining complex workflows, SCPline makes advanced computational tools accessible, enabling researchers to explore cellular heterogeneity and biological states, thus accelerating discoveries in developmental biology, disease pathogenesis, and therapeutic responses. Additionally, SCPline enhances reproducibility and rigor in proteomics research, contributing to breakthroughs in understanding cellular behavior and identifying novel therapeutic targets, shaping the future of biomedical research and precision medicine.

The nonintrusive load monitoring (NILM) technique, through its use of various deep neural networks (DNNs), is capable of learning residential appliances’ usage patterns from networked smart meters. However, such learned information may pose a serious privacy risk to users. In response to this privacy concern, in this article, we introduce an innovative adversarial attack. This attack can effectively restrict the NILM models’ ability to dissect power signals while maintaining accurate electricity charges for users. Given that previous adversarial attacks—which are designed for image classifiers and regressors with one-time output—cannot adequately handle NILM models and regressors with time-series output, we formally present the attack objective by leveraging the unique characteristics of regression and time-series data. Our proposed solution algorithms for this attack objective can generate imperceptible perturbations, effectively misleading the prediction of NILM models. To further ensure accurate billing calculation, we refine the attack objective to a practical version and propose a post-process that can iteratively remove the added perturbation in a certain period without compromising attack effectiveness. Experimental results on two real-world datasets, REDD and UK-DALE, demonstrate the effectiveness, transferability, and practicality of our proposed adversarial attack scheme.

Backdoor attacks pose a serious security threat to Natural Language Processing (NLP) models, allowing adversaries to manipulate model outputs through hidden triggers. Although backdoor detection methods have been developed to address this issue, existing approaches based on trigger inversion are effective only for simple, visible triggers. These methods struggle to handle semantically enhanced, invisible triggers and often fail to provide accurate backdoor determinations due to reliance on unreliable heuristics, making it difficult to reliably distinguish backdoored models from benign ones. This presents a critical gap in current detection techniques. To address these challenges, we propose a novel trigger inversionSemInvthat consists of two key contributions: consistent semantics inversion and identifiable condition inspection. Consistent semantics inversion introduces a new regularization technique into the trigger optimization process, enabling more effective inversion of semantically constrained triggers. Identifiable condition inspection assesses the attack performance margin across different identifiable conditions, providing robust evidence for distinguishing backdoored models from benign ones. We evaluateSemInvusing the TrojAI round 6–8 datasets and demonstrate that it significantly outperforms state-of-the-art approaches in both backdoor detection accuracy and trigger inversion performance. Our method also proves effective against models with stealthy triggers, advancing the field of NLP security by offering a more comprehensive solution for identifying backdoor attacks. The code repository is in https://github.com/Bluedask/SemInv.

Collaborative learning has gained significant traction for training deep learning models without sharing the original data of participants, particularly when dealing with sensitive data such as facial images. However, current gradient inversion attacks are employed to progressively reconstruct private data from gradients, and they have shown successful in extracting private training data. Nonetheless, our observations reveal that these methods exhibit suboptimal performance in face reconstruction and result in the loss of numerous facial details. In this paper, we propose DFLeak, an effective approach to boost face leakage from gradients using residual optimization and thwart the privacy of facial applications in collaborative learning. In particular, we first introduce a superior initialization method to stabilize the inversion process. Second, we propose to integrate prior-free face restoration (PFFR) results into the gradient inversion optimization process in a residual manner, which enriches facial details. We further design a pixel update schedule to mitigate the adverse effects of image regularization terms and preserve fine facial details. Comprehensive experimentation demonstrates the effectiveness of our approach in achieving more realistic and higher-quality facial image reconstructions, surpassing the performance of state-of-the-art gradient inversion attacks.

Smart grid, through networked smart meters employing the non-intrusive load monitoring (NILM) technique, can considerably discern the usage patterns of residential appliances. However, this technique also incurs privacy leakage. To address this issue, we propose an innovative scheme based on adversarial attack in this paper. The scheme effectively prevents NILM models from violating appliance-level privacy, while also ensuring accurate billing calculation for users. To achieve this objective, we overcome two primary challenges. First, as NILM models fall under the category of time-series regression models, direct application of traditional adversarial attacks designed for classification tasks is not feasible. To tackle this issue, we formulate a novel adversarial attack problem tailored specifically for NILM and providing a theoretical foundation for utilizing the Jacobian of the NILM model to generate imperceptible perturbations. Leveraging the Jacobian, our scheme can produce perturbations, which effectively misleads the signal prediction of NILM models to safeguard users' appliance-level privacy. The second challenge pertains to fundamental utility requirements, where existing adversarial attack schemes struggle to achieve accurate billing calculation for users. To handle this problem, we introduce an additional constraint, mandating that the sum of added perturbations within a billing period must be precisely zero. Experimental validation on real-world power datasets REDD and U.K.-DALE demonstrates the efficacy of our proposed solutions, which can significantly amplify the discrepancy between the output of the targeted NILM model and the actual power signal of appliances, and enable accurate billing at the same time. Additionally, our solutions exhibit transferability, making the generated perturbation signal from one target model applicable to other diverse NILM models.

In modern cloud platforms, it is becoming more important to preserve the privacy of guest virtual machines (VMs) from the untrusted host. To this end, Secure Encrypted Virtualization (SEV) is developed as a hardware extension to protect VMs by encrypting their memory pages and register states. Unfortunately, such confidential VMs are still vulnerable to micro-architectural side channels, and Hardware Performance Counters (HPCs) are a prominent information leakage source. To make matters worse, currently there is no systematic defense against the HPC side channels. We introduce Aegis, a unified framework for demystifying the inherent relations between the instruction execution and HPC event statistics, and defending VMs against HPC side channels with provable privacy guarantee and minimal performance overhead. Aegis consists of three modules. Application Profiler profiles the application offline and adopts information theory to quantitatively estimate the vulnerability of HPC events. Event Fuzzer leverages the fuzzing technique to automatically generate interesting inputs, i.e., instruction sequences, that can effectively alter the HPC observations. Event Obfuscator injects noisy instructions into the protected VM based on the differential privacy mechanisms for high efficiency and privacy. We present three case studies to demonstrate that Aegis can defeat different types of HPC side-channel attacks (i.e., website fingerprinting, DNN model extraction, keystroke sniffing). Evaluations show that Aegis can effectively decrease the attack accuracy from 90% to 2%, with only 3% overhead on the application execution time and 7% overhead on the CPU usage.

Generative Adversarial Networks (GANs) have been widely used in various application scenarios. Since the production of a commercial GAN requires substantial computational and human resources, the copyright protection of GANs is urgently needed. This paper presents a novel finger-printing scheme for the Intellectual Property (IP) protection of image-to-image GANs based on a trusted third party. We break through the stealthiness and robustness bottlenecks suffered by previous fingerprinting methods for classification models being naively transferred to GANs. Specifically, we innovatively construct a composite deep learning model from the target GAN and a classifier. Then we generate fingerprint samples from this composite model, and embed them in the classifier for effective ownership verification. This scheme inspires some concrete methodologies to practically protect the modern image-to-image translation GANs. Theoretical analysis proves that these methods can satisfy different security requirements necessary for IP protection. We also conduct extensive experiments to show that our solutions outperform existing strategies.

As one of the privacy threats to machine learning models, the membership inference attack (MIA) tries to infer whether a given sample is in the original training set of a victim model by analyzing its outputs. Recent studies only use the predicted hard labels to achieve impressive membership inference accuracy. However, such label-only MIA approach requires very high query budgets to evaluate the distance of the target sample from the victim model's decision boundary. We propose YOQO, a novel label-only attack to overcome the above limitation.YOQO aims at identifying a special area (called improvement area) around the target sample and crafting a query sample, whose hard label from the victim model can reliably reflect the target sample's membership. YOQO can successfully reduce the query budget from more than 1,000 times to only ONCE. Experiments demonstrate that YOQO is not only as effective as SOTA attack methods, but also performs comparably or even more robustly against many sophisticated defenses.

Modern Deep Learning Training (DLT) schedulers in GPU datacenters are designed to be very sophisticated with many configurations. These configurations need to be adjusted delicately as they can significantly affect the scheduling performance. Existing schedulers require the datacenter operator to tune the configurations only once before they are deployed, based on the historical workload traces. Unfortunately, workloads in a datacenter would experience dynamic changes and deviate a lot from the historical ones over time, making the pre-determined configurations less effective.

Text-to-image (T2I) diffusion models enjoy great popularity and many individuals and companies build their applications based on publicly released T2I diffusion models. Previous studies have demonstrated that backdoor attacks can elicit T2I diffusion models to generate unsafe target images through textual triggers. However, existing backdoor attacks typically demand substantial tuning data for poisoning, limiting their practicality and potentially degrading the overall performance of T2I diffusion models. To address these issues, we propose EvilEdit, a training-free and data-free backdoor attack against T2I diffusion models. EvilEdit directly edits the projection matrices in the cross-attention layers to achieve projection alignment between a trigger and the corresponding backdoor target. We preserve the functionality of the backdoored model using a protected whitelist to ensure the semantic of non-trigger words is not accidentally altered by the backdoor. We also propose a visual target attack EvilEdit VTA, enabling adversaries to use specific images as backdoor targets. We conduct empirical experiments on Stable Diffusion and the results demonstrate that the EvilEdit can backdoor T2I diffusion models within one second with up to 100% success rate. Furthermore, our EvilEdit modifies only 2.2% of the parameters and maintains the model's performance on benign prompts. Our code is available at https://github.com/haowang-cqu/EvilEdit.

Monocular Depth Estimation (MDE) enables the prediction of scene depths from a single RGB image, having been widely integrated into production-grade autonomous driving systems, e.g., Tesla Autopilot. Current adversarial attacks to MDE models focus on attaching an optimized adversarial patch to a designated obstacle. Although effective, this approach presents two inherent limitations: its reliance on specific obstacles and its limited malicious impact. In contrast, we propose a pioneering attack to MDE models that \textit{decouples obstacles from patches physically and deploys optimized patches on roads}, thereby extending the attack scope to arbitrary traffic participants. This approach is inspired by our groundbreaking discovery: \textit{various MDE models with different architectures, trained for autonomous driving, heavily rely on road regions} when predicting depths for different obstacles. Based on this discovery, we design the Adversarial Road Marking (AdvRM) attack, which camouflages patches as ordinary road markings and deploys them on roads, thereby posing a continuous threat within the environment. Experimental results from both dataset simulations and real-world scenarios demonstrate that AdvRM is effective, stealthy, and robust against various MDE models, achieving about 1.507 of Mean Relative Shift Ratio (MRSR) over 8 MDE models. The code is available at \url{https://github.com/a-c-a-c/AdvRM.git}

Federated learning is a technique that enables multiple parties to collaboratively train a model without sharing raw private data, and it is ideal for smart healthcare. However, it raises new privacy concerns due to the risk of privacy-sensitive medical data leakage. It is not until recently that the privacy-preserving FL (PPFL) has been introduced as a solution to ensure the privacy of training processes. Unfortunately, most existing PPFL schemes are highly dependent on complex cryptographic mechanisms or fail to guarantee the accuracy of training models. Besides, there has been little research on the fairness of the payment procedure in the PPFL with incentive mechanisms. To address the above concerns, we first construct an efficient non-interactive designated decryptor function encryption (NDD-FE) scheme to protect the privacy of training data while maintaining high communication performance. We then propose a blockchain-based PPFL framework with fair payment for medical image detection, namely ESB-FL, by combining the NDD-FE and an elaborately designed blockchain. ESB-FL not only inherits the characteristics of the NDD-FE scheme, but it also ensures the interests of each participant. We finally conduct extensive security analysis and experiments to show that our new framework has enhanced security, good accuracy, and high efficiency.

Deep Neural Networks are well-known to be vulnerable to Adversarial Examples. Recently, advanced gradient-based attacks were proposed (e.g., BPDA and EOT), which can significantly increase the difficulty and complexity of designing effective defenses. In this paper, we present a study towards the opportunity of mitigating those powerful attacks with only pre-processing operations. We make the following two contributions. First, we perform an in-depth analysis of those attacks and summarize three fundamental properties that a good defense solution should have. Second, we design a lightweight preprocessing function with these properties and the capability of preserving the model's usability and robustness against these threats. Extensive evaluations indicate that our solutions can effectively mitigate all existing standard and advanced attack techniques, and beat 11 state-of-the-art defense solutions published in top-tier conferences over the past 2 years.

Perceptual image encryption degrades image quality by selectively encrypting some key information of the plain images. The encrypted images are partially perceptible according to the security or quality requirements. Although several types of attacks have tried to infer privacy information from the encrypted images, they can only either extract statistical information or enhance image sketch. In this paper, we take one step further and fully recover the plain images from perceptually encrypted counterparts by designing a non-local attack network (NL-ANet). NL-ANet is composed of densely cascaded multiscale non-local modules (MSNL) and a hierarchical attention fusion module (HAFM). In particular, to better reconstruct encryption distortion, we introduce MSNL to capture powerful hierarchical features from different scales, and propose HAFM to adaptively aggregate and enhance informative hierarchical features for reconstruction. We also propose a new instantiation of the multi-head non-local block with channel attention (MHCA) to explore the long-range dependencies of global contextual information. Extensive experiments show that NL-ANet is encryption-agnostic and superior on different perceptual encryption schemes under different encryption strengths. NL-ANet also achieves better performance than state-of-the-art image restoration methods.

In the context of drone swarms, achieving efficient group secure communication is a challenging problem, due to the inherent limitations imposed by the drones’ limited energy and constrained resources. Physical layer group key generation (PLGK) is a promising technology to enable efficient group security communication. However, most existing PLGK schemes struggle to adapt to the dynamic nature of drone swarms. To address this gap, this paper proposes a novel satellite cluster state information (SCSI)-based PLGK, which leverages signal status information from all visible navigation satellites to establish the group key. The presented method utilizes the regional similarity of SCSI as a random information source to generate group keys between different drones, and employs a novel updating framework based on a fuzzy generator and a hash chain to enhance key update and alignment robustness. The proposed scheme not only significantly reduces the overhead of group key generation also mitigates the issues of key loss and reconstruction. The security of the proposed scheme is validated through formal protocol security proof and security analysis against possible attacks. Finally, experiments with real-world drones demonstrate the efficiency and effectiveness of the SCSI-based PLGK.

There is an increasing tendency to fine-tune large-scale pre-trained language models (LMs) using small private datasets to improve their capability for downstream applications. In this paper, we systematically analyze the pre-train and then fine-tune the process of generative LMs and show that the fine-tuned LMs would leak sensitive keywords of the private datasets even without any prior knowledge of the downstream tasks. Specifically, we propose a novel and efficient keyword inference attack framework to accurately and maximally recover sensitive keywords. Owing to the fine-tuning process, pre-trained and fine-tuned models might respond differently to identical input prefixes. To identify potential sensitive sentences for training the fine-tuend LM, we introduce a contrast difference score that assesses the response variations between a pre-trained LM and its corresponding fine-tuned LM. Following this, we iteratively fine-tune the pre-trained model using these sensitive sentences to minimize the disparity between the target model and the pre-trained model, thereby maximizing the number of inferred sensitive keywords. We implement two types of keyword inference attacks (i.e., domain and private) according to our framework and conduct comprehensive experiments on three downstream applications to evaluate the performance. The experimental results demonstrate that our domain keyword inference attack achieves a precision of 85%, while our private keyword inference attack can extract highly sensitive personal information for a significant number of individuals (approximately 0.3% of all customers in the private fine-tuning dataset, which contains 40,000 pieces of personal information).

In recent decades, Generative Adversarial Network (GAN) and its variants have achieved unprecedented success in image synthesis. However, well-trained GANs are under the threat of illegal steal or leakage. The prior studies on remote ownership verification assume a black-box setting where the defender can query the suspicious model with specific inputs, which we identify is not enough for generation tasks. To this end, in this paper, we propose a novel IP protection scheme for GANs where ownership verification can be done by checking outputs only, without choosing the inputs (i.e., box-free setting). Specifically, we make use of the unexploited potential of the discriminator to learn a hypersphere that captures the unique distribution learned by the paired generator. Extensive evaluations on two popular GAN tasks and more than 10 GAN architectures demonstrate our proposed scheme to effectively verify the ownership. Our proposed scheme shown to be immune to popular input-based removal attacks and robust against other existing attacks. The source code and models are available at https://github.com/AbstractTeen/gan_ownership_verification.

Visual Question Answering (VQA) is the vision-language task of answering text-based questions presented in an image and has been advanced by the remarkable success of multimodal deep networks. Similar to unimodal networks, multimodal VQA models are also vulnerable to adversarial examples, which raises severe threats to the corresponding applications. Although several adversarial training methods have been proposed, most of them focus on improving the generalization ability of VQA models on clean samples instead of mitigating the adversarial attacks. In this paper, we systemically analyze the core structure of multimodal VQA networks and propose a novel adversarial training algorithm to mitigate adversarial attacks on VQA models. Specifically, our key component is a regularization term with our carefully designed Contrastive Fusion Representation (CFR), which can reduce the sensitivity of VQA models to adversarial perturbations of both the vision and language inputs. We further enhance the adversarial training with augmented CFRs. Comprehensive experimental results show that our method can mitigate adversarial attacks as well as preserve the generalization ability on clean samples under various system settings and outperforms other defense methods.

Collaborative learning has gained great popularity due to its benefit of data privacy protection: participants can jointly train a Deep Learning model without sharing their training sets. However, recent works discovered that an adversary can fully recover the sensitive training samples from the shared gradients. Such reconstruction attacks pose severe threats to collaborative learning. Hence, effective mitigation solutions are urgently desired. In this paper, we systematically analyze existing reconstruction attacks and propose to leverage data augmentation to defeat these attacks: by preprocessing sensitive images with carefully-selected transformation policies, it becomes infeasible for the adversary to extract training samples from the corresponding gradients. We first design two new metrics to quantify the impacts of transformations on data privacy and model usability. With the two metrics, we design a novel search method to automatically discover qualified policies from a given data augmentation library. Our defense method can be further combined with existing collaborative training systems without modifying the training protocols. We conduct comprehensive experiments on various system settings. Evaluation results demonstrate that the policies discovered by our method can defeat state-of-the-art reconstruction attacks in collaborative learning, with high efficiency and negligible impact on the model performance.

Decentralized image classification plays a key role in various scenarios due to its attractive properties, including tolerating high network latency and less prone to single-point failures. Unfortunately, training such a decentralized image classification model is more vulnerable to data privacy leaks compared to other distributed training frameworks. Existing efforts exclusively use differential privacy as the cornerstone to alleviate the threat to data privacy. However, differential privacy is implemented at the expense of accuracy, which goes against our motivation for designing an image classification model without loss of accuracy. To address this problem, we propose D2-MHE, thefirstsecure and efficient decentralized training framework with lossless precision. Inspired by the latest developments in the homomorphic encryption technology, we design a multiparty version of Brakerski-Fan-Vercauteren (BFV), one of the most advanced cryptosystems, and use it to implement private gradient updates of users’ local models. D2-MHE can reduce the communication complexity of general Secure Multiparty Computation (MPC) tasks from quadratic to linear in the number of users, making it very suitable and scalable for large-scale decentralized learning systems. Moreover, D2-MHE provides strict semantic security protection even if the majority of users are dishonest with collusion. We conduct extensive experiments on MNIST, CIFAR-10, and ImageNet to demonstrate the superiority of D2-MHE. Experimental results show that D2-MHE achieves up to$5.5\times $reduction in computation overhead, and at least$12\times $reduction in communication overhead compared to existing schemes.

Vision-based practical applications, such as consumer photography and automated driving systems, greatly rely on enhancing the visibility of images captured in night-time environments. For this reason, various image enhancement algorithms (EHAs) have been proposed. However, little attention has been given to the quality evaluation of enhanced night-time images. In this paper, we conduct the first dedicated exploration of the subjective and objective quality evaluation of enhanced night-time images. First, we build an enhanced night-time image quality (EHNQ) database, which is the largest of its kind so far. It includes 1,500 enhanced images generated from 100 real night-time images using 15 different EHAs. Subsequently, we perform a subjective quality evaluation and obtain subjective quality scores on the EHNQ database. Thereafter, we present an objective blind quality index for enhanced night-time images (BEHN). Enhanced night-time images usually suffer from inappropriate brightness and contrast, deformed structure, and unnatural colorfulness. In BEHN, we capture perceptual features that are highly relevant to these three types of corruptions, and we design an ensemble training strategy to map the extracted features into the quality score. Finally, we conduct extensive experiments on EHNQ and EAQA databases. The experimental and analysis results validate the performance of the proposed BEHN compared with the state-of-the-art approaches. Our EHNQ database is publicly available for download athttps://sites.google.com/site/xiangtaooo/.

Deep hiding, embedding images with others using deep neural networks, has demonstrated impressive efficacy in increasing the message capacity and robustness of secret sharing. In this paper, we challenge the robustness of existing deep hiding schemes by preventing the recovery of secret images, building on our in-depth study of state-of-the-art deep hiding schemes and their vulnerabilities. Leveraging our analysis, we first propose a simple box-free removal attack on deep hiding that does not require any prior knowledge of the deep hiding schemes. To improve the removal performance on the deep hiding schemes that may be enhanced by adversarial training, we further design a more powerful removal attack, efficient box-free removal attack (EBRA), which employs image inpainting techniques to remove secret images from container images. In addition, to ensure the effectiveness of our attack and preserve the fidelity of the processed container images, we design an erasing phase based on the locality of deep hiding to remove secret information and then make full use of the visual information of container images to repair the erased visual content. Extensive evaluations show our method can completely remove secret images from container images with negligible impact on the quality of container images.

Adversarial attacks have threatened the application of deep neural networks in security-sensitive scenarios. Most existing black-box attacks fool the target model by interacting with it many times and producing global perturbations. However, all pixels are not equally crucial to the target model; thus, indiscriminately treating all pixels will increase query overhead inevitably. In addition, existing black-box attacks take clean samples as start points, which also limits query efficiency. In this article, we propose a novel black-box attack framework, constructed on a strategy of dual transferability (DT), to perturb the discriminative areas of clean examples within limited queries. The first kind of transferability is the transferability of model interpretations. Based on this property, we identify the discriminative areas of clean samples for generating local perturbations. The second is the transferability of adversarial examples, which helps us to produce local pre-perturbations for further improving query efficiency. We achieve the two kinds of transferability through an independent auxiliary model and do not incur extra query overhead. After identifying discriminative areas and generating pre-perturbations, we use the pre-perturbed samples as better start points and further perturb them locally in a black-box manner to search the corresponding adversarial examples. The DT strategy is general; thus, the proposed framework can be applied to different types of black-box attacks. We conduct extensive experiments to show that, under various system settings, our framework can significantly improve the query efficiency of existing black-box attacks and attack success rates.

With the continued proliferation of mobile Internet and geo-locating technologies, carpooling as a green transport mode is widely accepted and becoming tremendously popular worldwide. In this paper, we focus on a popular carpooling service calledride hitching, which is typically implemented using a publish/subscribe approach. In a ride hitching service, drivers subscribe ride orders published by riders and continuously receive matching ride orders until one is picked. The current systems (e.g., Didi Hitch) adopt a threshold-based approach to filter ride orders. That is, a new ride order will be sent to all subscribing drivers whose planned trips can match the ride order within a pre-defined detour threshold. A limitation of this approach is that it is difficult for drivers to specify a reasonable detour threshold in practice. In addressing this problem, we propose a novel type of top-$k$subscription queries calledTop-$k$kRideSubscription (TkRS)query, which continuously returns the best$k$ride orders that match drivers’ trip plans to them. We propose two efficient algorithms to enable the top-$k$result maintenance. We also design a novel hybrid grid index and a two-level buffer structure to efficiently track the top-$k$results for allTkRSqueries. Finally, extensive experiments on real-life datasets suggest that our proposed algorithms are capable of achieving desirable performance in practical settings.

Medical image fusion generates a fused image containing multiple features extracted from different source images, and it is of great help in clinical analysis and diagnosis. However, training a deep learning model for image fusion usually requires enormous computing power, especially for large volumes of medical data. Meanwhile, the privacy of images is also a critical issue. In this article, we propose a privacy-preserving blockchain-based medical image fusion (BMIF) framework. First, to ensure fusion performance, we design a new medical image fusion model based on convolutional neural network and Inception network and integrate the proposed model into the consensus process of blockchain. Next, to save computing power of blockchain, we design a consensus mechanism by requesting consensus nodes to train the fusion model instead of calculating useless hash values in traditional blockchain. Then, to protect data privacy, we further present an efficient homomorphic encryption to realize the training of fusion model on encrypted medical data. Finally, we conduct theoretical analysis and extensive experiments on public datasets to evaluate the feasibility and the performance of our proposed BMIF. The results exhibit that BMIF is efficient and secure, and our medical image fusion network performs better than state-of-the-art approaches.

Training Deep Neural Networks (DNN) is a time-consuming process and requires a large amount of training data, which motivates studies working on protecting the intellectual property (IP) of DNN models by employing various watermarking techniques. Unfortunately, in recent years, adversaries have been exploiting the vulnerabilities of the employed watermarking techniques to remove the embedded watermarks. In this paper, we investigate and introduce a novel watermark removal attack, called AdvNP, against all the existing four different types of DNN watermarking schemes via input preprocessing by injecting Adversarial Naturalness-aware Perturbations. In contrast to the prior studies, our proposed method is the first work that generalizes all the existing four watermarking schemes well without involving any model modification, which preserves the fidelity of the target model. We conduct the experiments against four state-of-the-art (SOTA) watermarking schemes on two real tasks (e.g., image classification on ImageNet, face recognition on CelebA) across multiple DNN models. Overall, our proposed AdvNP significantly invalidates the watermarks against the four watermarking schemes on two real-world datasets, i.e., 60.9% on the average attack success rate and up to 97% in the worse case. Moreover, our AdvNP could well survive the image denoising techniques and outperforms the baseline in both the fidelity preserving and watermark removal. Furthermore, we introduce two defense methods to enhance the robustness of DNN watermarking against our AdvNP. Our experimental results pose real threats to the existing watermarking schemes and call for more practical and robust watermarking techniques to protect the copyright of pre-trained DNN models. The source code and models are available at ttps://github.com/GitKJ123/AdvNP.

Deep neural networks (DNNs) have shown their powerful capability in scene text editing (STE). With carefully designed DNNs, one can alter texts in a source image with other ones while maintaining their realistic look. However, such editing tools provide a great convenience for criminals to falsify documents or modify texts without authorization. In this paper, we propose to actively defeat text editing attacks by designing invisible "armors" for texts in the scene. We turn the adversarial vulnerability of DNN-based STE into strength and design local perturbations (i.e., "armors") specifically for texts using an optimized normalization strategy. Such local perturbations can effectively mislead STE attacks without affecting the perceptibility of scene background. To strengthen our defense capabilities, we systemically analyze and model STE attacks and provide a precise defense method to defeat attacks on different editing stages. We conduct both subjective and objective experiments to show the superior of our optimized local adversarial perturbation against state-of-the-art STE attacks. We also evaluate the portrait and landscape transferability of our perturbations.

Leilei Gan, Jiwei Li, Tianwei Zhang, Xiaoya Li, Yuxian Meng, Fei Wu, Yi Yang, Shangwei Guo, Chun Fan. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022.

Modern deep neural networks are highly vulnerable to adversarial examples, which attracts more and more researchers' attention to craft powerful adversarial examples. Most of these generation algorithms create global perturbations that would affect the visual quality of adversarial examples. To mitigate such drawbacks, some attacks attempt to generate local perturbations. However, existing local adversarial attacks are time-consuming and the generated adversarial examples are still distinguishable from clean images. In this paper, we propose a novel efficient local adversarial attack (ELAA) using model interpreters to generate severe local perturbations and improve the imperceptibly of the generated adversarial examples. Specifically, we take advantage of model interpretation methods to search the discriminative regions of clean images. Then, we generate local adversarial examples by adding masks to original clean images. We also propose a new optimization method to reduce the redundancy of local perturbations. Through extensive experiments, we show our ELAA can maintain a high attack ability while preserving the visual quality of clean images. Experimental results also demonstrate our local attack outperforms state-of-the-art local attack methods under various system settings.

Image classification is a fundamental artificial intelligence task that labels images into one of some predefined classes. However, training complex image classification models requires a large amount of computation resources and data in order to reach state-of-the-art performance. This demand drives the growth of distributed deep learning, where multiple agents cooperatively train global models with their individual datasets. Among such learning systems, decentralized learning is particularly attractive, as it can improve the efficiency and fault tolerance by eliminating the centralized parameter server, which could be the single point of failure or performance bottleneck. Although the agents do not need to disclose their training image samples, they exchange parameters with each other at each iteration, which can put them at the risk of data privacy leakage. Past works demonstrated the possibility of recovering training images from the exchanged parameters. One common defense direction is to adopt Differential Privacy (DP) to secure the optimization algorithms such as Stochastic Gradient Descent (SGD). Those DP-based methods mainly focus on standalone systems, or centralized distributed learning. How to enforce and optimize DP protection in decentralized learning systems is unknown and challenging, due to their complex communication topologies and distinct learning characteristics. In this paper, we design TOP- DP, a novel solution to optimize the differential privacy protection of decentralized image classification systems. The key insight of our solution is to leverage the unique features of decentralized communication topologies to reduce the noise scale and improve the model usability. (1) We enhance the DP-SGD algorithm with thistopology-awarenoise reduction strategy, and integrate the time-aware noise decay technique. (2) We design two novel learning protocols (synchronous and asynchronous) to protect systems with different network connectivities and topologies. We formally analyze and prove the DP requirement of our proposed solutions. Experimental evaluations demonstrate that our solution achieves a better trade-off between usability and privacy than prior works. To the best of our knowledge, this is the first DP optimization work from the perspective of network topologies.

Decentralized learning has gained great popularity to improve learning efficiency and preserve data privacy. Each computing node makes equal contribution to collaboratively learn a Deep Learning model. The elimination of centralized Parameter Servers (PS) can effectively address many issues such as privacy, performance bottleneck and single-point-failure. However, how to achieve Byzantine Fault Tolerance in decentralized learning systems is rarely explored, although this problem has been extensively studied in centralized systems. In this paper, we present an in-depth study towards the Byzantine resilience of decentralized learning systems with two contributions. First, from the adversarial perspective, we theoretically illustrate that Byzantine attacks are more dangerous and feasible in decentralized learning systems: even one malicious participant can arbitrarily alter the models of other participants by sending carefully crafted updates to its neighbors. Second, from the defense perspective, we propose Ubar, a novel algorithm to enhance decentralized learning with Byzantine Fault Tolerance. Specifically, Ubar provides aUniformByzantine-resilientAggregationRule for benign nodes to select the useful parameter updates and filter out the malicious ones in each training iteration. It guarantees that each benign node in a decentralized system can train a correct model under very strong Byzantine attacks with an arbitrary number of faulty nodes. We conduct extensive experiments on standard image classification tasks and the results indicate that Ubar can effectively defeat both simple and sophisticated Byzantine attacks with higher performance efficiency than existing solutions.

Deep Neural Networks (DNN) are gaining higher commercial values in computer vision applications, e.g., image classification, video analytics, etc. This calls for urgent demands of the intellectual property (IP) protection of DNN models. In this paper, we present a novel watermarking scheme to achieve the ownership verification of DNN architectures. Existing works all embedded watermarks into the model parameters while treating the architecture as public property. These solutions were proven to be vulnerable by an adversary to detect or remove the watermarks. In contrast, we claim the model architectures as an important IP for model owners, and propose to implant watermarks into the architectures. We design new algorithms based on Neural Architecture Search (NAS) to generate watermarked architectures, which are unique enough to represent the ownership, while maintaining high model usability. Such watermarks can be extracted via side-channel-based model extraction techniques with high fidelity. We conduct comprehensive experiments on watermarked CNN models for image classification tasks and the experimental results show our scheme has negligible impact on the model performance, and exhibits strong robustness against various model transformations and adaptive attacks.

Unrestricted adversarial examples allow the attacker to start attacks without given clean samples, which are quite aggressive and threatening. However, existing works for generating unrestricted adversary examples are quite inefficient and cannot achieve a high success rate. In this article, we explore an end-to-end and effective solution for unrestricted adversary example generation. To stabilize the training process and make our generative model converge to satisfactory results, we design a novel decoupled two-step efficient generative model (EGM), which contains a conditional reference generator and a conditional adversarial transformer. The former is responsible for generating reference samples from noises and source classes. The latter is responsible for converting the reference sample into adversarial examples corresponding to target classes. To improve the success rate, we design a new strategy, augmentation of adversarial labels to produce dynamic target labels and enhance the exploration ability of EGM. Such a strategy can be also applied to existing attacks to improve their attack success rates, which is of independent interest. We conduct extensive experiments to evaluate our proposed model and demonstrate the necessity of decoupling the generation process in EGM. Experimental results show our EGM is much faster and achieves a higher success rate than the state-of-the-art attacks.

With the advent of cloud computing, it has become more and more popular to outsource various services to the cloud for releasing the burden of local data storage and maintenance. However, it may cause serious privacy problems because the cloud may be untrusted. In this article, we study the privacy-preserving reverse nearest neighbor (PPRNN) query over encrypted spatial data. First, we introduce the concept of reference-locked order-preserving encryption (RL-OPE) with its construction and security proof, which reveals less information than traditional order-preserving encryption (OPE). Then, we present a novel PPRNN scheme in static setting based on structured encryption (SE) and the proposed RL-OPE, called sPPRNN. After that, we design a generic method that extends a PPRNN scheme in static setting to the counterpart in dynamic setting, called dPPRNN. Furthermore, we present a thorough privacy analysis of our proposal. Finally, we demonstrate its efficiency and effectiveness for practical deployment through extensive experiments.

Public resources and services (e.g., datasets, training platforms, pre-trained models) have been widely adopted to ease the development of Deep Learning-based applications. However, if the third-party providers are untrusted, they can inject poisoned samples into the datasets or embed backdoors in those models. Such an integrity breach can cause severe consequences, especially in safety- and security-critical applications. Various backdoor attack techniques have been proposed for higher effectiveness and stealthiness. Unfortunately, existing defense solutions are not practical to thwart those attacks in a comprehensive way.

This paper presents the first model extraction attack against Deep Reinforcement Learning (DRL), which enables an external adversary to precisely recover a black-box DRL model only from its interaction with the environment. Model extraction attacks against supervised Deep Learning models have been widely studied. However, those techniques cannot be applied to the reinforcement learning scenario due to DRL models' high complexity, stochasticity and limited observable information. We propose a novel methodology to overcome the above challenges. The key insight of our approach is that the process of DRL model extraction is equivalent to imitation learning, a well-established solution to learn sequential decision-making policies. Based on this observation, our methodology first builds a classifier to reveal the training algorithm family of the targeted black-box DRL model only based on its predicted actions, and then leverages state-of-the-art imitation learning techniques to replicate the model from the identified algorithm family. Experimental results indicate that our methodology can effectively recover the DRL models with high fidelity and accuracy. We also demonstrate two use cases to show that our model extraction attack can (1) significantly improve the success rate of adversarial attacks, and (2) steal DRL models stealthily even they are protected by DNN watermarks. These pose a severe threat to the intellectual property and privacy protection of DRL applications.

Collaborative learning has gained great popularity due to its benefit of data privacy protection: participants can jointly train a Deep Learning model without sharing their training sets. However, recent works discovered that an adversary can fully recover the sensitive training samples from the shared gradients. Such reconstruction attacks pose severe threats to collaborative learning. Hence, effective mitigation solutions are urgently desired.In this paper, we propose to leverage data augmentation to defeat reconstruction attacks: by preprocessing sensitive images with carefully-selected transformation policies, it becomes infeasible for the adversary to extract any useful information from the corresponding gradients. We design a novel search method to automatically discover qualified policies. We adopt two new metrics to quantify the impacts of transformations on data privacy and model usability, which can significantly accelerate the search speed. Comprehensive evaluations demonstrate that the policies discovered by our method can defeat existing reconstruction attacks in collaborative learning, with high efficiency and negligible impact on the model performance.

Watermarking has become the tendency in protecting the intellectual property of DNN models. Recent works, from the adversary's perspective, attempted to subvert watermarking mechanisms by designing watermark removal attacks. However, these attacks mainly adopted sophisticated fine-tuning techniques, which have certain fatal drawbacks or unrealistic assumptions. In this paper, we propose a novel watermark removal attack from a different perspective. Instead of just fine-tuning the watermarked models, we design a simple yet powerful transformation algorithm by combining imperceptible pattern embedding and spatial-level transformations, which can effectively and blindly destroy the memorization of watermarked models to the watermark samples. We also introduce a lightweight fine-tuning strategy to preserve the model performance. Our solution requires much less resource or knowledge about the watermarking scheme than prior works. Extensive experimental results indicate that our attack can bypass state-of-the-art watermarking solutions with very high success rates. Based on our attack, we propose watermark augmentation techniques to enhance the robustness of existing watermarks.

Perceptual encryption is an efficient way of protecting image content by only selectively encrypting a portion of significant data in plain images. Existing security analysis of perceptual encryption usually resorts to traditional cryptanalysis techniques, which require heavy manual work and strict prior knowledge of encryption schemes. In this paper, we introduce a new end-to-end method of analyzing the visual security of perceptually encrypted images, without any manual work or knowing any prior knowledge of the encryption scheme. Specifically, by leveraging convolutional neural networks (CNNs), we propose a progressive recovery network (PRNet) to recover visual content from perceptually encrypted images. Our PRNet is stacked with several dense attention recovery blocks (DARBs), where each DARB contains two branches: feature extraction branch and image recovery branch. These two branches cooperate to rehabilitate more detailed visual information and generate efficient feature representation via densely connected structure and dual-saliency mechanism. We conduct extensive experiments to demonstrate that PRNet works on different perceptual encryption schemes with different settings, and the results show that PRNet significantly outperforms the state-of-the-art CNN-based image restoration methods.

We demonstrate vCBIR, a verifiable search engine for Content-Based Image Retrieval. vCBIR allows a small or medium-sized enterprise to outsource its image database to a cloud-based service provider and ensures the integrity of query processing. Like other common data-as-a-service (DaaS) systems, vCBIR consists of three parties: (i) the image owner who outsources its database, (ii) the service provider who executes the authenticated query processing, and (iii) the client who issues search queries. By employing a novel query authentication scheme proposed in our prior work [4], the system not only supports cloud-based image retrieval, but also generates a cryptographic proof for each query, by which the client could verify the integrity of query results. During the demonstration, we will showcase the usage of vCBIR and also provide attendees interactive experience of verifying query results against an untrustworthy service provider through graphical user interface (GUI).

Perceptual/selective encryption has been gaining widespread attention as an emerging technology for image privacy protection. However, few studies focus on the visual security evaluation of perceptually encrypted images, which has a significant impact on measuring the effectiveness and practicality of these encryption methods. In this paper, we propose an image importance-based visual security index (IIBVSI) by leveraging spatial contrast and texture features. Based on the characteristics of perceptually encrypted images, we present an averaged high-order gradient magnitude map to describe the spatial contrast feature and introduce a combined local amplitude map of multiple log-Gabor filters to represent the texture feature. Specifically, the multiresolution representation of an image is first created by downsampling to simulate the hierarchical property of the human visual system. Next, for each scale of image resolution, the spatial contrast and the texture feature maps are extracted from both plain and encrypted images. Similarity measurements are then conducted on these feature maps to generate the contrast and the texture similarity maps. An image importance-based pooling strategy is subsequently proposed to combine these measurements and generate a visual security score. The final IIBVSI score is computed by averaging the visual security scores of all scales of image resolution. Extensive experiments are conducted on several publicly available databases, and the results demonstrate the superiority and robustness of our proposed IIBVSI compared with existing state-of-the-art work in the low and moderate image quality ranges.

Perceptual image encryption provides an efficient and effective way to preserve the confidentiality of visual information, and the measurement of content leakage is of fundamental importance for perceptually encrypted images. Numerous visual security indexes (VSIs) have been proposed to evaluate visual content leakage. Due to the lack of perceptually encrypted image databases, image quality assessment (IQA) databases are widely adopted to evaluate the performance of existing VSIs. However, there are huge differences between VSIs and IQAs. The misuse of databases may lead to an inaccurate evaluation. In this paper, we propose a perceptually encrypted image database (PEID) which contains 1080 encrypted images from 20 plain images with 10 well-known perceptual encryption techniques. Both visual quality and content leakage scores of the encrypted images are obtained through a comprehensive subjective evaluation. We also propose a systemic methodology to accurately evaluate the monotonicity, fitness, and accuracy of VSIs. We conduct extensive experiments on the proposed PEID to evaluate the performance of existing state-of-the-art VSIs. We have made the database publicly available for download and hope that the proposed PEID can facilitate the research of visual security evaluation and beyond.

Blind image quality assessment (BIQA) aims to develop quantitative measures to automatically and accurately estimate the visual quality of an image without any prior information about its reference image. This issue has been attracting a great deal of attention for a long time; however, little work has been done on night-time images, which are crucially important for consumer photography and practical applications such as automated driving systems. In this paper, to the best of our knowledge, we conduct the first exploration on subjective and objective quality assessment of night-time images. First, we build a large-scale natural night-time image database (NNID) containing 2240 images with 448 different image contents captured by different photographic equipment in real-world scenarios. Subsequently, we carry out a subjective experiment to evaluate the perceptual quality of all the images in the NNID database. Thereafter, we perform objective assessment of night-time images by proposing a blind night-time image quality assessment metric using brightness and texture features (BNBT). Finally, extensive experiments are conducted to evaluate the performance and efficiency of the proposed BNBT metric on the NNID database. The experimental results demonstrate that this metric outperforms existing state-of-the-art BIQA methods in terms of all evaluation criteria and has an acceptable computational cost at the same time. We have made the NNID database publicly available for downloading at https://sites.google.com/site/xiangtaooo/.

With the explosive growth of online images and the popularity of search engines, a great demand has arisen for small and medium-sized enterprises to build and outsource large-scale image retrieval systems to cloud platforms. While reducing storage and retrieval burdens, enterprises are at risk of facing untrusted cloud service providers. In this paper, we take the first step in studying the problem of query authentication for large-scale image retrieval. Due to the large size of image files, the main challenges are to (i) design efficient authenticated data structures (ADSs) and (ii) balance search, communication, and verification complexities. To address these challenges, we propose two novel ADSs, the Merkle randomized k-d tree and the Merkle inverted index with cuckoo filters, to ensure the integrity of query results in each step of image retrieval. For each ADS, we develop corresponding search and verification algorithms on the basis of a series of systemic design strategies. Furthermore, we put together the ADSs and algorithms to design the final authentication scheme for image retrieval, which we name ImageProof. We also propose several optimization techniques to improve the performance of the proposed ImageProof scheme. Security analysis and extensive experiments are performed to show the robustness and efficiency of ImageProof.

With the development in recent decades of various efficient image encryption algorithms, such as selective encryption, a great demand has arisen for methods of evaluating the visual security of encrypted images. Existing solutions usually adopt well-known metrics of visual quality assessment to measure the quality of encrypted images, but they often exhibit undesired behavior on perceptually encrypted images of low quality. In this paper, we propose a novel visual security index (VSI) based on the human visual system. The proposed VSI evaluates two aspects of the content similarity between plain and encrypted images: the edge similarity extracted via multi-threshold edge detection and the texture similarity measured by means of the co-occurrence matrix. These two components are further integrated to obtain the proposed VSI through adaptive similarity weighting. Extensive experiments were performed on two publicly available image databases. Our experimental results demonstrate that compared with many existing state-of-the-art visual security metrics, the proposed VSI exhibits a better performance and stability on low-quality images.