Xueluan Gong

dblp:268/5832 · DBLP profile ↗
← Back
29ranked-venue papers
17as first author
28since 2021 · last 2026
0000-0003-2190-8117ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Security and privacy · 20 · 13 first-author · 20 since 2021Computer networks · 4 · 1 first-author · 3 since 2021Artificial intelligence and machine learning · 3 · 2 first-author · 3 since 2021Graphics, computer vision, multimedia, augmented reality and games · 2 · 1 first-author · 2 since 2021Applied, interdisciplinary, general and emerging computing · 2 · 1 first-author · 2 since 2021Databases, data management, data science and information retrieval · 1 · 1 first-author · 1 since 2021
YearPublicationVenuePosition
2026 Armor: Shielding Unlearnable Examples Against Data Augmentation
abstract
Private data, when published online, may be collected by unauthorized parties to train deep neural networks (DNNs). To protect privacy, defensive noises can be added to original samples to degrade their learnability by DNNs. Recently, unlearnable examples (Huang et al., 2021) are proposed to minimize the training loss such that the model learns almost nothing. However, raw data are often pre-processed before being used for training, which may restore the private information of protected data. In this paper, we reveal the data privacy violation induced by data augmentation, a commonly used data pre-processing technique to improve model generalization capability, which is the first of its kind as far as we are concerned. We demonstrate that data augmentation can significantly raise the accuracy of the model trained on unlearnable examples from 21.3% to 66.1%. To address this issue, we propose a defense framework, dubbed Armor, to protect data privacy from potential breaches of data augmentation. To overcome the difficulty of having no access to the model training process, we design a non-local module-assisted surrogate model that better captures the effect of data augmentation. In addition, we design a surrogate augmentation selection strategy that maximizes distribution alignment between augmented and non-augmented samples, to choose the optimal augmentation strategy for each class. We also use a dynamic step size adjustment algorithm to enhance the defensive noise generation process. Extensive experiments are conducted on 4 datasets and 5 data augmentation methods to verify the performance of Armor. Comparisons with 6 state-of-the-art defense methods have demonstrated that Armor can preserve the unlearnability of protected private data under data augmentation. Armor reduces the test accuracy of the model trained on augmented protected samples by as much as 60% more than baselines. We also show that Armor is robust to adversarial training. We will open-source our codes upon publication.
Xueluan Gong, Yuji Wang, Yanjiao Chen, Haocheng Dong, Yiming Li 0004, Mengyuan Sun 0001, Shuaike Li, Qian Wang 0002
IEEE Trans. Pattern Anal. Mach. Intell.1
2026 Megatron: Evasive Clean-Label Backdoor Attacks Against Vision Transformer
abstract
Vision transformers have achieved impressive performance in various vision-related tasks, but their vulnerability to backdoor attacks is under-explored. A handful of existing works focus on dirty-label attacks with wrongly-labeled poisoned training samples, which may fail if a benign model trainer corrects the labels. In this paper, we proposeMegatron, an evasive clean-label backdoor attack against vision transformers, where the attacker injects the backdoor without manipulating the data-labeling process. To generate an effective trigger, we employ a local surrogate vision transformer to approximate the victim model and customize two attention-based loss terms: latent loss and attention diffusion loss. The latent loss aligns the last attention layer between triggered samples and clean samples of the target label. The attention diffusion loss emphasizes the attention diffusion area that encompasses the trigger. A theoretical analysis is provided to underpin the rationale behind the attention diffusion loss. Extensive experiments on CIFAR-10, GTSRB, CIFAR-100, and Tiny ImageNet demonstrate the effectiveness ofMegatron.Megatroncan achieve attack success rates of over 90% even when the position of the trigger is slightly shifted during testing. Furthermore,Megatronachieves better evasiveness than baselines regarding both human visual inspection and defense strategies (i.e., DBAVT, BAVT, Beatrix, TeCo, and SAGE).
Xueluan Gong, Bowei Tian, Meng Xue 0001, Shuaike Li, Yanjiao Chen, Qian Wang 0002
IEEE Trans. Dependable Secur. Comput.1
2026 Sleight: Hidden Data Privacy Breaches in Federated Learning
abstract
Federated Learning (FL) has emerged as a paradigm for conducting machine learning across broad and decentralized datasets, promising enhanced privacy by obviating the need for direct data sharing. However, recent studies show that attackers can steal private data through model manipulation or gradient analysis. Existing attacks are constrained by low theft quantity or low-resolution data, and they are often easily detected through anomaly monitoring in gradients or weights. In this paper, we propose Sleight, a novel data-reconstruction attack, supported by two key techniques, i.e., distinctive and sparse encoding design and block partitioning. Unlike conventional methods that require detectable changes to the model, Sleight stealthily embeds a hidden model using parameter sharing to systematically extract sensitive data. The Fibonacci-based index design ensures efficient, structured retrieval of memorized data, while the block partitioning method enhances Sleight's capability to handle high-resolution images by dividing them into smaller, manageable units. Extensive experiments on 4 datasets confirmed that Sleight is superior to 5 state-of-the-art data-reconstruction attacks under 5 respective detection methods. Sleight can handle large-scale and high-resolution data without being detected or mitigated by state-of-the-art data reconstruction defense methods. In contrast to baselines, Sleight can be directly applied to both FedAvg and FedSGD scenarios, underscoring the need for developers to devise new defenses against such vulnerabilities. We will open-source our code upon acceptance.
Xueluan Gong, Yuji Wang, Shuike Li, Mengyuan Sun 0001, Chen Chen 0115, Qian Wang 0002, Kwok-Yan Lam
IEEE Trans. Dependable Secur. Comput.1
2025 URVFL: Undetectable Data Reconstruction Attack on Vertical Federated Learning
Duanyi Yao, Xueluan Gong, Sizai Hou
NDSS3
2025 PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMs
Xueluan Gong, Fengyuan Ran, Chen Chen 0115, Yanjiao Chen, Qian Wang 0002, Kwok-Yan Lam
USENIX Security Symposium1
2025 An Effective and Resilient Backdoor Attack Framework Against Deep Neural Networks and Vision Transformers
abstract
Recent studies have revealed the vulnerability of Deep Neural Network (DNN) models to backdoor attacks. However, existing backdoor attacks arbitrarily set the trigger mask or use a randomly selected trigger, which restricts the effectiveness and robustness of the generated backdoor triggers. In this paper, we propose a novel attention-based mask generation methodology that searches for the optimal trigger shape and location. We also introduce a Quality-of-Experience (QoE) term into the loss function and carefully adjust the transparency value of the trigger in order to make the backdoored samples to be more natural. To further improve the prediction accuracy of the victim model, we propose an alternating retraining algorithm in the backdoor injection process. The victim model is retrained with mixed poisoned datasets in even iterations and with only benign samples in odd iterations. Besides, we launch the backdoor attack under a co-optimized attack framework that alternately optimizes the backdoor trigger and backdoored model to further improve the attack performance. Apart from DNN models, we also extend our proposed attack method against vision transformers. We evaluate our proposed method with extensive experiments on VGG-Flower, CIFAR-10, GTSRB, CIFAR-100, and ImageNette datasets. It is shown that we can increase the attack success rate by as much as 82% over baselines when the poison ratio is low and achieve a high QoE of the backdoored samples. Our proposed backdoor attack framework also showcases robustness against state-of-the-art backdoor defenses.
Xueluan Gong, Bowei Tian, Meng Xue 0001, Yuan Wu 0007, Yanjiao Chen, Qian Wang 0002
IEEE Trans. Dependable Secur. Comput.1
2025 Artemis: Defending Against Backdoor Attacks via Distribution Shift
abstract
Backdoor attacks can exploit vulnerabilities in the training process of Deep Neural Networks (DNNs), introducing hidden malicious functionality that can be activated by a specific input pattern. Existing defenses typically rely on the assumption of a significant difference between poisoned and clean samples. However, subtle differences in dynamic and low poisoning ratio attacks can conceal poisoning features, thereby evading defenses. In this work, we propose a novel backdoor defense approach calledArtemis, which utilizes distribution shifts to eliminate the discrepancy between poisoned and benign samples in the feature space. Additionally,Artemislearns from domain-invariant features after the shift. To further enhance the purification of backdoors in DNNs, we incorporate soft knowledge distillation intoArtemisto guide the alignment of features between the source dataset domain and the generated distribution-shift dataset domain. We extensively compare our proposed method with 5 state-of-the-art (SOTA) defensive techniques under 9 SOTA attacks on 4 datasets to demonstrate its effectiveness and robustness. Our results show that our method,Artemis, can successfully purify dynamic and low poisoning ratio backdoor attacks and outperform existing defenses by a significant margin. We release the code athttps://github.com/xmyun/Artemis.
Meng Xue 0001, Zhixian Wang, Qian Zhang 0001, Xueluan Gong, Yanjiao Chen
IEEE Trans. Dependable Secur. Comput.4
2025 Augmenting Model Extraction Attacks Against Disruption-Based Defenses
abstract
Existing research has demonstrated that deep neural networks are susceptible to model extraction attacks, where an attacker can construct a substitute model with similar functionality to the victim model by querying the black-box victim model. To counter such attacks, various disruption-based defenses have been proposed. These defenses disrupt the output results of queries before returning them to potential attackers. In this paper, we propose the first defense-penetrating model extraction attack framework, aimed at breaking disruption-based defense methods. Our proposed attack framework comprises two key modules: disruption detection and disruption recovery, which can be integrated into generic model extraction attacks. Specifically, the disruption detection module uses a novel meta-learning-based algorithm to infer the defense strategy employed by the defender, by learning the key differences between the distributions of disrupted and undisrupted query results. Once the defense method is inferred, the disruption recovery module is designed to restore clean query results from the disrupted query results, using a carefully-designed generative model. We conducted extensive experiments on 5 commonly-used datasets to evaluate the effectiveness of our proposed framework. The results demonstrate that the substitute model accuracy of current model extraction attacks can be significantly improved by up to 82.42%, even when faced with four state-of-the-art model extraction defenses. Moreover, our attack approach shows promising results in penetrating unknown defenses in real-world cloud service APIs hosted by Microsoft Azure and Face++.
Xueluan Gong, Shuaike Li, Yanjiao Chen, Rubin Wei, Qian Wang 0002, Kwok-Yan Lam
IEEE Trans. Inf. Forensics Secur.1
2025 FingerVib: Fortifying Acoustic-Based Authentication With Finger Vibration Biometric on Smartphone
abstract
Due to the widespread use of mobile devices, it is essential to authenticate users on mobile devices to prevent sensitive information leakage. Biometrics-based authentication is prevalent on smart devices to verify the legitimacy of users, but is vulnerable to replay attacks. In this paper, we propose to leverage the distinctive finger tap gesture during unlocking smartphone to establish a secure multi-factor authentication system, named FingerVib. Compared with other biometric-based authentication systems, FingerVib does not require users to remember any complicated information (e.g., hand gestures, doodles) and the working type is unobtrusive. When users unlock their phones by tapping, FingerVib utilizes the microphone to record the sound produced by fingers tapping on the phone and adopts IMU (Inertial Measurement Unit) to extract the vibration of users’ smartphones. One key contribution is that we model the inherent correlation between sounds and vibration signals. Specifically, FingerVib captures two novel reactions to describe how the individual’s contact palm modulates signals in two different domains. Based on these two responses, we develop a real-time noise-resistant unlocking activity detection algorithm, which allows accurate unlocking signal segmentation even if the two modalities are interfered. Further, we develop a modal fusion model where the model extracts cross-modal features and acquires inter-modal correlation features to ensure consistent performance of inference even when modalities are disturbed. In a user study with 41 participants, FingerVib achieves an authentication accuracy of 98.53% and an average performance of 1.36% FAR, 2.76% FRR and 2.72% EER against replay attacks and impersonation attacks. FingerVib’s fusion approach improves identification performance by roughly 9.7% and 11.6% over Wavocie and AUDIOIMU, respectively, within existing multi-modal fusion systems. Extensive experimental results demonstrate the effectiveness and robustness of FingerVib under various conditions.
Yuan Wu 0007, Shoudu Bai, Runmin Lv, Xueluan Gong, Yanjiao Chen
IEEE Trans. Inf. Forensics Secur.4
2024 Beowulf: Mitigating Model Extraction Attacks Via Reshaping Decision Regions
abstract
Machine Learning as a Service (MLaaS) enables resource-constrained users to access well-trained models through a publicly accessible Application Programming Interface (API) on a pay-per-query basis.Nevertheless, model owners may face the potential threats of model extraction attacks where malicious users replicate valuable commercial models based on query results.Existing defenses against model extraction attacks, however, either sacrifice prediction accuracy or fail to thwart more advanced attacks.In this paper, we propose a novel model extraction defense, dubbed Beowulf 1 , which draws inspiration from theoretical findings that models with complex and narrow decision regions are difficult to be reproduced.Rather than arbitrarily altering decision regions, which may jeopardize the predictive capacity of the victim model, we introduce a dummy class, carefully synthesized using both random and adversarial noises.The random noise broadens the coverage of the dummy class, and the adversarial noise impacts decision regions near decision boundaries with normal classes.To further improve the model utility, we propose to employ data augmentation methods to seamlessly integrate the dummy class and the normal classes.Extensive evaluations on CIFAR-10, GTSRB, CIFAR-100, and ImageNette datasets * Yanjiao Chen and Qian Wang are corresponding authors.
Xueluan Gong, Rubin Wei, Yuchen Sun 0003, Jiawen Peng, Yanjiao Chen, Qian Wang 0002
CCS1
2024 KerbNet: A QoE-Aware Kernel-Based Backdoor Attack Framework
abstract
Deep neural networks are vulnerable to backdoor attacks, where a specially-designed trigger will lead to misclassification of any benign samples. However, existing backdoor attacks usually impose conspicuous patch triggers on images, which are easily detected by humans and defense algorithms. Existing works on invisible triggers, however, either have reduced attack success rate or yield detectable patterns to visual inspections. In this paper, we proposeKerbNet, a kernel-based backdoor attack framework, which applies kernel operations to clean samples as the trigger to incur misclassification. The kernel-processed samples achieve a high attack success rate while appearing natural with high Quality-of-Experience (QoE). We carefully design the kernel trigger generation algorithm by exploiting the neural network structure to propagate the influence of the trigger to the target misclassification label under the QoE constraint. We conduct extensive experiments on five datasets, i.e., MNIST, GTSRB, CIFAR-10, CelebA, and ImageNette to evaluate the effectiveness and practicality ofKerbNetunder the impact of various factors, including neuron-residing layer, kernel size, base image, loss function, model structure, and so on. We also show that our proposed attacks can evade state-of-the-art defense strategies and visual inspections. Code will be available after publication.
Xueluan Gong, Yanjiao Chen, Huayang Huang, Weihan Kong, Chao Shen 0001, Qian Wang 0002
IEEE Trans. Dependable Secur. Comput.1
2024 Palette: Physically-Realizable Backdoor Attacks Against Video Recognition Models
abstract
Backdoor attacks have been widely studied for image classification tasks, but rarely investigated for video recognition tasks. In this paper, we explore the possibility of physically-realizable backdoor attacks against video recognition models. Different from existing works that directly apply image backdoor attacks to videos, i.e., patch a visible trigger to each frame of a video, we carefully take into consideration the temporal interactions among frames in a video. Our proposed video backdoor attack, namedPalette, features two special design choices. The first is to utilize natural-light-alike RGB offset as triggers rather than traditional patch triggers. Such triggers may be applied in the physical world through lighting without the need to modify video files. The second is to make the backdoored model more robust to temporal asynchronization between the trigger and the video samples by performing rolling operations during sample poisoning. Extensive experiments show thatPaletteoutperforms existing video backdoor attacks, especially in the physical world. It is shown thatPaletteis also resistant to backdoor defense methods. We will open-source our codes upon publication.
Xueluan Gong, Zheng Fang 0014, Bowen Li 0016, Tao Wang 0081, Yanjiao Chen, Qian Wang 0002
IEEE Trans. Dependable Secur. Comput.1
2024 Backdoor Attack With Sparse and Invisible Trigger
abstract
Deep neural networks (DNNs) are vulnerable to backdoor attacks, where the adversary manipulates a small portion of training data such that the victim model predicts normally on the benign samples but classifies the triggered samples as the target class. The backdoor attack is an emerging yet threatening training-phase threat, leading to serious risks in DNN-based applications. In this paper, we revisit the trigger patterns of existing backdoor attacks. We reveal that they are either visible or not sparse and therefore are not stealthy enough. More importantly, it is not feasible to simply combine existing methods to design an effective sparse and invisible backdoor attack. To address this problem, we formulate the trigger generation as a bi-level optimization problem with sparsity and invisibility constraints and propose an effective method to solve it. The proposed method is dubbed sparse and invisible backdoor attack (SIBA). We conduct extensive experiments on benchmark datasets under different settings, which verify the effectiveness of our attack and its resistance to existing backdoor defenses. The codes for reproducing main experiments are available athttps://github.com/YinghuaGao/SIBA.
Yinghua Gao, Yiming Li 0004, Xueluan Gong, Zhifeng Li 0001, Shutao Xia, Qian Wang 0002
IEEE Trans. Inf. Forensics Secur.3
2024 Ubi-AD: Towards Ubiquitous, Passive Alzheimer Detection using the Smartwatch
abstract
Alzheimer’s disease (AD) is an insidious and progressive neurodegenerative disease, and the annual relevant social cost for AD patients can reach about $1 trillion worldwide. Therefore, early diagnosis and treatment of AD play a vital role in slowing disease progression. However, existing detection methods for cognitive impairment cannot consistently screen the stage of AD. To tackle this challenge, we propose an AD detection system, Ubi-AD, which combines the features of multiple biomarkers to realize passive and accurate AD detection. Unlike existing work, Ubi-AD can passively recognize the AD digital biomarkers during daily smartwatch usage without interfering with the user. At the user end, Ubi-AD first extracts the non-speech sounds (pause words, such as em, ah), which contain no privacy-sensitive content. Then, Ubi-AD recognizes the user’s walking activity, dining activity, and sleep activity from daily activities. Ubi-AD analyzes these data from smartwatch and predicts the AD stages using a multi-modal fusion neural network at the cloud end. We evaluate our model on a collected dataset from 45 volunteers. As a result, Ubi-AD can reach a detection accuracy of 93.4%, which means that Ubi-AD can provide multiple effective biomarkers for ubiquitous and passive detection in daily life.
Yuan Wu 0007, Yanjiao Chen, Jian Zhang 0010, Xueluan Gong, Hongliang Bi
ACM Trans. Sens. Networks4
2023 Orion: Online Backdoor Sample Detection via Evolution Deviance
abstract
Widely-used DNN models are vulnerable to backdoor attacks, where the backdoored model is only triggered by specific inputs but can maintain a high prediction accuracy on benign samples. Existing backdoor input detection strategies rely on the assumption that benign and poisoned samples are separable in the feature representation of the model. However, such an assumption can be broken by advanced feature-hidden backdoor attacks. In this paper, we propose a novel detection framework, dubbed Orion (online backdoor sample detection via evolution deviance). Specifically, we analyze how predictions evolve during a forward pass and find deviations between the shallow and deep outputs of the backdoor inputs. By introducing side nets to track such evolution divergence, Orion eliminates the need for the assumption of latent separability. Additionally, we put forward a scheme to restore the original label of backdoor samples, enabling more robust predictions. Extensive experiments on six attacks, three datasets, and two architectures verify the effectiveness of Orion. It is shown that Orion outperforms state-of-the-art defenses and can identify feature-hidden attacks with an F1-score of 90%, compared to 40% for other detection schemes. Orion can also achieve 80% label recovery accuracy on basic backdoor attacks.
Huayang Huang, Qian Wang 0002, Xueluan Gong, Tao Wang 0081
IJCAI3
2023 D-DAE: Defense-Penetrating Model Extraction Attacks
abstract
Recent studies show that machine learning models are vulnerable to model extraction attacks, where the adversary builds a substitute model that achieves almost the same performance of a black-box victim model simply via querying the victim model. To defend against such attacks, a series of methods have been proposed to disrupt the query results before returning them to potential attackers, greatly degrading the performance of existing model extraction attacks.In this paper, we make the first attempt to develop a defense-penetrating model extraction attack framework, named D-DAE, which aims to break disruption-based defenses. The linchpins of D-DAE are the design of two modules, i.e., disruption detection and disruption recovery, which can be integrated with generic model extraction attacks. More specifically, after obtaining query results from the victim model, the disruption detection module infers the defense mechanism adopted by the defender. We design a meta-learning-based disruption detection algorithm for learning the fundamental differences between the distributions of disrupted and undisrupted query results. The algorithm features a good generalization property even if we have no access to the original training dataset of the victim model. Given the detected defense mechanism, the disruption recovery module tries to restore a clean query result from the disrupted query result with well-designed generative models. Our extensive evaluations on MNIST, FashionMNIST, CIFAR-10, GTSRB, and ImageNette datasets demonstrate that D-DAE can enhance the substitute model accuracy of the existing model extraction attacks by as much as 82.24% in the face of 4 state-of-the-art defenses and combinations of multiple defenses. We also verify the effectiveness of D-DAE in penetrating unknown defenses in real-world APIs hosted by Microsoft Azure and Face++.
Yanjiao Chen, Xueluan Gong, Jianshuo Dong, Meng Xue 0001
SP3
2023 Redeem Myself: Purifying Backdoors in Deep Learning Models using Self Attention Distillation
abstract
Recent works have revealed the vulnerability of deep neural networks to backdoor attacks, where a backdoored model orchestrates targeted or untargeted misclassification when activated by a trigger. A line of purification methods (e.g., fine-pruning, neural attention transfer, MCR [69]) have been proposed to remove the backdoor in a model. However, they either fail to reduce the attack success rate of more advanced backdoor attacks or largely degrade the prediction capacity of the model for clean samples. In this paper, we put forward a new purification defense framework, dubbed SAGE, which utilizes self-attention distillation to purge models of backdoors. Unlike traditional attention transfer mechanisms that require a teacher model to supervise the distillation process, SAGE can realize self-purification with a small number of clean samples. To enhance the defense performance, we further propose a dynamic learning rate adjustment strategy that carefully tracks the prediction accuracy of clean samples to guide the learning rate adjustment. We compare the defense performance of SAGE with 6 state-of-the-art defense approaches against 8 backdoor attacks on 4 datasets. It is shown that SAGE can reduce the attack success rate by as much as 90% with less than 3% decrease in prediction accuracy for clean samples. We will open-source our codes upon publication.
Xueluan Gong, Yanjiao Chen, Qian Wang 0002, Yuzhe Gu, Huayang Huang, Chao Shen 0001
SP1
2023 Catch You and I Can: Revealing Source Voiceprint Against Voice Conversion
Jiangyi Deng, Yanjiao Chen, Yinan Zhong, Qianhao Miao, Xueluan Gong, Wenyuan Xu 0001
USENIX Security Symposium5
2023 NetGuard: Protecting Commercial Web APIs from Model Inversion Attacks using GAN-generated Fake Samples
abstract
Recently more and more cloud service providers (e.g., Microsoft, Google, and Amazon) have commercialized their well-trained deep learning models by providing limited access via web API interfaces. However, it is shown that these APIs are susceptible to model inversion attacks, where attackers can recover the training data with high fidelity, which may cause serious privacy leakage.Existing defenses against model inversion attacks, however, hinder the model performance and are ineffective for more advanced attacks, e.g., Mirror [4]. In this paper, we proposed NetGuard, a novel utility-aware defense methodology against model inversion attacks (MIAs). Unlike previous works that perturb prediction outputs of the victim model, we propose to mislead the MIA effort by inserting engineered fake samples during the training process. A generative adversarial network (GAN) is carefully built to construct fake training samples to mislead the attack model without degrading the performance of the victim model. Besides, we adopt continual learning to further improve the utility of the victim model. Extensive experiments on CelebA, VGG-Face, and VGG-Face2 datasets show that NetGuard is superior to existing defenses, including DP [37] and Ad-mi [32] on state-of-the-art model inversion attacks, i.e., DMI [8], Mirror [4], Privacy [12], and Alignment [34].
Xueluan Gong, Yanjiao Chen, Qian Wang 0002, Cong Wang 0001, Chao Shen 0001
WWW1
2023 MARNet: Backdoor Attacks Against Cooperative Multi-Agent Reinforcement Learning
abstract
Recent works have revealed that backdoor attacks against Deep Reinforcement Learning (DRL) could lead to abnormal action selections of the agent, which may result in failure or even catastrophe in crucial decision processes. However, existing attacks only consider single-agent reinforcement learning (RL) systems, in which the only agent can observe the global state and have full control of the decision process. In this article, we explore a new backdoor attack paradigm in cooperative multi-agent reinforcement learning (CMARL) scenarios, where a group of agents coordinate with each other to achieve a common goal, while each agent can only observe the local state. In the proposed MARNet attack framework, we carefully design a pipeline of trigger design, action poisoning, and reward hacking modules to accommodate the cooperative multi-agent settings. In particular, as only a subset of agents can observe the triggers in their local observations, we maneuver their actions to the worst actions suggested by an expert policy model. Since the global reward in CMARL is aggregated by individual rewards from all agents, we propose to modify the reward in a way that boosts the bad actions of poisoned agents (agents who observe the triggers) but mitigates the influence on non-poisoned agents. We conduct extensive experiments on three classical CMARL algorithms VDN, COMA, and QMIX, in two popular CMARL games Predator Prey and SMAC. The results show that the baselines extended from single-agent DRL backdoor attacks seldom work in CMARL problems while MARNet performs well by reducing the utility under attack by nearly 100%. We apply fine-tuning as a potential defense against MARNet and demonstrate that fine-tuning cannot entirely eliminate the effect of the attack.
Yanjiao Chen, Zhicong Zheng, Xueluan Gong
IEEE Trans. Dependable Secur. Comput.3
2023 Kaleidoscope: Physical Backdoor Attacks Against Deep Neural Networks With RGB Filters
abstract
Recent research has shown that deep neural networks are vulnerable to backdoor attacks. A carefully-designed backdoor trigger will mislead the victim model to misclassify any sample with the trigger to the target label. Nevertheless, existing works usually utilize visible triggers, such as a white square at the corner of the image, which are easily detected by human inspections. Current efforts on developing invisible triggers yield low attack success in the physical domain. In this paper, we propose Kaleidoscope, an RGB (red, green, and blue) filter-based backdoor attack method, which utilizes RGB filter operations as the backdoor trigger. To enhance the attack success rate, we design a novel model-dependent filter trigger generation algorithm. We also introduce two constraints in the loss function to make the backdoored samples more natural and less distorted. Extensive experiments on CIFAR-10, CIFAR-100, ImageNette, and VGG-Flower have demonstrated that RGB filter-processed samples not only achieve high attack success rate but also are unnoticeable to humans. It is shown that Kaleidoscope can reach an attack success rate of more than 84% in the physical world under different lighting intensities and shooting angles. Kaleidoscope is also shown to be robust to state-of-the-art backdoor defenses, such as spectral signature, STRIP, and MNTD.
Xueluan Gong, Yanjiao Chen, Meng Xue 0001, Qian Wang 0002, Chao Shen 0001
IEEE Trans. Dependable Secur. Comput.1
2023 A GAN-Based Defense Framework Against Model Inversion Attacks
abstract
With the development of deep learning, deep neural network (DNN)-based application have become an indispensable aspect of daily life. However, recent studies have shown that these well-trained DNN models are vulnerable to model inversion attacks (MIAs), where attackers can recover their training data with high fidelity. Although several defensive strategies have been proposed to mitigate the impact of such attacks, existing defenses will inevitably compromise the model performance and are ineffective against more sophisticated attacks, such as Mirror [5]. In this paper, we introduce a novel GAN-based defense approach against model inversion attacks. Unlike previous works that perturb the prediction vector of the model, we manipulate the training procedure of the victim model by incorporating carefully-designed GAN-based fake samples. We also adjust the loss of the inversed samples to inject misleading features into the protected label of the victim model. Additionally, we adopt the concept of continual learning to improve the utility of the model. Extensive experiments conducted on the CelebA, VGG-Face, and VGG-Face2 datasets demonstrate that our proposed method outperforms existing defenses against state-of-the-art model inversion attacks, including DMI [9], Mirror [5], Privacy [13], and AMI [42]. It is shown that our proposed method can also retain a high defense performance in black-box scenarios.
Xueluan Gong, Shuaike Li, Yanjiao Chen, Qian Wang 0002
IEEE Trans. Inf. Forensics Secur.1
2023 Data Poisoning Attacks in Internet-of-Vehicle Networks: Taxonomy, State-of-The-Art, and Future Directions
abstract
With the unprecedented development of deep learning, autonomous vehicles (AVs) have achieved tremendous progress nowadays. However, AV supported by DNN models is vulnerable to data poisoning attacks, hindering the large-scale application of autonomous driving. For example, by injecting carefully designed poisons into the training dataset of the DNN model in the traffic sign recognition system, the attacker can mislead the system to make targeted misclassification or cause a reduction in model classification accuracy. In this article, we conduct a thorough investigation of the state-of-the-art data poisoning attacks and defenses against AVs. According to whether the attacker needs to manipulate the data labeling process, we divide the state-of-the-art attack approaches into two categories, i.e., dirty-label attacks and clean-label attacks. We also differentiate the existing defense methods into two categories based on whether to modify the training data or the models, i.e., data-based defenses and model-based defenses. In addition to a detailed review of attacks and defenses in each category, we also give a qualitative comparison of the existing attacks and defenses. Besides, we provide a quantitative comparison of the existing attack and defense methods through experiments. Last but not least, we pinpoint several future directions for data poisoning attacks and defenses in AVs, providing possible ways for further research.
Yanjiao Chen, Xiaotian Zhu, Xueluan Gong, Xinjing Yi
IEEE Trans. Ind. Informatics3
2023 B3: Backdoor Attacks against Black-box Machine Learning Models
abstract
Backdoor attacks aim to inject backdoors to victim machine learning models during training time, such that the backdoored model maintains the prediction power of the original model towards clean inputs and misbehaves towards backdoored inputs with the trigger. The reason for backdoor attacks is that resource-limited users usually download sophisticated models from model zoos or query the models from MLaaS rather than training a model from scratch, thus a malicious third party has a chance to provide a backdoored model. In general, the more precious the model provided (i.e., models trained on rare datasets), the more popular it is with users. In this article, from a malicious model provider perspective, we propose a black-box backdoor attack, named B 3 , where neither the rare victim model (including the model architecture, parameters, and hyperparameters) nor the training data is available to the adversary. To facilitate backdoor attacks in the black-box scenario, we design a cost-effective model extraction method that leverages a carefully constructed query dataset to steal the functionality of the victim model with a limited budget. As the trigger is key to successful backdoor attacks, we develop a novel trigger generation algorithm that intensifies the bond between the trigger and the targeted misclassification label through the neuron with the highest impact on the targeted label. Extensive experiments have been conducted on various simulated deep learning models and the commercial API of Alibaba Cloud Compute Service. We demonstrate that B 3 has a high attack success rate and maintains high prediction accuracy for benign inputs. It is also shown that B 3 is robust against state-of-the-art defense strategies against backdoor attacks, such as model pruning and NC.
Xueluan Gong, Yanjiao Chen, Huayang Huang, Qian Wang 0002
ACM Trans. Priv. Secur.1
2022 ATTEQ-NN: Attention-based QoE-aware Evasive Backdoor Attacks
Xueluan Gong, Yanjiao Chen, Jianshuo Dong, Qian Wang 0002
NDSS1
2022 MPCN-RP: A Routing Protocol for Blockchain-Based Multi-Charge Payment Channel Networks
abstract
Blockchain-based cryptocurrencies are severely limited in transaction throughput and latency due to the need to seek consensus among all peers of the network. A promising solution to this issue is payment channels, which allow unlimited numbers of atomic and trust-free payments between two peers without exhausting the resources of the blockchain. A linked payment channel network enables payments between two peers without direct channels through a series of intermediate nodes that forward and charge for the transactions. However, the charging strategies of intermediate nodes vary with different payment channel networks. Existing works do not yet have a complete routing algorithm to provide the most economical path for users in a multi-charge payment channel network. In this work, we propose MPCN-RP, a general routing protocol for payment channel networks with multiple charges. Our extensive experimental results on both simulated and real payment channel networks show that MPCN-RP significantly outperforms the baseline algorithms in terms of time and fees.
Yanjiao Chen, Yuyang Ran, Jingyue Zhou, Jian Zhang 0010, Xueluan Gong
IEEE Trans. Netw. Serv. Manag.5
2021 InverseNet: Augmenting Model Extraction Attacks with Training Data Inversion
abstract
Cloud service providers, including Google, Amazon, and Alibaba, have now launched machine-learning-as-a-service (MLaaS) platforms, allowing clients to access sophisticated cloud-based machine learning models via APIs. Unfortunately, however, the commercial value of these models makes them alluring targets for theft, and their strategic position as part of the IT infrastructure of many companies makes them an enticing springboard for conducting further adversarial attacks. In this paper, we put forth a novel and effective attack strategy, dubbed InverseNet, that steals the functionality of black-box cloud-based models with only a small number of queries. The crux of the innovation is that, unlike existing model extraction attacks that rely on public datasets or adversarial samples, InverseNet constructs inversed training samples to increase the similarity between the extracted substitute model and the victim model. Further, only a small number of data samples with high confidence scores (rather than an entire dataset) are used to reconstruct the inversed dataset, which substantially reduces the attack cost. Extensive experiments conducted on three simulated victim models and Alibaba Cloud's commercially-available API demonstrate that InverseNet yields a model with significantly greater functional similarity to the victim model than the current state-of-the-art attacks at a substantially lower query budget.
Xueluan Gong, Yanjiao Chen, Guanghao Mei, Qian Wang 0002
IJCAI1
2021 Defense-Resistant Backdoor Attacks Against Deep Neural Networks in Outsourced Cloud Environment
abstract
The time and monetary costs of training sophisticated deep neural networks are exorbitant, which motivates resource-limited users to outsource the training process to the cloud. Concerning that an untrustworthy cloud service provider may inject backdoors to the returned model, the user can leverage state-of-the-art defense strategies to examine the model. In this paper, we aim to develop robust backdoor attacks (named RobNet) that can evade existing defense strategies from the standpoint of malicious cloud providers. The key rationale is to diversify the triggers and strengthen the model structure so that the backdoor is hard to be detected or removed. To attain this objective, we refine the trigger generation algorithm by selecting the neuron(s) with large weights and activations and then computing the triggers via gradient descent to maximize the value of the selected neuron(s). In stark contrast to existing works that fix the trigger location, we design a multi-location patching method to make the model less sensitive to mild displacement of triggers in real attacks. Furthermore, we extend the attack space by proposing multi-trigger backdoor attacks that can misclassify inputs with different triggers into the same or different target label(s). We evaluate the performance of RobNet on MNIST, GTSRB, and CIFAR-10 datasets, against four representative defense strategies Pruning, NeuralCleanse, Strip, and ABS. The comparison with two state-of-the-art baselines BadNets and Hidden Backdoors demonstrates that RobNet achieves higher attack success rate and is more resistant to potential defenses.
Xueluan Gong, Yanjiao Chen, Qian Wang 0002, Huayang Huang, Lingshuo Meng, Chao Shen 0001, Qian Zhang 0001
IEEE J. Sel. Areas Commun.1
2020 Crowdcaching: Incentivizing D2D-Enabled Caching via Coalitional Game for IoT
abstract
With the explosion of the Internet-of-Things (IoT) technology, numerous IoT terminal devices generate tremendous traffic. Device-to-device (D2D)-enabled caching can greatly relieve the pressure of massive resource-limited terminal devices in the IoT network. This article proposes a novel distributed framework, termed crowdcaching, which motivates selective file caching and cooperative file sharing among terminal devices via short-range (e.g., D2D) communications. After modeling file preference distributions and local connectivities, we optimize the caching strategy for any given coalition of cooperative users to minimize their total delay cost. In particular, if users have homogeneous file preferences and local connectivities, we can mathematically define a popularity index, according to which files are chosen to be cached in user devices. In a more general setting where users have heterogeneous file preferences and local connectivities, we propose a greedy algorithm of low complexity to determine the optimal caching strategy. Based on the cooperative caching strategy for any given coalition, we further investigate users' incentive to form crowdcaching coalitions through the coalitional game theory and propose a distributed algorithm to yield a stable coalition formulation. The simulation results show that crowdcaching can effectively reduce the average delay cost of users by as much as 45.64%.
Yanjiao Chen, Xueluan Gong, Runmin Ou, Lingjie Duan, Qian Zhang 0001
IEEE Internet Things J.2