Adversarial phenomena have been widely observed in machine learning (ML) systems, especially those using deep neural networks. These phenomena describe situations where ML systems may produce predictions that are inconsistent and incomprehensible to humans in certain specific cases. Such behavior poses a serious security threat to the practical application of ML systems. To exploit this vulnerability, several advanced attack paradigms have been developed, mainly including backdoor attacks, weight attacks, and adversarial examples. For each individual attack paradigm, various defense mechanisms have been proposed to enhance the robustness of models against the corresponding attacks. However, due to the independence and diversity of these defense paradigms, it is challenging to assess the overall robustness of an ML system against different attack paradigms. This survey aims to provide a systematic review of all existing defense paradigms from a unified lifecycle perspective. Specifically, we decompose a complete ML system into five stages: pre-training, training, post-training, deployment, and inference. We then present a clear taxonomy to categorize representative defense methods at each stage. The unified perspective and taxonomy not only help us analyze defense mechanisms but also enable us to understand the connections and differences among different defense paradigms. It inspires future research to develop more advanced and comprehensive defense strategies.

This work studies the task of poisoned sample detection for defending against data poisoning based backdoor attacks. Its core challenge is finding a generalizable and discriminative metric to distinguish between clean and various types of poisoned samples (e.g., various triggers, various poisoning ratios). Inspired by a common phenomenon in backdoor attacks that the backdoored model tend to map significantly different poisoned and clean samples within the target class to similar activation areas, we introduce a novel perspective of the circular distribution of the gradients w.r.t. sample activation, dubbed gradient circular distribution (GCD). And, we find two interesting observations based on GCD. One is that the GCD of samples in the target class is much more dispersed than that in the clean class. The other is that in the GCD of target class, poisoned and clean samples are clearly separated. Inspired by above two observations, we develop an innovative three-stage poisoned sample detection approach, called Activation Gradient based Poisoned sample Detection (AGPD). First, we calculate GCDs of all classes from the model trained on the untrustworthy dataset. Then, we identify the target class(es) based on the difference on GCD dispersion between target and clean classes. Last, we filter out poisoned samples within the identified target class(es) based on the clear separation between poisoned and clean samples. Extensive experiments under various settings of backdoor attacks demonstrate the superior detection performance of the proposed method to existing poisoned detection approaches according to sample activation-based metrics.

Backdoor learning is an emerging and vital topic for studying deep neural networks' vulnerability (DNNs). Many pioneering backdoor attack and defense methods are being proposed, successively or concurrently, in the status of a rapid arms race. However, we find that the evaluations of new methods are often unthorough to verify their claims and accurate performance, mainly due to the rapid development, diverse settings, and the difficulties of implementation and reproducibility. Without thorough evaluations and comparisons, it is not easy to track the current progress and design the future development roadmap of the literature. To alleviate this dilemma, we build a comprehensive benchmark of backdoor learning called BackdoorBench. It consists of an extensible modular-based codebase (currently including implementations of 8 state-of-the-art (SOTA) attacks and 9 SOTA defense algorithms) and a standardized protocol of complete backdoor learning. We also provide comprehensive evaluations of every pair of 8 attacks against 9 defenses, with 5 poisoning ratios, based on 5 models and 4 datasets, thus 8,000 pairs of evaluations in total. We present abundant analysis from different perspectives about these 8,000 evaluations, studying the effects of different factors in backdoor learning. All codes and evaluations of BackdoorBench are publicly available at https://backdoorbench.github.io.

Smart home is an indispensable part of Internet of Things(IoT) owing to the prompt development and application of smart devices. However, the data collected from smart homes usually need to be processed by a cloud server, which means there is a risk of leaking the privacy of users during the transmission. In this situation, edge computing is considered to be an ideal platform for smart home, which enable data to be processed at edge nodes. Unfortunately, because of unsecured Wi-Fi connection and smart devices, edge nodes also have the possibility to encounter malicious attacks. Hence, in this paper, we designed an intrusion detection system (IDS) to be deployed on edge nodes. We convert network traffic to images which are applied to train a convolutional neural network (CNN) to classify the categories of network traffic. Furthermore, Auxiliary Classifier Generative Adversarial Network (AC-GAN) is adopted to generate synthesized samples to expand the intrusion detection dataset. We experiment on the UNSW-NB15 dataset which contains substantial network traffic about the normal and anomalies. The proposed scheme is effective to minor categories of which precision could be improved 12%. Besides, the precision can reach 96% in binary classification about normal and anomaly.

With the rapid development of the mobile Internet and the increasing popularity of smart terminals, various mobile social applications are emerging. Medical data has become a valuable data asset and is being continuously explored and utilized, which has greatly promoted the improvement of the medical service level. However, publishing and using user data makes the user vulnerable to reasoning attacks. Due to the special nature of the medical field, medical data not only carries the health status of patients and medical process information but also involves individual sensitive information of a large number of patients. Allowing users to fully enjoy the advantages brought by social networks while ensuring security is an important issue that needs to be solved urgently in the era of big data. In this paper, we first provide an overview of social network data privacy risks and various types of attacks. Aiming at the privacy leakage of weighted social networks, we propose a privacy protection recommendation algorithm based on differential privacy. The algorithm utilizes the change of edge weight grouping, which greatly reduces the amount of calculation and satisfies the user's rapid response. It minimizes a privacy leak of user private data under data availability while supporting personalized rankings. Compared to the most advanced methods, this method protects users from reasoning attacks and reduces the distortion of ranking results caused by data confusion to ensure the accuracy of recommendations. Experiments on real-world datasets show that our framework can achieve more effective and lasting protection for user-sensitive data.

Deep learning algorithms, especially convolution neural networks, have attracted huge attention in the field of medical image analysis. A hospital could train a neural network to detect disease based on medical images possessing. However, the number of medical images would affect the results of training. If medical images of all hospitals are collected together, there's a risk of privacy leakage. In this paper, we apply collaborative deep learning to medical image analysis, which could help to improve the training effect. Besides, we also exploit differential privacy, the analytic Gaussian Mechanism, to prevent the leakage of information about medical images. We experiment on the Chest X-ray Images (Pneumonia) dataset. Results show that the analytic Gaussian Mechanism can protect the privacy of medical images effectively, while the influence on the results of training is small. The accuracy can be improved about 19\% via collaborative deep learning and can still remain about 18\% even when the analytic Gaussian Mechanism was used.