Federated learning (FL) enables resource-constrained node devices to learn a shared model while keeping the training data local. Since recent research has demonstrated multiple privacy leakage attacks in FL, e.g., gradient inference attacks and membership inference attacks, differential privacy (DP) is applied to serve as one of the most effective privacy protection mechanisms. Despite the benefit DP brings, we observe that the introduction of DP also brings random changes to client updates, which will affect the robust aggregation algorithms. We reveal a novel poisoning attack under the cover of DP, named the DP-Poison attack in FL. Specifically, the DP-Poison attack is designed to achieve four goals: (1) maintaining the main task performance; (2) launching a successful attack; (3) escaping the robust aggregation algorithms in FL; and (4) keeping the effectiveness of DP privacy protection. To achieve these goals, we design multiple optimization goals to generate DP noise through a genetic algorithm. The optimization ensures that while the benign updates change randomly, the malicious updates can change toward the global model after adding the DP noise, so that it is easier to be accepted by the robust aggregation algorithms. Extensive experiments show that DP-Poison achieves a nearly 100% attack success rate while maintaining the proposed four goals.

With the development of deep learning processors and accelerators, deep learning models have been widely deployed on edge devices as part of the Internet of Things. Edge device models are generally considered as valuable intellectual properties that are worth for careful protection. Unfortunately, these models have a great risk of being stolen or illegally copied. The existing model protections using encryption algorithms are suffered from high computation overhead which is not practical due to the limited computing capacity on edge devices. In this work, we propose a light-weight, practical, and general Edge device model Protection method at neuron level, denoted as EdgePro. Specifically, we select several neurons as authorization neurons and set their activation values to locking values and scale the neuron outputs during training, where the authorization neurons, locking value, and scale factor together form the “passwords”. Then, we design lock training to implement model property protection through alternately locking and releasing, which correspond to model performance preservation and encryption, respectively. EdgePro protects the model by ensuring it can only work correctly when the “passwords” are met, at the cost of encrypting and storing the information of the “passwords” instead of the whole model. Extensive experimental results indicate that EdgePro can work well on the task of protecting models on different datasets. The inference time increase of EdgePro is only 60% of state-of-the-art methods, and the accuracy loss is less than 1%. Additionally, EdgePro is robust against adaptive attacks including fine-tuning, reverse engineering, and pruning, which makes it more practical in real-world applications. EdgePro is also open sourced to facilitate future research:https://github.com/Leon022/EdgePro.

The backdoor attacks have posed a severe threat to deep neural networks (DNNs). Online training platforms and third-party model training providers are more vulnerable to backdoor attacks due to uncontrollable data sources, untrusted developers or unmonitorable training processes. Researchers have proposed to detect the backdoor in the well-trained models, and then remove them by some mitigation techniques, e.g., retraining and pruning. However, they are still limited from two aspects: (i) real-time - they cannot detect in time at the beginning of training due to their reliance on well-trained models; (ii) mitigation effect - the later discovery of backdoors usually leads to 1) deeper backdoors, 2) less effective mitigation, and 3) greater costs. To address these challenges, we rethink the evolution of the backdoor, and intend to cope with backdoors along with the online training process, that is to detect the backdoors sooner rather than later. We propose BackdoorTracer, a novel framework that detects the backdoor in the training phase. BackdoorTracer constructs the model into an equivalent graph based on the activated neural path during training, thereby detecting the backdoor through multiple graph metrics. BackdoorTracer can incorporate any existing backdoor mitigation approaches that require accessing training to stop the impact of backdoors as soon as possible. It differs from previous works in several key aspects: (i) lightweight - BackdoorTracer is independent of the training process, and thus it has little negative impact on the training efficiency and testing accuracy; (ii) generalizable - it works different modalities of data, models and different backdoor attacks. BackdoorTracer outperforms the state-of-the-art (SOTA) detection approaches in experiments on 5 modes, 10 models and 9 backdoor attack scenarios. Compared with the existing 5 backdoor detection methods, our method can detect backdoors earlier ($\sim ~1.5$epochs) and higher detection rate (~ +10%), effectively improving the effectiveness of backdoor defense (ASR. ~ -78%, ACC. +47%). Finally, we make BackdoorTracer a plug-and-play backdoor detector, which enables real-time backdoor tracing in the training phase.

Deep neural networks (DNNs) have demonstrated their outperformance in various domains. However, it raises a social concern whether DNNs can produce reliable and fair decisions especially when they are applied to sensitive domains involving valuable resource allocation, such as education, loan, and employment. It is crucial to conduct fairness testing before DNNs are reliably deployed to such sensitive domains, i.e., generating as many instances as possible to uncover fairness violations. However, the existing testing methods are still limited from three aspects: interpretability, performance, and generalizability. To overcome the challenges, we propose NeuronFair, a new DNN fairness testing framework that differs from previous work in several key aspects: (1) interpretable - it quantitatively interprets DNNs' fairness violations for the biased decision; (2) effective - it uses the interpretation results to guide the generation of more diverse instances in less time; (3) generic - it can handle both structured and unstructured data. Extensive evaluations across 7 datasets and the corresponding DNNs demonstrate NeuronFair's superior performance. For instance, on structured datasets, it generates much more instances (~ ×5.84) and saves more time (with an average speedup of 534.56%) compared with the state-of-the-art methods. Besides, the instances of NeuronFair can also be leveraged to improve the fairness of the biased DNNs, which helps build more fair and trustworthy deep learning systems. The code of NeuronFair is open-sourced at https://github.com/haibinzheng/NeuronFair.

Strong passwords are fundamental to the security of password-based user authentication systems. In the recent years, much effort has been made to evaluate the password strength or to generate strong passwords. Unfortunately, the usability or memorability of the strong passwords has been largely neglected. In this article, we aim to bridge the gap between strong password generation and the usability of strong passwords. We propose to automatically generate textual password mnemonics, i.e., natural language sentences, which are intended to help users better memorize passwords. We introduceDeepMnemonic, a deep attentive encoder-decoder framework which takes a password as input and then automatically generates a mnemonic sentence for the password. We conduct extensive experiments to evaluate DeepMnemonic on the real-world data sets. The experimental results demonstrate that DeepMnemonic outperforms a well-known baseline for generating semantically meaningful mnemonic sentences. Moreover, the user study further validates that the generated mnemonic sentences by DeepMnemonic are useful in helping users memorize strong passwords.