Federated learning (FL) has emerged as an ideal privacy-preserving learning technique which can train a global model in a collaborative way while preserving the private data in the local. However, recent advances have demonstrated that FL is still vulnerable to inference attacks, such as reconstruction attack and membership inference. Among these attacks, the property inference attack, aiming to infer properties of the training data that are irrelevant with the learning objective, has not received too much attention while resulting in severe privacy leakage. Existing property inference attack approaches either cannot achieve satisfactory performance when the global model has converged or under dynamic FL where participants can drop in and drop out freely. In this paper, we propose a novel poisoning-assisted property inference attack (PAPI-attack) against FL. The key insight is that there exists underlying discriminative ability in the periodic model updates, which reflects the change of the data distribution, especially the occurrence of the sensitive property. Thus, a binary attack model can be constructed by a malicious participant for inferring the unintended information. More importantly, we present a property-specific poisoning mechanism by modifying the label of training data from the adversary to distort the decision boundary of shared (global) model in FL. Consequently, benign participants are induced to disclose more information about the sensitive property. Extensive experiments on real-world datasets demonstrate that PAPI-attack outperforms the state-of-the-art property inference attacks against FL.

Deep neural networks (DNNs) can be utilized maliciously for compromising the privacy stored in electronic devices, e.g., identifying the images stored in a mobile phone connected to the Internet of Things (IoT). However, recent studies demonstrated that DNNs are vulnerable to adversarial examples, which are artificially designed perturbations in the original samples for misleading DNNs. Adversarial examples can be used to protect the DNN-based privacy leakage in mobile phones by replacing the photos with adversarial examples. To avoid affecting the normal use of photos, the adversarial examples need to be highly similar to original images. To handle a large number of photos stored in the devices at a proper time, the time efficiency of a method needs to be high enough. Previous methods cannot do well on both sides. In this article, we propose a broad class of selective gradient sign iterative algorithms to make adversarial examples useful in protecting the privacy of photos in IoT devices. By neglecting the unimportant image pixels in the iterative process of attacks according to the sort of first-order partial derivative, we control the optimization direction meticulously to reduce image distortions of adversarial examples without leveraging high time-consuming tricks. Extensive experimental results show that the proposed methods successfully fool the neural network classifiers for the image classification task with a small change in the visual effects and consume little calculating time simultaneously.

Recent studies demonstrated that deep neural networks (DNNs) are vulnerable to adversarial examples, which would seriously threaten security-sensitive applications. Existing works synthesized the adversarial examples by perturbing the original/benign images by leveraging the$\mathcal {L}_p$-norm to penalize the perturbations, which restricts the pixel-wise distance between the adversarial images and correspondingly benign images. However, they added perturbations globally to the benign images without explicitly considering their content/spacial structure, resulting in noticeable artifacts especially in those originally clean regions, e.g., sky and smooth surface. In this paper, we propose an invisible adversarial attack, which synthesizes adversarial examples that are visually indistinguishable from benign ones. We adaptively distribute the perturbation according to human sensitivity to a local stimulus in the benign image, i.e., the higher insensitivity, the more perturbation. Two types of adaptive adversarial attacks are proposed: 1) coarse-grained and 2) fine-grained. The former conducts$\mathcal {L}_p$-norm regularized by the novel spatial constraints, which utilizes the rich information of the cluttered regions to mask perturbation. The latter, called Just Noticeable Distortion (JND)-based adversarial attack, utilizes the proposed JND$_p$metric for better measuring the perceptual similarity, and adaptively sets penalty by weighting the pixel-wise perceptual redundancy of an image. We conduct extensive experiments on the MNIST, CIFAR-10 and ImageNet datasets and a comprehensive user study with 50 participants. The experimental results demonstrate that JND$_p$is a better metric for measuring the perceptual similarity than$\mathcal {L}_p$-norm, and the proposed adaptive adversarial attacks can synthesize indistinguishable adversarial examples from benign ones and outperform the state-of-the-art methods.

The massive photos shared through the social networks nowadays, e.g., Facebook and Instagram, have aided malicious entities to snoop private information, especially by utilizing deep neural networks (DNNs) to learn from those personal photos. To protect photo privacy against DNNs, recent advances adopting adversarial examples could successfully fool DNNs. However, they are sensitive to those image compression methods that are commonly used on social networks to reduce transmission bandwidth or storage space. A recent work proposed to resist JPEG compression, while the compression methods adopted in social networks are black boxes, and variation of compression methods would significantly degrade the resistance.

Federated learning has emerged as an advanced privacy-preserving learning technique for mobile edge computing, where the model is trained in a decentralized manner by the clients, preventing the server from directly accessing those private data from the clients. This learning mechanism significantly challenges the attack from the server side. Although the state-of-the-art attacking techniques that incorporated the advance of Generative adversarial networks (GANs) could construct class representatives of the global data distribution among all clients, it is still challenging to distinguishably attack a specific client (i.e., user-level privacy leakage), which is a stronger privacy threat to precisely recover the private data from a specific client. To analyze the privacy leakage of federated learning, this paper gives the first attempt to explore user-level privacy leakage by the attack from a malicious server. We propose a framework incorporating GAN with a multi-task discriminator, called multi-task GAN - Auxiliary Identification (mGAN-AI), which simultaneously discriminates category, reality, and client identity of input samples. The novel discrimination on client identity enables the generator to recover user specified private data. Unlike existing works interfering the federated learning process, the proposed method works “invisibly” on the server side. Furthermore, considering the anonymization strategy for mitigating mGAN-AI, we propose a beforehand linkability attack which re-identifies the anonymized updates by associating the client representatives. A novel siamese network fusing the identification and verification models is developed for measuring the similarity of representatives. The experimental results demonstrate the effectiveness of the proposed approaches and the superior to the state-of-the-art.

Person re-identification (re-ID) is the task of matching person images across camera views, which plays an important role in surveillance and security applications. Inspired by great progress of deep learning, deep re-ID models began to be popular and gained state-of-the-art performance. However, recent works found that deep neural networks (DNNs) are vulnerable to adversarial examples, posing potential threats to DNNs based applications. This phenomenon throws a serious question about whether deep re-ID based systems are vulnerable to adversarial attacks. In this paper, we take the first attempt to implement robust physical-world attacks against deep re-ID. We propose a novel attack algorithm, called advPattern, for generating adversarial patterns on clothes, which learns the variations of image pairs across cameras to pull closer the image features from the same camera, while pushing features from different cameras farther. By wearing our crafted “invisible cloak”, an adversary can evade person search, or impersonate a target person to fool deep re-ID models in physical world. We evaluate the effectiveness of our transformable patterns on adversaries' clothes with Market1501 and our established PRCS dataset. The experimental results show that the rank-1 accuracy of re-ID models for matching the adversary decreases from 87.9% to 27.1% under Evading Attack. Furthermore, the adversary can impersonate a target person with 47.1% rank-1 accuracy and 67.9% mAP under Impersonation Attack. The results demonstrate that deep re-ID systems are vulnerable to our physical attacks.

Federated learning, i.e., a mobile edge computing framework for deep learning, is a recent advance in privacy-preserving machine learning, where the model is trained in a decentralized manner by the clients, i.e., data curators, preventing the server from directly accessing those private data from the clients. This learning mechanism significantly challenges the attack from the server side. Although the state-of-the-art attacking techniques that incorporated the advance of Generative adversarial networks (GANs) could construct class representatives of the global data distribution among all clients, it is still challenging to distinguishably attack a specific client (i.e., user-level privacy leakage), which is a stronger privacy threat to precisely recover the private data from a specific client. This paper gives the first attempt to explore user-level privacy leakage against the federated learning by the attack from a malicious server. We propose a framework incorporating GAN with a multi-task discriminator, which simultaneously discriminates category, reality, and client identity of input samples. The novel discrimination on client identity enables the generator to recover user specified private data. Unlike existing works that tend to interfere the training process of the federated learning, the proposed method works “invisibly” on the server side. The experimental results demonstrate the effectiveness of the proposed attacking approach and the superior to the state-of-the-art.