Local differential privacy (LDP) provides a way for an untrusted data collector to aggregate users' data without violating their privacy. Various privacy-preserving data analysis tasks have been studied under the protection of LDP, such as frequency estimation, frequent itemset mining, and machine learning. Despite its privacy-preserving properties, recent research has demonstrated the vulnerability of certain LDP protocols to data poisoning attacks. However, existing data poisoning attacks are focused on basic statistics under LDP, such as frequency estimation and mean/variance estimation. As an important data analysis task, the security of LDP frequent itemset mining has yet to be thoroughly examined. In this paper, we aim to address this issue by presenting novel and practical data poisoning attacks against LDP frequent itemset mining protocols. By introducing a unified attack framework with composable attack operations, our data poisoning attack can successfully manipulate the state-of-the-art LDP frequent itemset mining protocols and has the potential to be adapted to other protocols with similar structures. We conduct extensive experiments on three datasets to compare the proposed attack with four baseline attacks. The results demonstrate the severity of the threat and the effectiveness of the proposed attack.

Model publishing and deployment are essential for artificial intelligence applications. A major challenge in model publishing is efficiently distributing the models in a scalable way without violating the privacy of sensitive data. With the wide adoption of machine learning techniques, the privacy concern has also drawn much attraction. Differential privacy has become an important notion for privacy protection and is popular in private learning. However, it may bring much accuracy loss to fulfill data privacy. In addition, the private models are also hard to train in terms of convergence, which makes the existing approaches not scalable for private model publishing. This paper proposes a model publishing framework that provides a novel way to train privacy-preserving machine learning models with fast convergence and a lower privacy budget. By incorporating the concept of iterative machine teaching and the techniques in differential privacy, we have explored a way to privately select more suitable examples in the training process for achieving good accuracy with fewer iterations. Our analysis shows the privacy and convergence performance of the proposed method, and extensive experiments have been performed on real-world datasets to demonstrate its effectiveness.

Vertical federated learning (VFL) is being used more and more widely in industry. One of its most common application scenarios is a two-party setting: a participant (i.e., the host), who exclusively owns the labels but possesses insufficient number of features, wants to improve its model performance by combining features from another participant (i.e., the client) of a different business group. The best deep ML architecture suits for this scenario is considered to be Split Neural Network (SplitNN), in which each participant runs a self-defined bottom model to learn the hidden representations (i.e., the local embeddings) of its local data and then forwards them to the host, who runs a top model to aggregate both the local embeddings to produce the final predicts. In this paper, we assume the client is malicious and demonstrate that she/he could inject a stealthy backdoor into the top model during the training to misclassify any sample to a pre-selected target class with a high probability by just replacing its local embedding with a special trigger vector regardless of the host-side embedding. This task is non-trivial because existing data poison attacks for backdoor injection in traditional models usually require to modify the labels of a set of trigger-tagged samples of non-target classes, which is impossible here as the client has no rights to access or modify the labels exclusively owned by the host. Targeting this challenge, we propose a SplitNN-dedicated data poison attack which does not require to modify any labels but just replaces the local embeddings of a very small number of target-class samples with a carefully constructed trigger vector during training. The experiments on four datasets show that our attack can achieve an attack rate as high as 94%, while bringing negligible side-effects to the model accuracy. Moreover, it is stealthy enough to resist various anomaly detection methods.