Pre-trained Language Models (PLM) have enabled a cost-effective approach to handling various downstream applications via Parameter-Efficient-Fine-Tuning (PEFT) techniques. In this context, service providers have introduced a popular fine-tuning-based product service known as Model-as-a-Service (MaaS). This service offers users access to extensive PLMs and training resources. With MaaS, users can fine-tune, deploy, and utilize their customized models seamlessly, leveraging a one-stop platform that allows them to work with their private datasets efficiently. However, this service paradigm has recently been exposed to the possibility of leaking user private data. To this end, we identify the data privacy leakage risks in MaaS-based PEFT and propose a Split-and-Privatize (SAP) framework, mitigating the privacy leakage by integrating split learning and differential privacy into MaaS PEFT. Furthermore, we propose Contributing-Token-Identification (CTI), a novel method to balance model utility degradation and privacy leakage. As a result, the proposed framework is comprehensively evaluated, demonstrating a 65% improvement in empirical privacy with only a 1% degradation in model performance on the Stanford Sentiment Treebank dataset, outperforming existing state-of-the-art baselines.

Federated learning (FL) has attracted widespread attention in the Internet of Things domain recently. With FL, multiple distributed devices can cooperatively train a global model by transmitting model updates without disclosing the original data. However, the distributed nature of FL makes it vulnerable to data poisoning attacks. In practice, malicious clients can launch the label-flipping attack (LFA) by simply tampering with the labels of local data, thus causing the global model to misclassify the samples of a selected class as the target class. Although some defense mechanisms have been proposed, they rely on specific assumptions about data distribution, and their performance degrades significantly when the data on clients are non-IID. Besides, most existing methods require clients to upload model updates in plaintext so that the server can identify and remove the malicious updates. But, direct transmission of model updates may still reveal private information. Considering these issues, we develop a label-flipping-robust and privacy-preserving FL (LFR-PPFL) algorithm, which is applicable to both independent and identically distributed (IID) and non-IID data. We first propose a detection method based on temporal analysis on cosine similarity to distinguish malicious clients from benign clients. Then, we propose a privacy-preserving computation protocol based on homomorphic encryption to implement this detection method and perform federated aggregation while protecting the privacy of clients. Besides, a detailed theoretical analysis is given to demonstrate the privacy guarantee of the proposed protocol. Experimental results on real-world data sets show that the proposed algorithm can effectively defend against LFAs under various data distributions.

Federated learning (FL), which enables multiple distributed devices (clients) to collaboratively train a global model without transmitting their private data, has attracted much attention in the Internet of Things (IoT) domain. Compared with centralized learning, FL has obvious privacy advantages because it can protect the clients’ raw data from direct access by adversaries. Furthermore, to prevent the adversaries from inferring private information from the transmitted parameters, several FL algorithms based on differential privacy (DP) have been proposed, where the clients add artificial noise to their local parameters for privacy protection. Nevertheless, the added noise would disrupt the learning process and degrade the performance of the trained model. Considering this, in this article, we develop a performance-enhanced DP-based FL (PEDPFL) algorithm, where a classifier-perturbation regularization method is proposed to improve the robustness of the trained model against DP-injected noise. We derive the theoretical privacy and convergence analysis of the proposed algorithm, and also demonstrate the influence of some hyperparameters on the convergence performance. Simulation results on real-world data sets show that the proposed algorithm has better classification performance than the existing DP-based FL algorithms at the same level of privacy protection, and thus, it is more applicable to IoT applications.

In the era of Big Data, as vast amounts of data are collected and shared among collaborators or uploaded to the Internet, the attacks on data privacy become more and more serious. Compressive privacy (CP) is a kind of privacy-preserving dimension-reduced projection schemes such that the projected data can be well used for the intended utility task but not for malicious applications. Nevertheless, most of existing CP approaches belong to centralized processing, which are not applicable to the cases that data is dispersedly collected/stored at distributed nodes and cannot be centralized to one node for processing due to various reasons. To tackle this problem, we propose a distributed differential utility/cost analysis (dDUCA), in which each node in the network is only allowed to exchange and combine the compressive-and-lossy projection matrix with its one-hop neighbors. Using the projection matrix, the classification of the projected data is performed. Experiments on several datasets confirm the effectiveness of the proposed method in terms of both privacy protection and utility retention.