Federated learning (FL) is a distributed and privacy‐preserving machine learning paradigm that coordinates multiple clients to train a model while keeping the raw data localized. However, this traditional FL poses some challenges, including privacy risks, data heterogeneity, communication bottlenecks, and system heterogeneity issues. To tackle these challenges, knowledge distillation (KD) has been widely applied in FL since 2020. KD is a validated and efficacious model compression and enhancement algorithm. The core concept of KD involves facilitating knowledge transfer between models by exchanging logits at intermediate or output layers. These properties make KD an excellent solution for the long‐lasting challenges in FL. Up to now, there have been few reviews that summarize and analyze the current trend and methods for how KD can be applied in FL efficiently. This article aims to provide a comprehensive survey of KD‐based FL, focusing on addressing the above challenges. First, we provide an overview of KD‐based FL, including its motivation, basics, taxonomy, and a comparison with traditional FL and where KD should execute. We also analyze the critical factors in KD‐based FL in the Appendix, including teachers, knowledge, data, and methods. We discuss how KD can address the challenges in FL, including privacy protection, data heterogeneity, communication efficiency, and personalization. Finally, we discuss the challenges facing KD‐based FL algorithms and future research directions. We hope this survey can provide insights and guidance for researchers and practitioners in the FL area.

Abstract Machine learning is widely deployed in society, unleashing its power in a wide range of applications owing to the advent of big data. One emerging problem faced by machine learning is the discrimination from data, and such discrimination is reflected in the eventual decisions made by the algorithms. Recent study has proved that increasing the size of training (labeled) data will promote the fairness criteria with model performance being maintained. In this work, we aim to explore a more general case where quantities of unlabeled data are provided, indeed leading to a new form of learning paradigm, namely fair semi-supervised learning. Taking the popularity of graph-based approaches in semi-supervised learning, we study this problem both on conventional label propagation method and graph neural networks, where various fairness criteria can be flexibly integrated. Our developed algorithms are proved to be non-trivial extensions to the existing supervised models with fairness constraints. Extensive experiments on real-world datasets exhibit that our methods achieve a better trade-off between classification accuracy and fairness than the compared baselines.

With the ubiquity of social media, privacy leakage has become a urgent problemfor social media managers. Studying how the privacy information diffuses through social media has attracted much attention. As a prerequisite, modeling privacy information diffusion is important research. Current approaches for modeling information diffusion are not available for privacy information since they did not consider the propagation features of privacy information in social media. Thispaper discusses the problem of modeling privacy information in social media and its challenges. We first analyse the information diffusion paths in the basic parameters of complex network and the high-order structures. We find that the privacy information is different in propagation features and the size of star structures. Second, a new information diffusion model is illustrated to simulate the diffusion process of information in social media by considering the following three parameters: 1) the probability of users receiving this message, 2) the probability that users have a tendency to forward this message and 3) the interest the users hold for this message. Finally, a block mechanism is designed to congest the diffusion of privacy information in social media.

Social media has become a ubiquitous tool for spreading news, messages, and generally allowing for communication between individuals. Hence, studying how our privacy information might also spread across social media is important research. To date, many studies have used information diffusion models to simulate and then examine how information flows through social networks. But these models are theoretical, and newsworthy information may not behave in the same way as privacy information, raising the question: Are the observed phenomena indicative of real privacy propagation? To explore this question, we assembled a dataset from Twitter comprising propagated information flows for both private and normal information. We then built a graph convolutional network to trace and classify differences in the way each type of information spreads throughout the platform. The results reveal that there are indeed key differences in the diffusion processes of the two types of information. More importantly, we design privacy-preserving methods to reduce the privacy propagation in social media.

Deep generative models have gained much attention given their ability to generate data for applications as varied as healthcare to financial technology to surveillance, and many more - the most popular models being generative adversarial networks (GANs) and variational auto-encoders (VAEs). Yet, as with all machine learning models, ever is the concern over security breaches and privacy leaks and deep generative models are no exception. In fact, these models have advanced so rapidly in recent years that work on their security is still in its infancy. In an attempt to audit the current and future threats against these models, and to provide a roadmap for defense preparations in the short term, we prepared this comprehensive and specialized survey on the security and privacy preservation of GANs and VAEs. Our focus is on the inner connection between attacks and model architectures and, more specifically, on five components of deep generative models: the training data, the latent code, the generators/decoders of GANs/VAEs, the discriminators/encoders of GANs/VAEs, and the generated data. For each model, component and attack, we review the current research progress and identify the key challenges. The paper concludes with a discussion of possible future attacks and research directions in the field.

Federated learning is a promising distributed machine learning paradigm that has been playing a significant role in providing privacy-preserving learning solutions. However, alongside all its achievements, there are also limitations. First, traditional frameworks assume that all the clients are voluntary and so will want to participate in training only for improving the model’s accuracy. However, in reality, clients usually want to be adequately compensated for the data and resources they will use before participating. Second, today’s frameworks do not offer sufficient protection against malicious participants who try to skew a jointly trained model with poisoned updates. To address these concerns, we have developed a more robust federated learning scheme based on joint differential privacy. The framework provides two game-theoretic mechanisms to motivate clients to participate in training. These mechanisms are dominant-strategy truthful, individual rational, and budget-balanced. Further, the influence an adversarial client can have is quantified and restricted, and data privacy is similarly guaranteed in quantitative terms. Experiments with different training models on real-word datasets demonstrate the effectiveness of the proposed approach.

Federated learning is a promising distributed machine learning paradigm that has been playing a significant role in privacy-preserving machine learning tasks. However, alongside all its achievements, the framework has limitations. First, traditional frameworks assume that all clients want to improve model accuracy and so participation is voluntary. However, in reality, clients usually want to be appropriately compensated for the data and resources they will need to commit to the training process before contributing. Second, today's frameworks allow clients to perturb their parameter updates locally, which introduces a great deal of noise to the trained model and can seriously impact model accuracy. To address these concerns, we have developed a private reward game that incentivizes clients to contribute high-quality data to the training process. The game converges to a Nash equilibrium under the guarantee of joint differential privacy, and each client maximizes their reward following an equilibrium strategy. The noise injected into the model is reduced by introducing a centralized differential privacy model that aggregates the parameters and compensates clients via a data trading market. Experimental simulations show the rationales behind and effectiveness of the proposed game approach. Additionally, we present comparisons between different training models to demonstrate the performance of the proposed approach in real-world scenarios.

In multi-agent reinforcement learning, transfer learning is one of the key techniques used to speed up learning performance through the exchange of knowledge among agents. However, there are three challenges associated with applying this technique to real-world problems. First, most real-world domains are partially rather than fully observable. Second, it is difficult to pre-collect knowledge in unknown domains. Third, negative transfer impedes the learning progress. We observe that differentially private mechanisms can overcome these challenges due to their randomization property. Therefore, we propose a novel differential transfer learning method for multi-agent reinforcement learning problems, characterized by the following three key features. First, our method allows agents to implement real-time knowledge transfers between each other in partially observable domains. Second, our method eliminates the constraints on the relevance of transferred knowledge, which expands the knowledge set to a large extent. Third, our method improves robustness to negative transfers by applying differentially exponential noise and relevance weights to transferred knowledge. The proposed method is the first to use the randomization property of differential privacy to stimulate the learning performance in multi-agent reinforcement learning system. We further implement extensive experiments to demonstrate the effectiveness of our proposed method.

A growing specter in the rise of machine learning is whether the decisions made by machine learning models are fair. While research is already underway to formalize a machine-learning concept of fairness and to design frameworks for building fair models with sacrifice in accuracy, most are geared toward either supervised or unsupervised learning. Yet two observations inspired us to wonder whether semi-supervised learning might be useful to solve discrimination problems. First, previous study showed that increasing the size of the training set may lead to a better trade-off between fairness and accuracy. Second, the most powerful models today require an enormous of data to train which, in practical terms, is likely possible from a combination of labeled and unlabeled data. Hence, in this paper, we present a framework of fair semi-supervised learning in the pre-processing phase, including pseudo labeling to predict labels for unlabeled data, a re-sampling method to obtain multiple fair datasets and lastly, ensemble learning to improve accuracy and decrease discrimination. A theoretical decomposition analysis of bias, variance and noise highlights the different sources of discrimination and the impact they have on fairness in semi-supervised learning. A set of experiments on real-world and synthetic datasets show that our method is able to use unlabeled data to achieve a better trade-off between accuracy and discrimination.

Networks that constantly transmit information and change structure are becoming increasingly prevalent. However, traditional privacy models are designed to protect static information, such as records in a database or a person’s profile information, which seldom changes. This conflict between static models and dynamic environments is dramatically hindering the effectiveness and efficiency of privacy preservation in today’s dynamic world. Hence, in this paper, we formally define the concept of dynamic privacy, present two novel perspectives, privacy propagation and accumulation, on the way private information can spread through dynamic cyberspace, and develop associated theories and mechanisms for preserving privacy in advanced complex networks, such as social networking sites where data are constantly being released, shared, and exchanged.

Artificial Intelligence (AI) has attracted a great deal of attention in recent years. However, alongside all its advancements, problems have also emerged, such as privacy violations, security issues and model fairness. Differential privacy, as a promising mathematical model, has several attractive properties that can help solve these problems, making it quite a valuable tool. For this reason, differential privacy has been broadly applied in AI but to date, no study has documented which differential privacy mechanisms can or have been leveraged to overcome its issues or the properties that make this possible. In this paper, we show that differential privacy can do more than just preserve privacy. It can also be used to improve security, stabilize learning, build fair models, and impose composition in selected areas of AI. With a focus on regular machine learning, distributed machine learning, deep learning, and multi-agent systems, the purpose of this article is to deliver a new view on many possibilities for improving AI performance with differential privacy techniques.

A critical challenge in machine learning is the vulnerability of learning models in defending attacks from malicious adversaries. In this research, we propose game theoretical learning between a variational adversary and a Convolutional Neural Network (CNN), participating in a variable-sum two-player sequential Stackelberg game. Our adversary manipulates the input data distribution to make the CNN misclassify the manipulated data. Our ideal adversarial manipulation is a minimum change to the data which yet is large enough to mislead the CNNs. We propose an optimization procedure to find optimal adversarial manipulations by solving for the Nash equilibrium of the Stackelberg game. Specifically, the adversary's payoff function depends on the data manipulation which is determined by a Variational Autoencoder, while the CNN classifier's payoff functions are evaluated by misclassification errors. The optimization of our adversarial manipulations is defined by Alternating Least Squares and Simulated Annealing. Experimental results demonstrate that our game-theoretic manipulations are able to mislead CNNs that are well trained on the original data as well as on data generated by other models. We then let the CNNs to incorporate our manipulated data which leads to secure classifiers that are empirically the most robust in defending various types of adversarial attacks.

As one of the most influential privacy definitions, differential privacy provides a rigorous and provable privacy guarantee for data publishing. However, the curator has to release a large number of queries in a batch or a synthetic dataset in the Big Data era. Two challenges need to be tackled: one is how to decrease the correlation between large sets of queries, while the other is how to predict on fresh queries. This paper transfers the data publishing problem to a machine learning problem, in which queries are considered as training samples and a prediction model will be released rather than query results or synthetic datasets. When the model is published, it can be used to answer current submitted queries and predict results for fresh queries from the public. Compared with the traditional method, the proposed prediction model enhances the accuracy of query results for non-interactive publishing. We prove that learning model can successfully retain the utility of published queries while preserving privacy.

Differential privacy is an essential and prevalent privacy model that has been widely explored in recent decades. This survey provides a comprehensive and structured overview of two research directions: differentially private data publishing and differentially private data analysis. We compare the diverse release mechanisms of differentially private data publishing given a variety of input data in terms of query type, the maximum number of queries, efficiency, and accuracy. We identify two basic frameworks for differentially private data analysis and list the typical algorithms used within each framework. The results are compared and discussed based on output accuracy and efficiency. Further, we propose several possible directions for future research and possible applications.

In the context of collaborative filtering, the well-known data sparsity issue makes two like-minded users have little similarity, and consequently renders the k nearest neighbour rule inapplicable. In this paper, we address the data sparsity problem in the neighbourhood-based CF methods by proposing an Adaptive-Maximum imputation method (AdaM). The basic idea is to identify an imputation area that can maximize the imputation benefit for recommendation purposes, while minimizing the imputation error brought in. To achieve the maximum imputation benefit, the imputation area is determined from both the user and the item perspectives; to minimize the imputation error, there is at least one real rating preserved for each item in the identified imputation area. A theoretical analysis is provided to prove that the proposed imputation method outperforms the conventional neighbourhood-based CF methods through more accurate neighbour identification. Experiment results on benchmark datasets show that the proposed method significantly outperforms the other related state-of-the-art imputation-based methods in terms of accuracy.

As a popular technique in recommender systems, Collaborative Filtering (CF) has received extensive attention in recent years. However, its privacy-related issues, especially for neighborhood-based CF methods, can not be overlooked. The aim of this study is to address the privacy issues in the context of neighborhood-based CF methods by proposing a Private Neighbor Collaborative Filtering (PNCF) algorithm. The algorithm includes two privacy-preserving operations: Private Neighbor Selection and Recommendation-Aware Sensitivity. Private Neighbor Selection is constructed on the basis of the notion of differential privacy to privately choose neighbors. Recommendation-Aware Sensitivity is introduced to enhance the performance of recommendations. Theoretical and experimental analysis are provided to show the proposed algorithm can preserve differential privacy while retaining the accuracy of recommendations.

Tagging recommender systems allow Internet users to annotate resources with personalized tags. The connection among users, resources and these annotations, often called afolksonomy, permits users the freedom to explore tags, and to obtain recommendations. Releasing these tagging datasets accelerates both commercial and research work on recommender systems. However, adversaries may re-identify a user and her/his sensitivity information from the tagging dataset using a little background information. Recently, several private techniques have been proposed to address the problem, but most of them lack a strict privacy notion, and can hardly resist the number of possible attacks. This paper proposes an private releasing algorithm to perturb users' profile in a strict privacy notion, differential privacy, with the goal of preserving a user's identity in a tagging dataset. The algorithm includes three privacy preserving operations: Private Tag Clustering is used to shrink the randomized domain and Private Tag Selection is then applied to find the most suitable replacement tags for the original tags. To hide the numbers of tags, the third operation, Weight Perturbation, finally adds Lap lace noise to the weight of tags We present extensive experimental results on two real world datasets, Delicious and Bibsonomy. While the personalization algorithmis successful in both cases.

Two rating patterns exist in the user × item rating matrix and influence each other: the personal rating patterns are hidden in each user's entire rating history, while the global rating patterns are hidden in the entire user × item rating matrix. In this paper, a Rating Pattern Subspace is proposed to model both of the rating patterns simultaneously by iteratively refining each other with an EM-like algorithm. Firstly, a low-rank subspace is built up to model the global rating patterns from the whole user × item rating matrix, then, the projection for each user on the subspace is refined individually based on his/her own entire rating history. After that, the refined user projections on the subspace are used to improve the modelling of the global rating patterns. Iteratively, we can obtain a well-trained low-rank Rating Pattern Subspace, which is capable of modelling both the personal and the global rating patterns. Based on this subspace, we propose a RapSVD algorithm to generate Top-N recommendations, and the experiment results show that the proposed method can significantly outperform the other state-of-the-art Top-N recommendation methods in terms of accuracy, especially on long tail item recommendations.

As each user tends to rate a small proportion of available items, the resulted Data Sparsity issue brings significant challenges to the research of recommender systems. This issue becomes even more severe for neighborhood-based collaborative filtering methods, as there are even lower numbers of ratings available in the neighborhood of the query item. In this paper, we aim to address the Data Sparsity issue in the context of the neighborhood-based collaborative filtering. Given the (user, item) query, a set of key ratings are identified, and an auto-adaptive imputation method is proposed to fill the missing values in the set of key ratings. The proposed method can be used with any similarity metrics, such as the Pearson Correlation Coefficient and Cosine-based similarity, and it is theoretically guaranteed to outperform the neighborhood-based collaborative filtering approaches. Results from experiments prove that the proposed method could significantly improve the accuracy of recommendations for neighborhood-based Collaborative Filtering algorithms.

In this paper, we observe that the user preference styles tend to change regularly following certain patterns. Therefore, we propose a Preference Pattern model to capture the user preference styles and their temporal dynamics, and apply this model to improve the accuracy of the Top-N recommendation. Precisely, a preference pattern is defined as a set of user preference styles sorted in a time order. The basic idea is to model user preference styles and their temporal dynamics by constructing a representative subspace with an Expectation-Maximization (EM)-like algorithm, which works in an iterative fashion by refining the global and the personal preference styles simultaneously. Then, the degree which the recommendations match the active user's preference styles, can be estimated by measuring its reconstruction error from its projection on the representative subspace. The experiment results indicate that the proposed model is robust to the data sparsity problem, and can significantly outperform the state-of-the-art algorithms on the Top-N recommendation in terms of accuracy.

In this paper, we propose a novel model for web-based database systems based on the multicast and anycast' protocols. In the model, we design a middleware, castway, which locates between the database server and the Web server. Every castway in a distributed system operates as a multicast node and an anycast node independently, respectively. The proposed mechanism can balance the workload among the distributed database servers, and offers the "best" server to serve for a query. Three algorithms are employed for the model: the requirement-based probing algorithm for anycast routing, the atomic multicast update algorithm for database synchronization, and the job deviation algorithm for system workload balance. The simulations and experiments show that the proposed model works very well.