The data stream generated by users on web applications is often collected using a local differential privacy (LDP) approach to ensure privacy. This approach offers rigorous theoretical guarantees and low computational overhead, albeit at the expense of data utility. Data utility encompasses both the value of individual data points and the temporal relevance that exists between them, but existing studies primarily focus on enhancing the former utility while neglecting the latter. Furthermore, the collected data often requires cleaning, and we have demonstrated through a case study that data stream lacking time relevance poses a significant risk to users' privacy during the cleaning process. In this paper, for the first time we present an online LDP publishing mechanism while preserving the inherent temporal relevance for the infinite stream, called the Sampling Period Perturbation Algorithm (SPPA). Specifically, we model the temporal relevance between data points as the Fourier interpolation function, resulting in a computational complexity reduction from O(n2) to O(n log n) when compared with the conventional Markov approach in the offline setting. To strike a better balance between privacy and utility, we add noise to the sampling period due to its minimal impact on sensitivity, which is analyzed by our novel concepts of (ε,τ)-temporal indistinguishability and (ε,w,τ)-event LDP. Through extensive experiments, SPPA exhibits superior performance in terms of both data utility and privacy preservation compared to the state-of-the-art baselines. In particular, when ε=1, compared with the state-of-the-art baseline, SPPA diminishes the MSE by up to 64.2%, and raises the event monitoring efficiency by up to 21.4%.

Federated learning (FL) has attracted tremendous attention due to its capability to preserve data privacy. In FL, a parameter server (PS) without accessing clients' raw data can assist decentralized clients in completing model training by aggregating and distributing model parameters for multiple iterations. From the clients' perspective, exposing model parameters to the PS can still result in privacy leakage. To further enhance privacy protection, differentially private federated learning (DPFL) is invented, in which clients add differentially private (DP) noises to distort their parameters to be exposed. However, the main challenge of DPFL lies in inferior model accuracy due to DP noises. To overcome this challenge, in this paper we propose a novel algorithmic framework for DPFL, which is called CODP, by cascading and offsetting DP noises between iterations. In existing works, each DPFL client only considers how to protect its model parameters based on the number of iterations to expose parameters overlooking the underlying relation of model parameters in consecutive iterations. The novelty of CODP lies in cascading DP noises from each iteration to its subsequent iteration so that DP noises can be offset in the subsequent iteration, and hence model accuracy can be improved. Additionally, we theoretically prove that CODP can substantially improve the convergence rate of DPFL without compromising privacy preservation by leveraging the most widely used Laplace and Gaussian mechanisms, respectively. We conduct comprehensive experiments using MNIST, Fashion-MNIST, and Lending Club datasets to demonstrate that the model accuracy of DPFL can be remarkably improved by CODP with a fixed privacy budget.

Federated learning (FL) is an emerging machine learning paradigm for preserving data privacy. However, diverse client hardware often has varying computation resources. Such system heterogeneity limits the participation of resource-constrained clients in FL, and hence degrades the global model accuracy. To enable heterogeneous clients to participate in and contribute to FL training, previous works tackle this problem by assigning customized sub-models to individual clients with model pruning, distillation, or low-rank based techniques. Unfortunately, the global model trained by these methods still encounters performance degradation due to heterogeneous sub-model aggregation. Besides, most methods are heuristic-based and lack convergence analysis. In this work, we propose the FedLMT framework to bridge the performance gap, by assigning clients with a homogeneous pre-factorized low-rank model to substantially reduce resource consumption without conducting heterogeneous aggregation. We theoretically prove that the convergence of the low-rank model can guarantee the convergence of the original full model. To further meet clients’ personalized resource needs, we extend FedLMT to pFedLMT, by separating model parameters into common and custom ones. Finally, extensive experiments are conducted to verify our theoretical analysis and show that FedLMT and pFedLMT outperform other baselines with much less communication and computation costs.

Although Federated Learning (FL) prevents the exposure of original data samples when collaboratively training machine learning models among decentralized clients, it has been revealed that vanilla FL is still susceptible to adversarial attacks if model parameters are leaked to malicious attackers. To enhance the protection level of FL, Differential Private Federated Learning (DPFL) has been proposed in recent years. DPFL injects zero-mean noises randomly generated by differential private (DP) mechanisms on local model parameters before they are disclosed. Nevertheless, DP noises can significantly deteriorate model utility jeopardizing the practicality of DPFL. In this paper, we are among the first to explore how to improve the model utility of DPFL by tuning the number of local iterations (LIs) on DPFL clients. Our work shows that such a local iteration tuning approach can well mitigate the adverse influence of DP noises on the final model utility. Formally, we derive the sensitivity (a measure of the maximum change of the output given two adjacent inputs) with respect to the number of LIs conducted on DPFL clients for the Laplace mechanism, and the aggregated variances of Laplace noises at the server side. We further conduct convergence rate analysis to quantify the influence of the Laplace noises on the final model accuracy and determine how to optimally set the number of LIs. Finally, to verify our theoretical findings, we perform extensive experiments using three real-world datasets, namely, Lending Club, MNIST and Fashion-MNIST. The results not only corroborate our analysis, but also demonstrate that our approach significantly improves the practicality of DPFL.

Different from conventional federated learning, personalized federated learning (PFL) is able to train a customized model for each individual client according to its unique requirement. The mainstream approach is to adopt a kind of weighted aggregation method to generate personalized models, in which weights are determined by the loss value or model parameters among different clients. However, such kinds of methods require clients to download others' models. It not only sheer increases communication traffic but also potentially infringes data privacy. In this paper, we propose a new PFL algorithm called FedDWA (Federated Learning with Dynamic Weight Adjustment) to address the above problem, which leverages the parameter server (PS) to compute personalized aggregation weights based on collected models from clients. In this way, FedDWA can capture similarities between clients with much less communication overhead. More specifically, we formulate the PFL problem as an optimization problem by minimizing the distance between personalized models and guidance models, so as to customize aggregation weights for each client. Guidance models are obtained by the local one-step ahead adaptation on individual clients. Finally, we conduct extensive experiments using five real datasets and the results demonstrate that FedDWA can significantly reduce the communication traffic and achieve much higher model accuracy than the state-of-the-art approaches.

Instead of training a single global model to fit the needs of all clients, personalized federated learning aims to train multiple client-specific models to better account for data disparities across participating clients. However, existing solutions suffer from serious unfairness among clients in terms of model accuracy and slow convergence under non-lID data. In this paper, we propose a novel personalized federated learning framework, called D PFed, which employs deep reinforcement learning (D RL) to identify relationship between clients and enable closer collaboration among similar clients. By exploiting such relationships, DPFed can personalize model aggregation for each client and achieve fast convergence. Moreover, by regularizing the reward function of DRL, we can reduce the variance of model accuracy across clients and achieve a higher level of fairness. Finally, we conduct extensive experiments to evaluate the effectiveness of our proposed framework under a variety of datasets and degrees of non-lID data distribution. The results demonstrate that DPFed outperforms other alternatives in terms of convergence speed, model accuracy, and fairness.