Quantization is a pivotal technique for enhancing communication efficiency in Federated Learning (FL). Traditional quantization methods often set uniform intervals, may fail to adequately characterize non-uniform data distributions, thus leading to substantial estimation errors and degrated model performance. Non-uniform quantization can better solve the problem. However, when applied to FL, it would bring additional communication overheads for the alignment of parameter distributions among distributed models. To address this issue, we propose Bisection Interval Quantization (BIQ), a novel non-uniform quantization framework for FL with great communication efficiency. In particular, BIQ works by optimizing the interval selection through recursive bisection among distributed clients without extra parameter communication. For scenarios involving amounts of boundary inputs, we further design Weighted Bisection Interval Quantization (WBIQ), which incorporates maximum likelihood estimation to refine boundary value reconstruction to enhance the estimation quality of boundary inputs. Our theoretical analysis rigorously establishes, for the first time under biased quantization conditions, that both BIQ and WBIQ achieve tighter error bounds and enhanced stability. Extensive experiments validate that both BIQ and WBIQ significantly accelerate the convergence of FL model training when compared to the state-of-the-art quantizers under both convex and non-convex settings.

Local Differential Privacy (LDP) enables massive data collection and analysis while protecting end users' privacy against untrusted aggregators. It has been applied to various data types (e.g., categorical, numerical, and graph data) and application settings (e.g., static and streaming). Recent findings indicate that LDP protocols can be easily disrupted by poisoning or manipulation attacks, where an attacker can leverage injected/corrupted fake users to send crafted data to the aggregator in order to manipulate the final estimate of the aggregator. However, current attacks primarily target static protocols, neglecting the security of LDP protocols in the streaming settings. Our research fills the gap by developing novel fine-grained manipulation attacks to LDP protocols for data streams. By reviewing the attack surfaces in existing algorithms, we introduce a unified attack framework with composable modules, which can manipulate the LDP estimated stream toward a target stream. Our attack framework can adapt to state-of-the-art streaming LDP algorithms with different analytic tasks (e.g., frequency and mean) and LDP models (event-level, user-level, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$w$</tex-math></inline-formula>-event level). We verify our attacks theoretically and validate them through extensive experiments on real-world datasets. Finally, we explore a possible defense mechanism for mitigating our attacks.

Graph federated learning enables the collaborative extraction of high-order information from distributed subgraphs while preserving the privacy of raw data. However, graph data often exhibits overlap among different clients. Previous research has demonstrated certain benefits of overlapping data in mitigating data heterogeneity. However, the negative effects have not been explored, particularly in cases where the overlaps are imbalanced across clients. In this paper, we uncover the unfairness issue arising from imbalanced overlapping subgraphs through both empirical observations and theoretical reasoning. To address this issue, we propose FairGFL (FAIRness-aware subGraph Federated Learning), a novel algorithm that enhances cross-client fairness while maintaining model utility in a privacy-preserving manner. Specifically, FairGFL incorporates an interpretable weighted aggregation approach to enhance fairness across clients, leveraging privacy-preserving estimation of their overlapping ratios. Furthermore, FairGFL improves the tradeoff between model utility and fairness by integrating a carefully crafted regularizer into the federated composite loss function. Through extensive experiments on four benchmark graph datasets, we demonstrate that FairGFL outperforms four representative baseline algorithms in terms of both model utility and fairness.

The integration of Differential Privacy (DP) with diffusion models (DMs) presents a promising yet challenging frontier, particularly due to the substantial memorization capabilities of DMs that pose significant privacy risks. Differential privacy offers a rigorous framework for safeguarding individual data points during model training, with Differential Privacy Stochastic Gradient Descent (DP-SGD) being a prominent implementation. Diffusion method decomposes image generation into iterative steps, theoretically aligning well with DP's incremental noise addition. Despite the natural fit, the unique architecture of DMs necessitates tailored approaches to effectively balance privacy-utility trade-off. Recent developments in this field have highlighted the potential for generating high-quality synthetic data by pre-training on public data (i.e., ImageNet) and fine-tuning on private data, however, there is a pronounced gap in research on optimizing the trade-offs involved in DP settings, particularly concerning parameter efficiency and model scalability. Our work addresses this by proposing a parameter-efficient fine-tuning strategy optimized for private diffusion models, which minimizes the number of trainable parameters to enhance the privacy-utility trade-off. We empirically demonstrate that our method achieves state-of-the-art performance in DP synthesis, significantly surpassing previous benchmarks on widely studied datasets (e.g., with only 0.47M trainable parameters, achieving a more than 35% improvement over the previous state-of-the-art with a small privacy budget on the CelebA-64 dataset). Anonymous codes available at https://anonymous.4open.science/r/DP-LORA-F02F.

Network topology planning is an essential multi-phase process to build and jointly optimize the multi-layer network topologies in wide-area networks (WANs). Most existing practices target single-phase/layer planning, and are incapable of satisfying all rigorous topological structure constraints (e.g., dual-homing rings) defined by network standards and operators, especially in large-scale networks. These significantly limit their usability and performance in production networks. We consider a general topology planning problem with typical structure constraints over three essential phases (greenfield, reconfiguration, and site expansion) and topological layers (optical, IP, and routing topologies). We present, T3Planner, a novel practical solver to this problem in production. Specifically, we develop a structure-driven encoder based on graph neural network (GNN) for concise structure encoding, and design a new learning framework with optical-centric layer compression/reconstruction and rule-aided reinforcement learning (RL) for fast convergence and high performance. Extensive experiments on nine real topologies demonstrate that T3Planner scales to large optical networks with hundreds of sites, saves 46.6% cost, and supports$3.12\times $more demand when compared to related existing approaches.

In mobile edge computing (MEC), resource scheduling is crucial to task requests’ performance and service providers’ cost, involving multi-layer heterogeneous scheduling decisions. Existing MEC schedulers typically adopt static-timescale scheduling, where scheduling decisions are updated regularly at fixed intervals for all layers. The inflexible updating timescales lead to poor performance in the production networks. In this paper, we propose EdgeTimer, an unprecedented approach that automatically and adaptively determines respective updating timescales of multiple scheduling layers to achieve a better trade-off between the operation cost and service performance. Specifically, we design (i) a three-layer hierarchical deep reinforcement learning (DRL) framework for efficient learning of tightly coupled policies, (ii) a tailored multi-agent DRL algorithm for decentralized scheduling, with the convergence strictly proved, and (iii) a lightweight system defender for deterministic reliability assurance. Furthermore, we apply EdgeTimer to a wide range of Kubernetes scheduling rules, and evaluate it using production traces with different workload patterns. Through extensive trace-driven experiments, we demonstrate that EdgeTimer can significantly decrease the operation cost for service providers without sacrificing the delay performance, thereby improving overall profits, compared with the state-of-the-art approaches.

Federated Learning (FL) succeeds in collaborative and privacy-preserving ML model training among multiple distributed data owners. To maintain a healthy FL ecosystem, it is crucial to estimate the contributions of all participants fairly. Due to provable fairness, Shapley value (SV) is widely used for contribution estimation in FL. However, current studies focus on static scenarios with fixed participants and neglect the dynamic settings with the random joining or leaving of participants in practice. This paper fills the gap by proposing FedDSV, a novel contribution estimation framework for FL with dynamic participation. FedDSV supports flexible weighting mechanisms and is compatible with the SV fairness properties in dynamic scenarios. To reduce the computational complexity, we propose a Monte Carlo variant sampling method (SMC), which can adapt well to dynamic scenarios and approximate the true SVs. To evaluate the effectiveness and efficiency of our proposed approaches, extensive experiments under different settings (e.g., frequency switching, low-quality detection, etc.) are conducted on both i.i.d and non-i.i.d. distributions. Experimental results demonstrate that FedDSV can reflect the real utility contribution of data sources for dynamic FL, and SMC can approximate the exact dynamic SVs with larger similarities in a much shorter time than the state-of-the-art methods.

In mobile edge computing (MEC), resource scheduling is crucial to task requests’ performance and service providers’ cost, involving multi-layer heterogeneous scheduling decisions. Existing schedulers typically adopt static timescales to regularly update scheduling decisions of each layer, without adaptive adjustment of timescales for different layers, resulting in potentially poor performance in practice.We notice that the adaptive timescales would significantly improve the trade-off between the operation cost and delay performance. Based on this insight, we propose EdgeTimer, the first work to automatically generate adaptive timescales to update multi-layer scheduling decisions using deep reinforcement learning (DRL). First, EdgeTimer uses a three-layer hierarchical DRL framework to decouple the multi-layer decision-making task into a hierarchy of independent sub-tasks for improving learning efficiency. Second, to cope with each sub-task, EdgeTimer adopts a safe multi-agent DRL algorithm for decentralized scheduling while ensuring system reliability. We apply EdgeTimer to a wide range of Kubernetes scheduling rules, and evaluate it using production traces with different workload patterns. Extensive trace-driven experiments demonstrate that EdgeTimer can learn adaptive timescales, irrespective of workload patterns and built-in scheduling rules. It obtains up to 9:1 more profit than existing approaches without sacrificing the delay performance.

Data synthesis is a promising solution to share data for various downstream analytic tasks without exposing raw data. However, without a theoretical privacy guarantee, a synthetic dataset would still leak some sensitive information in raw data. As a countermeasure, differential privacy is widely adopted to safeguard data synthesis by strictly limiting the released information. This technique is advantageous yet presents significant challenges in the vertical federated setting, where data attributes are distributed among different data parties. The main challenge lies in maintaining privacy while efficiently and precisely reconstructing the correlation between attributes. In this paper, we propose a novel algorithm called VertiMRF, designed explicitly for generating synthetic data in the vertical setting and providing differential privacy protection for all information shared from data parties. We introduce techniques based on the Flajolet-Martin (FM) sketch for encoding local data satisfying differential privacy and estimating cross-party marginals. We provide theoretical privacy and utility proof for encoding in this multi-attribute data. Collecting the locally generated private Markov Random Field (MRF) and the sketches, a central server can reconstruct a global MRF, maintaining the most useful information. Two critical techniques introduced in our VertiMRF are dimension reduction and consistency enforcement, preventing the noise of FM sketch from overwhelming the information of attributes with large domain sizes when building the global MRF. These two techniques allow flexible and inconsistent binning strategies of local private MRF and the data sketching module, which can preserve information to the greatest extent. We conduct extensive experiments on four real-world datasets to evaluate the effectiveness of VertiMRF. End-to-end comparisons demonstrate the superiority of VertiMRF.

Safety protective gear (SPG) detection based on the machine learning model plays an important role in improving outdoor personnel safety in the electric power industry. However, the detection method of transmitting video to the cloud faces a series of challenges, such as privacy disclosure and high latency. To solve this problem, we present FtlSPG, a federated transfer learning framework for SPG detection. In particular, under the three-layer pyramid architecture of “site-companyCloud-globalServer,” we propose a federated personalized model based on local batch normalization and dynamical weighting for the source domain with labeled video. Moreover, a federated domain adaptation model based on a federated deep adversarial network and model self-training is presented for the target domain with unlabeled video. Finally, we verify the effectiveness of FtlSPG in real-world power companies. Extensive experiments demonstrate that FtlSPG can significantly outperform existing schemes, in terms of privacy protection, detection precision, and response latency.

Knowledge and data dual-driven (KDDD) represents a novel paradigm that leverages the strengths of data-driven methods in feature representation and knowledge transfer, while also incorporating expertise accumulated by domain experts. This integration allows KDDD methods to enhance the interpretability, reliability, and robustness of fault diagnosis (FD) approaches, making them widely studied in the field of industrial equipment (IE) FD. Despite the existence of systematic and valuable reviews on IE FD, there remains a gap in the literature regarding the review of KDDD IE FD methods. Therefore, conducting a comprehensive investigation into KDDD IE FD methods is of utmost importance and necessity. Such an investigation will facilitate readers’ understanding of advanced technologies and enable the rapid design of effective solutions for real-world IE FD problems. In this survey, we first outline the limitations of data-driven and knowledge-based FD methods, highlighting the need for KDDD methods. Subsequently, we delve into the details of how domain knowledge can be effectively integrated with deep learning models. Additionally, we analyze challenges of KDDD methods in real-world IE FD applications, while also discussing novel solutions for prospective research directions. Finally, we conclude this survey, emphasizing the inspiration it offers to researchers interested in advancing IE FD, and its potential to stimulate practical IE FD research.

The fusion of federated learning and differential privacy can provide more comprehensive and rigorous privacy protection, thus attracting extensive interests from both academia and industry. However, facing the system-level challenge of device heterogeneity, most current synchronous FL paradigms exhibit low efficiency due to the straggler effect, which can be significantly reduced by Asynchronous FL (AFL). However, AFL has never been comprehensively studied, which imposes a major challenge in the utility optimization of DP-enhanced AFL. Here, theoretically motivated multi-stage adaptive private algorithms are proposed to improve the trade-off between model utility and privacy for DP-enhanced AFL. In particular, we first build two DP-enhanced AFL frameworks with consideration of universal factors for different adversary models. Then, we give a solid analysis on the model convergence of AFL, based on which, DP can be adaptively achieved with high utility. Through extensive experiments on different training models and benchmark datasets, we demonstrate that the proposed algorithms achieve the overall best performances and improve up to 24% test accuracy with the same privacy loss and have faster convergence compared with the state-of-the-art algorithms. Our frameworks provide an analytical way for private AFL and adapt to more complex FL application scenarios.

Bitrate adaptation (also known as ABR) is a crucial technique to improve the quality of experience (QoE) for video streaming applications. However, existing ABR algorithms suffer from severe traffic wastage, which refers to the traffic cost of downloading the video segments that users do not finally consume, for example, due to early departure or video skipping. In this paper, we carefully formulate the dynamics of buffered data volume (BDV), a strongly correlated indicator of traffic wastage, which, to the best of our knowledge, is the first time to rigorously clarify the effect of downloading plans on potential wastage. To reduce wastage while keeping a high QoE, we present a bandwidth-efficient bitrate adaptation algorithm (named BE-ABR), achieving consistently low BDV without distinct QoE losses. Specifically, we design a precise, time-aware transmission delay prediction model over the Transformer architecture, and develop a fine-grained buffer control scheme. Through extensive experiments conducted on emulated and real network environments including WiFi, 4G, and 5G, we demonstrate that BE-ABR performs well in both QoE and bandwidth savings, enabling a 60.87% wastage reduction and a comparable, or even better, QoE, compared to the state-of-the-art methods.

Visual Prompting (VP) is an emerging and powerful technique that allows sample-efficient adaptation to downstream tasks by engineering a well-trained frozen source model. In this work, we explore the benefits of VP in constructing compelling neural network classifiers with differential privacy (DP). We explore and integrate VP into canonical DP training methods and demonstrate its simplicity and efficiency. In particular, we discover that VP in tandem with PATE, a state-of-the-art DP training method that leverages the knowledge transfer from an ensemble of teachers, achieves the state-of-the-art privacy-utility tradeoff with minimum expenditure of privacy budget. Moreover, we conduct additional experiments on cross-domain image classification with a sufficient domain gap to further unveil the advantage of VP in DP. Lastly, we also conduct extensive ablation studies to validate the effectiveness and contribution of VP under DP consideration. Our code is available at https://github.com/EzzzLi/Prompt-PATE.

The development of cloud and edge computing has enabled the easy access of artificial intelligence (AI) services for massive heterogeneous and resource-constrained devices. Particularly, computation-intensive AI services can be orchestrated and deployed in the cloud or edge according to varying performance and cost requirements. Nonetheless, the improved accessibility of deep learning (DL) model variants and the evolving of computational intelligence paradigms pose great challenges for orchestrating large-scale DL inference services in the cloud-edge continuum. Focusing on cloud or edge-based deployment, existing work on multi-variant service orchestration often has a limited solution space of deployment plans. To address this limitation, we first propose a novel multi-paradigm deployment model (MPDM) for service orchestration, which not only considers the model variants but also allows the co-existence of multiple paradigms for large-scale inference service deployment. The service deployment in the MPDM model is then formulated as a multiobjective optimization problem of seeking a better tradeoff among the system accuracy, service scale, and deployment cost. To solve the multiobjective optimization, we further propose a weighted metric-based constructive heuristic algorithm (WCH), which can efficiently obtain an approximately optimal Pareto frontier. Extensive experimental results have validated the effectiveness and efficiency of WCH, and revealed the impacts of both multi-paradigm deployment and edge-cloud collaborative intelligence (ECCI) paradigm on large-scale DL serving systems.

Emerging intelligent applications based on accurate and timely stream analytics require real-time CNN inference of massive data continuously generated at the pervasive end devices. Due to the resource constraints, neither computing locally at end devices nor transmitting to remote servers is competent for computation-intensive CNN inference on large-volume images in real-time. Therefore, Collaborative Inference (CI), which conducts inference sequentially from the local device to the remote server with compressed intermediate inference data, is rapidly promoted. Due to the essential communication in collaboration, the CI efficiency is sensitive to network conditions, and will degrade under the unpredictable network fluctuations in practice, which may cause a severe delay in CI and degrade the responsiveness of stream analytics. For accurate and timely stream analytics in practical fluctuating networks, we present RTCoInfer, the real-time CI framework with run-time transmission adaption considering the network conditions. Specifically, we propose a novel Switchable CNN integrating CNNs with different compression rates on the partition layer for the run-time transmission adjustment, and construct a real-time controller determining the compression rate to maintain the real-time CI for stream analytics. Extensive experiments show that, compared with state-of-the-art methods, RTCoInfer achieves better efficiency and unprecedented resilience in real-time stream analytics.

Local differential privacy (LDP) is promising for private streaming data collection and analysis. However, existing few LDP studies over streams either apply to finite streams only or may suffer from insufficient protection. This paper investigates this problem by proposing LDP-IDS, a novel w-event LDP paradigm to provide practical privacy guarantee for infinite streams. By constructing a unified error analysis, we adapt the existing budget division framework in centralized differential privacy (CDP) for LDP-IDS, which however incurs prohibitive noise and expensive communication cost. To this end, we propose a novel and extensible framework of population division and recycling, as well as online adaptive population division algorithms for LDP-IDS. We provide theoretical guarantees and demonstrate, through extensive discussions, that our proposed framework not only achieves significant reduction in utility loss and communication overhead, but also enjoys great compatibility for varied analytic tasks and flexibility of incorporating ideas of many existing stream algorithms. Extensive experiments on synthetic and real-world datasets validate the high effectiveness, efficiency, and flexibility of our proposed framework and methods.

Federated learning (FL) is capable of analyzing tremendous data from smart edge devices in Industrial Internet of Things (IIoTs), empowering numerous industrial applications. However, the increasing privacy concerns and deployment costs of IIoT environment have been posing new challenges for FL. This article proposes PCFed, a novel privacy-enhanced and communication-efficient FL framework to provide higher model accuracy with rigorous privacy guarantees and great communication efficiency. In particular, we develop a sampling-based intermittent communication strategy via a PID (proportional, integral, and derivative) controller on the cloud server to adaptively reduce the communication frequency. In addition, we design a budget allocation mechanism to balance the tradeoff between model accuracy and privacy loss. Then, we develop PCFed+, an enhanced variant for PCFed, with further consideration of infinite data streams on edge servers. Extensive experiments demonstrate that both PCFed and PCFed+ can significantly outperform existing schemes, in terms of communication efficiency, privacy protection, and model accuracy.

An edge-cloud collaborative intelligent (ECCI) platform is of great significance for the agile development and rapid deployment of ECCI applications, which are essential for realizing smart industry in the era of Industry 4.0. However, the existing platforms lack considering the high real-time latency demand of industrial operations, which severely hinders the development of smart industry and may even lead to severe industrial accidents. To effectively reduce the response latency of industrial applications, in this article, we propose an ECCI platform IndustEdge. It takes time-sensitive networking as the deterministic transport for the link layer, and provides an extensible ECCI orchestration component to reduce the system level latency. Furthermore, IndustEdge has an ECCI algorithm library for different collaborative modes and provides the complete life cycle management for ECCI applications. We implement platforms for both the real-world prototype and emulated-world emulation, and conduct two case studies to evaluate the effectiveness of IndustEdge.

Federated learning (FL) is an emerging privacy-preserving paradigm that enables multiple participants collaboratively to train a global model without uploading raw data. Considering heterogeneous computing and communication capabilities of different participants, asynchronous FL can avoid the stragglers effect in synchronous FL and adapts to scenarios with vast participants. Both staleness and non-IID data in asynchronous FL would reduce the model utility. However, there exists an inherent contradiction between the solutions to the two problems. That is, mitigating the staleness requires to select less but consistent gradients while coping with non-IID data demands more comprehensive gradients. To address the dilemma, this paper proposes a two-stage weighted$K$asynchronous FL with adaptive learning rate (WKAFL). By selecting consistent gradients and adjusting learning rate adaptively, WKAFL utilizes stale gradients and mitigates the impact of non-IID data, which can achieve multifaceted enhancement in training speed, prediction accuracy and training stability. We also present the convergence analysis for WKAFL under the assumption of unbounded staleness to understand the impact of staleness and non-IID data. Experiments implemented on both benchmark and synthetic FL datasets show that WKAFL has better overall performance compared to existing algorithms.

With the increasing popularity of crowdsourcing services, high-dimensional crowdsourced data provides a wealth of knowledge. Nonetheless, unprecedented privacy threats to participants have emerged, due to complex correlations among multiple attributes and the vulnerabilities of untrusted crowdsourcing servers. Differential privacy-based paradigms have been proposed to release privacy-preserving datasets with statistical approximation. Nonetheless, most existing schemes are limited when facing highly correlated attributes, and cannot prevent privacy threats from untrusted crowdsourcing servers. To address this issue, we propose two novel solutions, namelyLoCopandDR_LoCop, which guarantee local differential privacy based on the randomized response technique while synthesizing and releasing high-dimensional crowdsourced data with high data utility. Particularly,LoCopleverages copula theory to synthesize high-dimensional crowdsourced data via univariate marginal distribution and attribute dependence. Univariate marginal distribution is estimated by the Lasso-based regression algorithm from aggregated privacy-preserving bit strings. Dependencies among attributes are modeled as multivariate Gaussian copula. Based onLoCop, the enhanced solutionDR_LoCopnot only takes advantage of C-vine copula to reflect conditional dependencies among high-dimensional attributes, but also achieves dimension reduction. Extensive experiments on real-world datasets demonstrate that our solutions substantially outperform the state-of-the-art techniques in terms of both data utility and computational overhead.

In Internet-of-Things (IoT)-driven smart-world systems, real-time crowdsourced databases from multiple distributed servers can be aggregated to extract dynamic statistics from a larger population, thus providing more reliable knowledge for our society. Particularly, multiple distributed servers in a decentralized network can realize real-time collaborative statistical estimation by disseminating statistics from their separate databases. Despite no raw data sharing, the real-time statistics could still expose the data privacy of crowdsourcing participants. For mitigating the privacy concern, while the traditional differential privacy (DP) mechanism can be simply implemented to perturb the statistics in each timestamp and independently for each dimension, this may suffer a great utility loss from the real-time and multidimensional crowdsourced data. Also, the real-time broadcasting would bring significant overheads in the whole network. To tackle the issues, we propose a novel privacy preserving and communication-efficient decentralized statistical estimation algorithm (DPCrowd), which only requires intermittently sharing the DP protected parameters with one-hop neighbors by exploiting the temporal correlations in real-time crowdsourced data. Then, with further consideration of spatial correlations, we develop an enhanced algorithm, DPCrowd+, to deal with multidimensional infinite crowd-data streams. Extensive experiments on several data sets demonstrate that our proposed schemes DPCrowd and DPCrowd+ can significantly outperform existing schemes in providing accurate and consensus estimation with rigorous privacy protection and great communication efficiency.

The massive generation, extensive sharing, and deep exploitation of data in the big data era have raised unprecedented privacy threats. To address privacy concerns, various privacy paradigms have been proposed to achieve a good tradeoff between privacy and data utility. Particularly, differential privacy has been well accepted as one of the de facto standard for privacy preservation, and numerous schemes guaranteeing differential privacy have been proposed. Nonetheless, most of the existing works claiming a superior utility-privacy tradeoff only present specific methods, with distinct perspectives, and a complete comparative analysis and evaluation study has not been fully investigated. To this end, in this paper we review and investigate existing schemes on providing differential privacy from a broad and encompassing perspective to provide a comprehensive survey with respect to both the privacy guarantee and the effectiveness and efficiency in utility improvement. We categorize the existing schemes into distribution optimization, sensitivity calibration, transformation, decomposition, and correlations exploitation, based on their mechanisms in improving data utility. We also conduct some analysis and comparison of their various concepts and principles, focusing on improvements to data utility. Finally, we outline some challenges and provide future research directions.

Latent Dirichlet Allocation (LDA) is a popular topic modeling technique for hidden semantic discovery of text data and serves as a fundamental tool for text analysis in various applications. However, the LDA model as well as the training process of LDA may expose the text information in the training data, thus bringing significant privacy concerns. To address the privacy issue in LDA, we systematically investigate the privacy protection of the main-stream LDA training algorithm based on Collapsed Gibbs Sampling (CGS) and propose several differentially private LDA algorithms for typical training scenarios. In particular, we present the first theoretical analysis on the inherent differential privacy guarantee of CGS based LDA training and further propose a centralized privacy-preserving algorithm (HDP-LDA) that can prevent data inference from the intermediate statistics in the CGS training. Also, we propose a locally private LDA training algorithm (LP-LDA) on crowdsourced data to provide local differential privacy for individual data contributors. Furthermore, we extend LP-LDA to an online version as OLP-LDA to achieve LDA training on locally private mini-batches in a streaming setting. Extensive analysis and experiment results validate both the effectiveness and efficiency of our proposed privacy-preserving LDA training algorithms.

Latency-aware service placement is promising in reducing the overall service response latency of proliferating edge-cloud collaborative smart manufacturing systems. However, intuitive latency estimators used by existing service placement approaches cannot accurately depict the nonlinear end-to-end (E2E) latency of multihop microservices with complex dependencies, which is severely hindering the effectiveness of latency-aware service placement. To address this issue, in this article, we present a microservice placement mechanism for edge-cloud collaborative smart manufacturing (MPCSM), where a microservice placement algorithm latency-aware edge-cloud collaborative placement supported by an accurate data-driven E2E latency estimation method is proposed. We build a real-world collaborative prototype, and conduct a case study on semiconductor manufacturing to elaborate the construction of our latency estimator. Results of extensive experiments demonstrate that the error of our E2E latency estimator is up to 10× less than that of existing ones, and the overall service latency with MPCSM is up to 10× less than that with existing service placement approaches.

With the rapid development of artificial intelligence (AI), ethical issues surrounding AI have attracted increasing attention. In particular, autonomous vehicles may face moral dilemmas in accident scenarios, such as staying the course resulting in hurting pedestrians or swerving leading to hurting passengers. To investigate such ethical dilemmas, recent studies have adopted preference aggregation, in which each voter expresses her/his preferences over decisions for the possible ethical dilemma scenarios, and a centralized system aggregates these preferences to obtain the winning decision. Although a useful methodology for building ethical AI systems, such an approach can potentially violate the privacy of voters since moral preferences are sensitive information and their disclosure can be exploited by malicious parties resulting in negative consequences. In this paper, we report a first-of-its-kind privacy-preserving crowd-guided AI decision-making approach in ethical dilemmas. We adopt the formal and popular notion of differential privacy to quantify privacy, and consider four granularities of privacy protection by taking voter-/record-level privacy protection and centralized/distributed perturbation into account, resulting in four approaches VLCP, RLCP, VLDP, and RLDP, respectively. Moreover, we propose different algorithms to achieve these privacy protection granularities, while retaining the accuracy of the learned moral preference model. Specifically, VLCP and RLCP are implemented with the data aggregator setting a universal privacy parameter and perturbing the averaged moral preference to protect the privacy of voters' data. VLDP and RLDP are implemented in such a way that each voter perturbs her/his local moral preference with a personalized privacy parameter. Extensive experiments based on both synthetic data and real-world data of voters' moral decisions demonstrate that the proposed approaches achieve high accuracy of preference aggregation while protecting individual voter's privacy.

Latent Dirichlet Allocation (LDA) is a popular topic modeling technique for discovery of hidden semantic architecture of text datasets, and plays a fundamental role in many machine learning applications. However, like many other machine learning algorithms, the process of training a LDA model may leak the sensitive information of the training datasets and bring significant privacy risks. To mitigate the privacy issues in LDA, we focus on studying privacy-preserving algorithms of LDA model training in this paper. In particular, we first develop a privacy monitoring algorithm to investigate the privacy guarantee obtained from the inherent randomness of the Collapsed Gibbs Sampling (CGS) process in a typical LDA training algorithm on centralized curated datasets. Then, we further propose a locally private LDA training algorithm on crowdsourced data to provide local differential privacy for individual data contributors. The experimental results on real-world datasets demonstrate the effectiveness of our proposed algorithms.

Spatio-temporal monitoring of the Internet of Things (IoT) has enabled the development and proliferation of third-party computing services by extensively exploiting the massive amount of sensing data. In particular, continuously generated data stream are monitored in real-time and exploited to facilitate people's daily lives, such as traffic monitoring and epidemic prevention. In its simplest way of deployment, the direct publishing of various streams could seriously compromise the privacy of participating users. Hence, a more sophisticated scheme is needed to regulate the privately publishing of data streams, which may possibly require control to be applied dynamically. However, most existing solutions are non-adaptive to dynamic changes of the streams due to constraints of predefined parameters, thus are vulnerable to low data utility. In this paper, we present AdaPub, a data-adaptive framework for infinite multidimensional stream real-time publishing with ω-event differential privacy while ensuring high data utility. Without predefining the parameters, AdaPub could learn and update the parameters that reflect the spatio-temporal correlations of the stream in a data-adaptive manner. Specifically, we propose two modules DimParti and AdaCluster which are seamlessly incorporated into AdaPub to simultaneously learn dimension correlations and time correlations in a data-adaptive way, thus greatly improving the data utility of the sanitized streams. Extensive experiments on real-world datasets demonstrate that our solution substantially outperforms state-of-the-art solutions with much lower errors while achieving strong privacy guarantees.

It has been widely understood that differential privacy can guarantee rigorous privacy against adversaries with arbitrary prior knowledge. However, recent studies demonstrate that this may not be true for correlated data, and indicate that three factors could influence privacy leakage: the data correlation pattern, prior knowledge of adversaries, and sensitivity of the query function. This poses a fundamental problem: what is the mathematical relationship between the three factors and privacy leakage? In this paper, we present a unified analysis of this problem. A new privacy definition, named prior differential privacy (PDP), is proposed to evaluate privacy leakage considering the exact prior knowledge possessed by the adversary. We use two models, the weighted hierarchical graph and the multivariate Gaussian model, to analyze discrete and continuous data, respectively. We demonstrate that positive, negative, and hybrid correlations have distinct impacts on privacy leakage. Considering general correlations, a closed-form expression of privacy leakage is derived for continuous data, and a chain rule is presented for discrete data. Our results are valid for general linear queries, including count, sum, mean, and histogram. Numerical experiments are presented to verify our theoretical analysis.

High-dimensional crowdsourced data collected from numerous users produces rich knowledge about our society; however, it also brings unprecedented privacy threats to the participants. Local differential privacy (LDP), a variant of differential privacy, is recently proposed as a state-of-the-art privacy notion. Unfortunately, achieving LDP on high-dimensional crowdsourced data publication raises great challenges in terms of both computational efficiency and data utility. To this end, based on the expectation maximization (EM) algorithm and Lasso regression, we first propose efficient multi-dimensional joint distribution estimation algorithms with LDP. Then, we develop a local differentially private high-dimensional data publication algorithm (LoPub) by taking advantage of our distribution estimation techniques. In particular, correlations among multiple attributes are identified to reduce the dimensionality of crowdsourced data, thus speeding up the distribution learning process and achieving high data utility. Extensive experiments on real-world datasets demonstrate that our multivariate distribution estimation scheme significantly outperforms existing estimation schemes in terms of both communication overhead and estimation speed. Moreover, LoPub can keep, on average, 80% and 60% accuracy over the released datasets in terms of support vector machine and random forest classification, respectively.

Various paradigms, based on differential privacy, have been proposed to release a privacy-preserving dataset with statistical approximation. Nonetheless, most existing schemes are limited when facing highly correlated attributes, and cannot prevent privacy threats from untrusted servers. In this paper, we propose a novel Copula- based scheme to efficiently synthesize and release multi-dimensional crowdsourced data with local differential privacy. In our scheme, each participant's (or user's) data is locally transformed into bit strings based on a randomized response technique, which guarantees a participant's privacy on the participant (user) side. Then, Copula theory is leveraged to synthesize multi-dimensional crowdsourced data based on univariate marginal distribution and attribute dependence. Univariate marginal distribution is estimated by the Lasso-based regression algorithm from the aggregated privacy- preserving bit strings. Dependencies among attributes are modeled as multivariate Gaussian Copula, of which parameter is estimated by Pearson correlation coefficients. We conduct experiments to validate the effectiveness of our scheme. Our experimental results demonstrate that our scheme is effective for the release of multi-dimensional data with local differential privacy guaranteed to distributed participants.

In real-time monitoring systems, fine-grained measurements would pose great privacy threats to the participants as real-time measurements could disclose accurate people-centric activities. Differential privacy has been proposed to formalize and guide the design of privacy-preserving schemes. Nonetheless, due to the correlations and high fluctuations in time-series data, it is hard to achieve an effective privacy and utility tradeoff by differential privacy mechanisms. To address this issue, in this paper, we first proposed novel multi-dimensional decomposition based schemes to compress the noise and enhance the utility in differential privacy. The key idea is to decompose the measurements into multi-dimensional records and to achieve differential privacy in bounded dimensions so that the error caused by unbounded measurements can be significantly reduced. We then extended our developed scheme and developed a binary decomposition scheme for privacy-preserving time-series aggregation in real-time monitoring systems. Through a combination of extensive theoretical analysis and experiments, our data shows that our proposed schemes can effectively improve usability while achieving the same level of differential privacy than existing schemes.

Real-time monitoring systems can introduce numerous benefits to the participants in terms of performing data mining and analysis. Nonetheless, due to the correlations in time-series data, it is hard to achieve an effective privacy and utility tradeoff through a normal differential privacy mechanism. To address this issue, we propose novel multi-dimensional decomposition based schemes, which can greatly improve the utility in differential privacy. After extending the developed scheme, we then develop a binary decomposition scheme for time-series aggregation in real-time monitoring systems. Through both extensive theoretical analysis and experiments, our data shows that our proposed schemes can effectively improve usability while achieving the same level of differential privacy than existing schemes.

The smart grid poses great concern about the exposure of consumers' privacy as the fine-grained measurements in the smart metering system can expose consumer's privacy through the disclosure of accurate load profiles of home energy usage. To address this issue, in this paper we propose novel scaling perturbation based privacy-preserving schemes that can achieve great utility for fine-grained measurements in a privacy-friendly and cost-effective manner. Our schemes adopt the measurement-based scaling perturbation to hide original measurements with low cost. Through a combination of both extensive theoretical analysis and experiments, our results show that the proposed schemes can preserve consumers' privacy through fine-grained measurements and achieve a better utility-privacy tradeoff in comparison with the existing schemes.