Blockchain-assisted federated learning (BFL) can achieve decentralized storage and management of model data without relying on a central server. However, security issues caused by deliberate attacks in distributed systems and efficiency issues induced by heterogeneous computing consumption in resource-limited systems need to be urgently addressed in BFL. To address these issues, we propose a decentralized reputation management (DRM) mechanism for a trustworthy BFL (T-BFL) network, that explores, stores, and utilizes the endogenous reputation of distributed nodes to promote system security and efficiency. The proposed DRM includes three core modules, i.e., decentralized reputation evaluation, reputation-based model aggregation, and reputation-based blockchain consensus. Specifically, in the off-chain phase of T-BFL, the reputation value of each node is evaluated based on model quality, which other peer nodes can verify. This reputation value further determines the weight of global aggregation at each node. In the on-chain phase, the reputation of each node serves as the stake to dynamically adjust its consensus difficulty. Furthermore, we investigate the convergence rate of the T-BFL network under the poisoning attack, and dynamically optimize the energy allocation of local training, consensus, and communications by minimizing the upper bound of the global loss function. Extensive experiments are conducted to evaluate the performance of T-BFL on MNIST, Fashion-MNIST, and Cifar-10 datasets. The experimental results demonstrate that, compared with traditional BFL, T-BFL can achieve up to 56.12% accuracy improvement and$8.6\times $acceleration for reaching the target learning accuracy under the poisoning attack.

To defend the inference attacks and mitigate the sensitive information leakages in Federated Learning (FL), client-level Differentially Private FL (DPFL) is the de-facto standard for privacy protection by clipping local updates and adding random noise. However, existing DPFL methods tend to make a sharp loss landscape and have poor weight perturbation robustness, resulting in severe performance degradation. To alleviate these issues, we propose a novel DPFL algorithm named DP-FedSAM, which leverages gradient perturbation to mitigate the negative impact of DP. Specifically, DP-FedSAM integrates Sharpness Aware Minimization (SAM) optimizer to generate local flatness models with improved stability and weight perturbation robustness, which results in the small norm of local updates and robustness to DP noise, thereby improving the performance. To further reduce the magnitude of random noise while achieving better performance, we propose DP-FedSAM-$\operatorname{top}_{k}$topk by adopting the local update sparsification technique. From the theoretical perspective, we present the convergence analysis to investigate how our algorithms mitigate the performance degradation induced by DP. Meanwhile, we give rigorous privacy guarantees with Rényi DP, the sensitivity analysis of local updates, and generalization analysis. At last, we empirically confirm that our algorithms achieve state-of-the-art (SOTA) performance compared with existing SOTA baselines in DPFL.

Clustered vehicular federated learning (CVFL) facilitates data sharing and collaborative decision-making among vehicles, thus refining traffic behavior and demonstrating the immense potential for transforming intelligent transportation systems into a reality. However, non-independent and identically distributed data and diverse model requirements among vehicular clients hinder the feasibility of a one-size-fits-all model. Besides, “selfish” vehicular clients may be unwilling to participate in learning tasks because of the huge resource consumption of the training process. To address these challenges, in this paper, we first group local models using the adaptiveK-means-based model grouping method and then aggregate the models within each group to generate CVFL models for subsequent multi-model training. Secondly, we propose a dynamic matching-based clustering method based on the local data quality and similarity to achieve efficient vehicular client clustering. Subsequently, a meticulously crafted hierarchical incentive mechanism, grounded in a three-stage Stackelberg game, is introduced to incentivize both cluster heads and members in a layered fashion, with the initiation stemming from the CVFL server. To determine the optimal strategies for the three-stage game, an iterative algorithm is proposed, and near-optimal analytical solutions are obtained with reduced complexity. The simulation results demonstrate that our CVFL system, augmented with the hierarchical incentive mechanism, can effectively motivate multiple clusters to train multiple models in parallel, thus improving overall efficiency.

Decentralized federated learning (DFL) has the potential to address the issue of the aggregator’s single-point failure. However, in the absence of centralized coordination, DFL systems are vulnerable to malicious behaviors from clients. In this article, we propose a blockchain-based DFL framework to regulate the behaviors of enterprise clients in the context of cross-enterprise modeling. To be specific, we first design a novel mechanism for model plagiarism detection, wherein pseudonoise sequences are incorporated into local models, enabling to identify enterprises’ plagiarism behaviors. Then, we propose a model aggregation algorithm to improve the learning performance of the global model. Furthermore, we develop a plagiarism-aware proof-of-work consensus mechanism by adaptively adjusting enterprises’ mining difficulty based on their plagiarism records, which can efficiently demotivate them from engaging in plagiarism. The experimental results based on industrial datasets, including CWRU, PU, Milan, PV, NEU-CLS, and X-SDD, demonstrate that the proposed framework can achieve approximately 4%, 7%, and 12% of the learning accuracy improvement in the scenarios of 20%, 40%, and 60% plagiarism rates, respectively, compared to the conventional DFL system.

The edge intelligence (EI) has been widely applied recently. Splitting the model between device, edge server, and cloud can significantly improve the performance of EI. The model segmentation without user mobility has been investigated in detail in previous studies. However, in most EI use cases, the end devices are mobile. Few studies have been conducted on this topic. These works still have many issues, such as ignoring the energy consumption of mobile device, inappropriate network assumption, and low effectiveness on adapting user mobility, etc. Therefore, to address the disadvantages of model segmentation and resource allocation in previous studies, we propose mobility and cost aware model segmentation and resource allocation algorithm for accelerating the inference at edge (MCSA). Specifically, in the scenario without user mobility, the loop iteration gradient descent (Li-GD) algorithm is provided. When the mobile user has a large model inference task that needs to be calculated, it will take the energy consumption of mobile user, the communication and computing resource renting cost, and the inference delay into account to find the optimal model segmentation and resource allocation strategy. In the scenario with user mobility, the mobility aware Li-GD (MLi-GD) algorithm is proposed to calculate the optimal strategy. Then, the properties of the proposed algorithms are investigated, including convergence, complexity, and approximation ratio. The experimental results demonstrate the effectiveness of the proposed algorithms.

Traditional federated learning (FL) frameworks rely on a central server for model coordination among distributed mobile terminals (MTs). The centralization faces two critical challenges, i.e., single point of failure and potential privacy leakage. Differentially private decentralized FL (DP-DFL) has been proposed to address these challenges, wherein the MTs exchange models in a decentralized manner and maintain the differential privacy (DP) guarantee by adding noise to local models before model interaction. However, existing DP-DFL frameworks confront difficulty in achieving the expected privacy and convergence performance, simultaneously. To address this issue, we propose a novel DP-DFL framework (called randomized DP-DFL) that employs a randomized model interaction scheme to lower the model exposure frequency and hence reduce privacy budget consumption. Specifically, the scheme includes two sequential steps, i.e., randomized terminal assignment and randomized model transmission. In Step 1), the model interaction phase of DFL is further divided into several sequential substages. MTs are randomly assigned to each sub-stage. In Step 2), each MT sequentially transmits either a model previously received from its neighbors or its own local model according to the assigned sub-stage order. The proposed scheme enhances the MTs' privacy of DFL since the exposure probabilities of the MTs' local models are significantly reduced via these two randomized steps. Besides, we theoretically analyze the convergence and privacy performance of randomized DP-DFL. In particular, properly tuning the number of sub-stages in randomized DP-DFL can achieve an optimal balance between privacy and convergence. Experimental results show that randomized DP-DFL consistently outperforms traditional frameworks. Compared with baselines, randomized DP-DFL reduces 40.9% privacy loss under the same target accuracy while improving 9.5% learning accuracy under the same privacy loss on EMNIST and CIFAR-10, respectively

Federated learning (FL) based on the centralized design faces both challenges regarding the trust issue and a single point of failure. To alleviate these issues, blockchain-aided decentralized FL (BDFL) introduces the decentralized network architecture into the FL training process, which can effectively overcome the defects of centralized architecture. However, deploying BDFL in wireless networks usually encounters challenges, such as limited bandwidth, computing power, and energy consumption. Driven by these considerations, a dynamic stochastic optimization problem is formulated to minimize the average training delay by jointly optimizing the resource allocation and client selection under the constraints of limited energy budget and client participation. We solve the long-term mixed integer nonlinear programming problem by employing the tool of Lyapunov optimization and thereby propose the dynamic resource allocation and client scheduling BDFL (DRC-BDFL) algorithm. Furthermore, we analyse the learning performance of DRC-BDFL and derive an upper bound for convergence regarding the global loss function. Extensive experiments conducted on the SVHN and CIFAR-10 data sets demonstrate that the DRC-BDFL achieves comparable accuracy to the baseline algorithms while significantly reducing the training delay by 9.24% and 12.47%, respectively.

Driven by data and models, the digital twin technique presents a new concept of optimizing system design, process monitoring, decision-making and more, through performing comprehensive virtual-reality interaction and continuous mapping. By introducing serverless computing to Mobile Edge Computing (MEC) environments, the emerging serverless edge computing paradigm facilitates the communication-efficient digital twin services and promises agile, fine-grained and cost-efficient provisioning of limited edge resources, where serverless functions are implemented by containers in cloudlets (edge servers). However, the nonnegligible cold start delay of containers deteriorates the responsiveness of digital twin services dramatically and the perceived user service experience. In this paper, we investigate delay-sensitive query service provisioning in digital twin-empowered serverless edge computing by considering user mobility. With digital twins of users deployed in the remote cloud, referred to as primary digital twins, we deploy their digital twin replicas based on serverless functions in cloudlets to mitigate the query service delay while enhancing user service satisfaction that is expressed as a utility function. We study two optimization problems with the aim of maximizing the accumulative utility gain: the digital twin replica placement problem per time slot, and the dynamic digital twin replica placement problem over a finite time horizon. We first formulate an Integer Linear Program (ILP) solution for the digital twin replica placement problem when the problem size is small; otherwise, we propose an approximation algorithm for the problem with a provable approximation ratio. We then design an online algorithm for the dynamic digital twin replica placement problem, and a performance-guaranteed online algorithm for a special case of the problem by assuming each user issues a query at each time slot. Finally, we evaluate the performance of the proposed algorithms for placing digital twin replicas in MEC networks through simulations. The results demonstrate the proposed algorithms are promising, outperforming their counterparts.

Federated learning (FL), as a type of distributed machine learning, is vulnerable to external attacks during parameter transmissions between learning agents and a model aggregator. In particular, malicious participant clients in FL can purposefully craft their uploaded model parameters to manipulate system outputs, which is know as a model poisoning (MP) attack. In this paper, we propose effective MP algorithms to attack the classical defensive aggregation Krum at the aggregator. The proposed algorithms are designed to evade detection, i.e., covert MP (CMP). Specifically, we first formulate the MP as an optimization problem by minimizing the Euclidean distance between the manipulated model and designated one, constrained by Krum. Then, we develop CMP algorithms against the Krum based on the solutions of this optimization problem. Furthermore, to reduce the optimization complexity, we propose low complexity CMP algorithms having only a slight performance degradation. Our experimental results demonstrate that the proposed CMP algorithms are effective and can substantially outperform existing attack mechanisms, such as Arjun's attack and the label flipping attack. More specifically, our original CMP can achieve a high rate of the attacker's accuracy ($\approx 90\%$). For example, in our experiments using the MNIST dataset, the proposed CMP attacking algorithm against Krum can successfully manipulate the aggregated model to incorrectly classify a given digit as a different one (e.g., 9 as 8). Meanwhile, our CMP algorithm with an approximated constraint can achieve a rate of 87% in terms of the attacker's accuracy (attacker-desired results), with a 73% complexity reduction compared to the original CMP.

Internet of Vehicles (IoV) plays a vital role in alleviating traffic issues. Object detection is one of the key technologies in IoV, which has been widely used to provide traffic management services by analyzing timely and sensitive vehicle-related information. However, the current object detection methods mostly rely on centralized deep training, that is, the sensitive data obtained by edge devices needs to be uploaded to the server, which raises latency and privacy issues. To tackle these issues, we propose to accomplish object detection through sparse federated training with dynamic model aggregation, namely FedWeg, to reduce the communication cost and privacy leakage induced by data transmission. Specifically, FedWeg performs sparse training in edge devices and uploads the lightweight models to the server. To reduce the unnecessary transmission overhead, we propose a dynamic sparsity adjustment scheme that gradually increases the sparsity ratios. Then, we propose to utilize the inverse ratio of sparsity ratios from different edge devices to calculate aggregate weights to diminish the negative impact of sparse training on learning performance. Moreover, we theoretically analyze the convergence rate of FedWeg, which reveals that the impact of network sparsity on model performance, and higher average sparsity rates result in greater errors. Finally, we conduct extensive experiments on four real-life datasets using YOLOv3 and VGG-16. The results show that our FedWeg algorithm outperforms baselines in terms of communication costs and test accuracy.

Federated learning (FL) can train collaboratively with several mobile terminals (MTs), which faces critical challenges in communication, resource, and privacy. Existing privacy-preserving methods usually adopt instance-level differential privacy (DP), which provides a rigorous privacy guarantee but with several bottlenecks: performance degradation, transmission overhead, and resource constraints. Therefore, we propose Fed-LTP, an efficient and privacy-enhanced FL framework withLotteryTicketHypothesis (LTH) and zero-concentrated DP(zCDP). It generates a pruned global model on the server side and conducts sparse-to-sparse training from scratch with zCDP on the client side. On the server side, two pruning schemes are proposed: (i) the weight-based pruning (LTH) determines the pruned global model structure; (ii) the iterative pruning further shrinks the size of the pruned model. Meanwhile, the performance of Fed-LTP is boosted via model validation based on the Laplace mechanism. On the client side, we use sparse-to-sparse training to solve the resource-constraints issue and provide tighter privacy analysis to reduce the privacy budget. We evaluate the effectiveness of Fed-LTP on several real-world datasets in both independent and identically distributed (IID) and non-IID settings. The results confirm the superiority of Fed-LTP over state-of-the-art (SOTA) methods in communication, computation, and memory efficiencies while realizing a better utility-privacy trade-off.

Age of Information (AoI) has emerged as a new metric to measure data freshness from the destination's perspective. To optimize the system AoI, most existing works focused on the point of scheduling of update transmissions. While at wireless-powered network edge, the source nodes can only transmit their updates after being charged ready, which means the system AoI is not only determined by the update transmission strategies, but also the charging strategies. Thus, in this paper, we investigate the first work to optimize the weighted peak AoI from the point of charging at wireless-powered network edge. Firstly, the problem of optimizing the weighted sum of average peak AoI with a directional charger is formulated, and then transformed to a charging time optimization problem with respect to the charging orientations and peak AoI, and an approximate algorithm is proposed to obtain the required charging time for each source node. Secondly, an age-based scheduling algorithm is proposed to compute the charging decisions and transmission decisions simultaneously, which can not only optimize the weighted sum of average peak AoI, but also guarantee the maximum peak AoI of each source node is bounded. The proposed algorithm is proved to have an approximation ratio of up to (1+$\varphi$), where$\varphi$is a small value related to the weight of each source node. When there exist multiple chargers, an approximate algorithm is also proposed to minimize the weighted sum of average peak AoI by scheduling the orientations of these chargers cooperatively. Finally, the extensive simulations demonstrate the high performance of the proposed algorithms in terms of peak AoI.

In decentralized federated learning (DFL), clients exchange their models with each other for global aggregation. Due to a lack of centralized supervision, a client may easily duplicate shared models to save its computing resources. Generally, this plagiarism behavior is hard to detect, while it is harmful to model training performance. To address this issue, we propose an anti-plagiarism DFL framework to efficiently detect plagiarism misconduct. Specifically, we first design a method for detecting plagiarism by adding a time-shift pseudo-noise (PN) sequence to each client's local model before broadcasting. Second, we develop an upper bound of the loss function of DFL with the proposed PN sequence detection method, which is proved to be the convex function of both the amplitude of PN sequences ($\alpha$) and the detection threshold ($\lambda$). Next, we propose an adaptive plagiarism detection (APD) algorithm by jointly optimizing$\alpha$and$\lambda$to enhance the learning performance. Finally, we conduct extensive experiments on MNIST, Adult, Cifar-10, and SVHN datasets to demonstrate that our analytical bounds are consistent with the experimental results. Remarkably, the proposed framework can recover up to a 10% classification accuracy loss in the presence of 40% plagiaristic clients.

With the rapid proliferation of smart mobile devices, federated learning (FL) has been widely considered for application in wireless networks for distributed model training. However, data heterogeneity, e.g., non-independently identically distributions and different sizes of training datasets among clients, poses major challenges to wireless FL. Limited communication resources complicate the implementation of fair scheduling which is required for training on heterogeneous data, and further deteriorate the overall performance. To address this issue, this paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation. Specifically, we first develop a closed-form expression for an upper bound on the FL loss function, with a particular emphasis on data heterogeneity described by a dataset size vector and a data divergence vector. Then we formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, uplink transmission power, channel allocation and the number of local epochs. Next, via the Lyapunov drift technique, we transform the optimization problem into a series of tractable problems. Extensive experiments on real-world datasets demonstrate that our method outperforms other benchmarks in terms of the learning accuracy and energy consumption.

To defend the inference attacks and mitigate the sensitive information leakages in Federated Learning (FL), clientlevel Differentially Private FL (DPFL) is the de-facto standard for privacy protection by clipping local updates and adding random noise. However, existing DPFL methods tend to make a sharper loss landscape and have poorer weight perturbation robustness, resulting in severe performance degradation. To alleviate these issues, we propose a novel DPFL algorithm named DP-FedSAM, which leverages gradient perturbation to mitigate the negative impact of DP. Specifically, DP-FedSAM integrates Sharpness Aware Minimization (SAM) optimizer to generate local flatness models with better stability and weight perturbation robustness, which results in the small norm of local updates and robustness to DP noise, thereby improving the performance. From the theoretical perspective, we analyze in detail how DP-FedSAM mitigates the performance degradation induced by DP. Meanwhile, we give rigorous privacy guarantees with Rényi DP and present the sensitivity analysis of local updates. At last, we empirically confirm that our algorithm achieves state-of-the-art (SOTA) performance compared with existing SOTA baselines in DPFL.

To mitigate the privacy leakages and communication burdens of Federated Learning (FL), decentralized FL (DFL) discards the central server and each client only communicates with its neighbors in a decentralized communication network. However, existing DFL suffers from high inconsistency among local clients, which results in severe distribution shift and inferior performance compared with centralized FL (CFL), especially on heterogeneous data or sparse communication topologies. To alleviate this issue, we propose two DFL algorithms named DFedSAM and DFedSAM-MGS to improve the performance of DFL. Specifically, DFedSAM leverages gradient perturbation to generate local flat models via Sharpness Aware Minimization (SAM), which searches for models with uniformly low loss values. DFedSAM-MGS further boosts DFedSAM by adopting Multiple Gossip Steps (MGS) for better model consistency, which accelerates the aggregation of local flat models and better balances communication complexity and generalization. Theoretically, we present improved convergence rates $\small \mathcal{O}\big(\frac{1}{\sqrt{KT}}+\frac{1}{T}+\frac{1}{K^{1/2}T^{3/2}(1-\lambda)^2}\big)$ and $\small \mathcal{O}\big(\frac{1}{\sqrt{KT}}+\frac{1}{T}+\frac{\lambda^Q+1}{K^{1/2}T^{3/2}(1-\lambda^Q)^2}\big)$ in non-convex setting for DFedSAM and DFedSAM-MGS, respectively, where $1-\lambda$ is the spectral gap of gossip matrix and $Q$ is the number of MGS. Empirically, our methods can achieve competitive performance compared with CFL methods and outperform existing DFL methods.

Federated self-supervised learning (FedSSL) is an emerging method in the domain of machine learning. It collaboratively learns a powerful feature extractor among multiple participants by utilizing distributed unlabeled data. However, conventional FedSSL suffers from statistical heterogeneity due to the non-independent and identically distributed (Non-IID) data among participants. In this work, we introduce a novel method to tackle the Non-IID data issue in FedSSL. First, the relation knowledge distillation is utilized to enhance the learning from global models. Then, we dynamically update the local model with divergence-aware update (DAU) method to preserve the client’s knowledge of Non-IID data. Our experimental results demonstrate that the proposed approach outperforms other methods by up to 8% on linear evaluation, verifying the effectiveness of our approach.

This paper proposes a dynamic resource trading scheme in unmanned aerial vehicle (UAV)-assisted mobile edge computing (MEC) network. A UAV-assisted MEC server adaptively adjusts its trajectory to sell the computation offloading services to the mobile users (MUs), where the MUs have stochastic task arrivals. In this context, we formulate the sequential resource trading problem as a stochastic Stackelberg game, which is composed of two stages for each trading round. In the first stage, the self-interested UAV jointly optimizes its trajectory and service price to maximize its long-term profits. In the second stage, the non-cooperative MUs optimize their binary offloading decisions to minimize the average task processing delay and service payment. However, it is challenging to obtain the equilibrium across the fully decentralized agents with constantly evolving and tightly coupled policies, where each agent is confronted with a non-stationary environment. To solve this problem, we propose an opponent modeling based double deep Q learning (OM-DDQN) algorithm, where each agent adopts opponent modeling to effectively predict the trading strategies of other agents in the network. Simulation results demonstrate that, compared with the baseline algorithms, the proposed algorithm can achieve a win-win resource trading outcome that not only enhances the UAV's profit but also reduces the MUs' costs.

Federated Learning (FL) empowers Industrial Internet of Things (IIoT) with distributed intelligence of industrial automation thanks to its capability of distributed machine learning without any raw data exchange. However, it is rather challenging for lightweight IIoT devices to perform computation-intensive local model training over large-scale deep neural networks (DNNs). Driven by this issue, we develop a communication-computation efficient FL framework for resource-limited IIoT networks that integrates DNN partition technique into the standard FL mechanism, wherein IIoT devices perform local model training over the bottom layers of the objective DNN, and offload the top layers to the edge gateway side. Considering imbalanced data distribution, we derive the device-specific participation rate to involve the devices with better data distribution in more communication rounds. Upon deriving the device-specific participation rate, we propose to minimize the training delay under the constraints of device-specific participation rate, energy consumption and memory usage. To this end, we formulate a joint optimization problem of device scheduling and resource allocation (i.e. DNN partition point, channel assignment, transmit power, and computation frequency), and solve the long-term min-max mixed integer non-linear programming based on the Lyapunov technique. In particular, the proposed dynamic device scheduling and resource allocation (DDSRA) algorithm can achieve a trade-off to balance the training delay minimization and FL performance. We also provide the FL convergence bound for the DDSRA algorithm with both convex and non-convex settings. Experimental results demonstrate the derived device-specific participation rate in terms of feasibility, and show that the DDSRA algorithm outperforms baselines in terms of test accuracy and convergence time.

Motivated by the advancing computational capacity of distributed end-user equipment (UE), as well as the increasing concerns about sharing private data, there has been considerable recent interest in machine learning (ML) and artificial intelligence (AI) that can be processed on distributed UEs. Specifically, in this paradigm, parts of an ML process are outsourced to multiple distributed UEs. Then, the processed information is aggregated on a certain level at a central server, which turns a centralized ML process into a distributed one and brings about significant benefits. However, this new distributed ML paradigm raises new risks in terms of privacy and security issues. In this article, we provide a survey of the emerging security and privacy risks of distributed ML from a unique perspective of information exchange levels, which are defined according to the key steps of an ML process, i.e., we consider the following levels: 1) the level of preprocessed data; 2) the level of learning models; 3) the level of extracted knowledge; and 4) the level of intermediate results. We explore and analyze the potential of threats for each information exchange level based on an overview of current state-of-the-art attack mechanisms and then discuss the possible defense methods against such threats. Finally, we complete the survey by providing an outlook on the challenges and possible directions for future research in this critical area.

Generative adversarial networks (GANs) have attracted increasing attention recently owing to their impressive abilities to generate realistic samples with high privacy protection. Without directly interacting with training examples, the generative model can be used to estimate the underlying distribution of an original dataset while the discriminator can examine model quality of the generated samples by comparing the label values with training examples. In considering privacy issues in GANS, existing works focus on perturbing the parameters and analyzing the corresponding privacy protection capability, and the parameters are not directly exchanged between the generator and discriminator in GANs. Thus, in this work, we propose a Rényi-differentially private-GAN (RDP-GAN), which achieves differential privacy (DP) in a GAN by carefully adding random Gaussian noise to the value of the exchanged loss function during training. Moreover, we derive analytical results characterizing the total privacy loss under the subsampling method and cumulative iterations, which show its effectiveness for the privacy budget allocation. In addition, in order to mitigate the negative impact of injecting noises, we enhance the proposed algorithm by adding an adaptive noise tuning step, which will change the amount of added noise according to the testing accuracy. Through extensive experimental results, we verify that the proposed algorithm can achieve a better privacy level while producing high-quality samples compared with a benchmark DP-GAN scheme based on noise perturbation on training gradients.

While preserving the privacy of federated learning (FL), differential privacy (DP) inevitably degrades the utility (i.e., accuracy) of FL due to model perturbations caused by DP noise added to model updates. Existing studies have considered exclusively noise with persistent root-mean-square amplitude and overlooked an opportunity of adjusting the amplitudes to alleviate the adverse effects of the noise. This paper presents a new DP perturbation mechanism with a time-varying noise amplitude to protect the privacy of FL and retain the capability of adjusting the learning performance. Specifically, we propose a geometric series form for the noise amplitude and reveal analytically the dependence of the series on the number of global aggregations and the (ϵ,δ)-DP requirement. We derive an online refinement of the series to prevent FL from premature convergence resulting from excessive perturbation noise. Another important aspect is an upper bound developed for the loss function of a multi-layer perceptron (MLP) trained by FL running the new DP mechanism. Accordingly, the optimal number of global aggregations is obtained, balancing the learning and privacy. Extensive experiments are conducted using MLP, supporting vector machine, and convolutional neural network models on four public datasets. The contribution of the new DP mechanism to the convergence and accuracy of privacy-preserving FL is corroborated, compared to the state-of-the-art Gaussian noise mechanism with a persistent noise amplitude.

Personalized federated learning (PFL), as a novel federated learning (FL) paradigm, is capable of generating personalized models for heterogenous clients. Combined with with a meta-learning mechanism, PFL can further improve the convergence performance with few-shot training. However, meta-learning based PFL has two stages of gradient descent in each local training round, therefore posing a more serious challenge in information leakage. In this paper, we propose a differential privacy (DP) based PFL (DP-PFL) framework and analyze its convergence performance. Specifically, we first design a privacy budget allocation scheme for inner and outer update stages based on the Rényi DP composition theory. Then, we develop two convergence bounds for the proposed DP-PFL framework under convex and non-convex loss function assumptions, respectively. Our developed convergence bounds reveal that 1) there is an optimal size of the DP-PFL model that can achieve the best convergence performance for a given privacy level, and 2) there is an optimal tradeoff among the number of communication rounds, convergence performance and privacy budget. Evaluations on various real-life datasets demonstrate that our theoretical results are consistent with experimental results. The derived theoretical results can guide the design of various DP-PFL algorithms with configurable tradeoff requirements on the convergence performance and privacy levels.

Blockchain technology has been extensively studied to enable distributed and tamper-proof data processing in federated learning (FL). Most existing blockchain assisted FL (BFL) frameworks have employed a third-party blockchain network to decentralize the model aggregation process. However, decentralized model aggregation is vulnerable to pooling and collusion attacks from the third-party blockchain network. Driven by this issue, we propose a novel BFL framework that features the integration of training and mining at the client side. To optimize the learning performance of FL, we propose to maximize the long-term time average (LTA) training data size under a constraint of LTA energy consumption. To this end, we formulate a joint optimization problem of training client selection and resource allocation (i.e., the transmit power and computation frequency at the client side), and solve the long-term mixed integer non-linear program based on a Lyapunov technique. In particular, the proposed dynamic resource allocation and client scheduling (DRACS) algorithm can achieve a trade-off of [$\mathcal {O}(1/V)$,$\mathcal {O}(\sqrt {V})$] to balance the maximization of the LTA training data size and the minimization of the LTA energy consumption with a control parameter$V$. Our experimental results show that the proposed DRACS algorithm achieves better learning accuracy than benchmark client scheduling strategies with limited time or energy consumption.

Federated learning (FL) has become a promising machine learning (ML) paradigm for training machine learning models over distributed datasets, owing to its low communication costs and privacy preserving property. To date, the most commonly adopted model fusion mechanism in FL is average aggregation. However, it has been shown that this average aggregation mechanism performs poorly in heterogeneous systems, especially for non-independent and identically distributed (NonIID) data. In order to address this challenge, we propose a weighted FL model aggregation strategy for each client based on clustering, termed CluFL. Specifically, CluFL first measures the similarities among uploaded models from clients through their parameters using a spectral clustering algorithm. Then, CluFL assigns aggregation weights according to the similarity of the intra-cluster global model for each cluster and the average model across the clusters. Further, we derive a convergence bound on the CluFL algorithm considering a practical nonconvex setting of neural network training. This bound reveals that the proposed CluFL algorithm can achieve a convergence speed in the order of O(1/T). Extensive experiments have been conducted on both FashionMNIST and CIFAR-10 datasets and show that CluFL outperforms the state-of-the-art FL algorithms in terms of accuracy and communication efficiency.

Abstract The error‐correcting performance of multi‐level‐cell (MLC) NAND flash memory is closely related to the block length of error‐correcting codes (ECCs) and log‐likelihood‐ratios of the read‐voltage thresholds. Driven by this issue, this paper optimizes the read‐voltage thresholds for MLC flash memory to improve the decoding performance of ECCs with finite block length. First, through the analysis of channel coding rate and decoding error probability under finite block length, the optimization problem of read‐voltage thresholds to minimize the maximum decoding error probability is formulated. Second, a cross‐iterative search algorithm to optimize read‐voltage thresholds under the perfect knowledge of flash memory channel is developed. However, it is challenging to analytically characterize the voltage distribution under the effect of data retention noise. To address this problem, a deep neural network (DNN)‐aided optimization strategy to optimize the read‐voltage thresholds is developed, where a multi‐layer perception network is employed to learn the relationship between voltage distribution and read‐voltage thresholds. Simulation results show that, compared with the existing schemes, the proposed DNN‐aided read‐voltage threshold optimization strategy with a well‐designed Low Density Parity Check (LDPC) code can not only improve the program‐and‐erase endurance but also reduce the read latency.

In federated learning (FL), model training is distributed over clients and local models are aggregated by a central server. The performance of uploaded models in such situations can vary widely due to imbalanced data distributions, potential demands on privacy protections, and quality of transmissions. In this paper, we aim to minimize FL training delay over wireless channels, constrained by overall training performance as well as each client’s differential privacy (DP) requirement. We solve this problem in a multi-agent multi-armed bandit (MAMAB) framework to deal with the situation where there are multiple clients confronting different unknown transmission environments, e.g., channel fading and interference. Specifically, we first transform long-term constraints on both training performance and each client’s DP into a virtual queue based on the Lyapunov drift technique. Then, we convert the MAMAB to a max-min bipartite matching problem at each communication round, by estimating rewards with the upper confidence bound (UCB) approach. More importantly, we propose two efficient solutions to this matching problem, i.e., a modified Hungarian algorithm and greedy matching with a better alternative (GMBA), of which the former can achieve the optimal solution with high complexity while the latter approaches a better trade-off by enabling verified low-complexity with little performance loss. In addition, we develop an upper bound on the expected regret of this MAMAB based FL framework, which shows a linear growth over the logarithm of communication rounds, justifying its theoretical feasibility. Extensive experimental results are conducted to validate the effectiveness of our proposed algorithms, and the impacts of various parameters on the FL performance over wireless edge networks are also discussed.

Federated learning (FL), as a type of collaborative machine learning framework, is capable of preserving private data from mobile terminals (MTs) while training the data into useful models. Nevertheless, from a viewpoint of information theory, it is still possible for a curious server to infer private information from the shared models uploaded by MTs. To address this problem, we first make use of the concept of local differential privacy (LDP), and propose a user-level differential privacy (UDP) algorithm by adding artificial noise to the shared models before uploading them to servers. According to our analysis, the UDP framework can realize$(\epsilon _{i}, \delta _{i})$-LDP for the$i$th MT with adjustable privacy protection levels by varying the variances of the artificial noise processes. We then derive a theoretical convergence upper-bound for the UDP algorithm. It reveals that there exists an optimal number of communication rounds to achieve the best learning performance. More importantly, we propose a communication rounds discounting (CRD) method. Compared with the heuristic search method, the proposed CRD method can achieve a much better trade-off between the computational complexity of searching and the convergence performance. Extensive experiments indicate that our UDP algorithm using the proposed CRD method can effectively improve both the training efficiency and model quality for the given privacy protection levels.

Federated learning (FL), as a distributed machine learning paradigm, promotes personal privacy by local data processing at each client. However, relying on a centralized server for model aggregation, standard FL is vulnerable to server malfunctions, untrustworthy servers, and external attacks. To address these issues, we propose a decentralized FL framework by integrating blockchain into FL, namely, blockchain assisted decentralized federated learning (BLADE-FL). In a round of the proposed BLADE-FL, each client broadcasts its trained model to other clients, aggregates its own model with received ones, and then competes to generate a block before its local training on the next round. We evaluate the learning performance of BLADE-FL, and develop an upper bound on the global loss function. Then we verify that this bound is convex with respect to the number of overall aggregation rounds$K$, and optimize the computing resource allocation for minimizing the upper bound. We also note that there is a critical problem of training deficiency, caused by lazy clients who plagiarize others’ trained models and add artificial noises to disguise their cheating behaviors. Focusing on this problem, we explore the impact of lazy clients on the learning performance of BLADE-FL, and characterize the relationship among the optimal$K$, the learning parameters, and the proportion of lazy clients. Based on the MNIST and Fashion-MNIST datasets, we see that the experimental results are consistent with the analytical ones. To be specific, the gap between the developed upper bound and experimental results is lower than$5\%$, and the optimized$K$based on the upper bound can effectively minimize the loss function.

Owing to the low communication costs and privacy-promoting capabilities, federated learning (FL) has become a promising tool for training effective machine learning models among distributed clients. However, with the distributed architecture, low-quality models could be uploaded to the aggregator server by unreliable clients, leading to a degradation or even a collapse of training. In this article, we model these unreliable behaviors of clients and propose a defensive mechanism to mitigate such a security risk. Specifically, we first investigate the impact on the models caused by unreliable clients by deriving a convergence upper bound on the loss function based on the gradient descent updates. Our bounds reveal that with a fixed amount of total computational resources, there exists an optimal number of local training iterations in terms of convergence performance. We further design a novel defensive mechanism, named deep neural network-based secure aggregation (DeepSA). Our experimental results validate our theoretical analysis. In addition, the effectiveness of DeepSA is verified by comparing with other state-of-the-art defensive mechanisms.

Federated learning (FL), as a type of distributed machine learning, is capable of significantly preserving clients’ private data from being exposed to adversaries. Nevertheless, private information can still be divulged by analyzing uploaded parameters from clients, e.g., weights trained in deep neural networks. In this paper, to effectively prevent information leakage, we propose a novel framework based on the concept of differential privacy (DP), in which artificial noise is added to parameters at the clients’ side before aggregating, namely, noising before model aggregation FL (NbAFL). First, we prove that the NbAFL can satisfy DP under distinct protection levels by properly adapting different variances of artificial noise. Then we develop a theoretical convergence bound on the loss function of the trained FL model in the NbAFL. Specifically, the theoretical bound reveals the following three key properties: 1) there is a tradeoff between convergence performance and privacy protection levels, i.e., better convergence performance leads to a lower protection level; 2) given a fixed privacy protection level, increasing the number$N$of overall clients participating in FL can improve the convergence performance; and 3) there is an optimal number aggregation times (communication rounds) in terms of convergence performance for a given protection level. Furthermore, we propose a$K$-client random scheduling strategy, where$K$($1\leq K< N$) clients are randomly selected from the$N$overall clients to participate in each aggregation. We also develop a corresponding convergence bound for the loss function in this case and the$K$-client random scheduling strategy also retains the above three properties. Moreover, we find that there is an optimal$K$that achieves the best convergence performance at a fixed privacy level. Evaluations demonstrate that our theoretical results are consistent with simulations, thereby facilitating the design of various privacy-preserving FL algorithms with different tradeoff requirements on convergence performance and privacy levels.

In this paper, we propose an effective read-voltage optimization method for multi-level-cell (MLC) NAND flash memory to improve the performance of error correcting codes (ECCs) with finite blocklength. Specifically, we first obtain the maximal channel coding rate achievable at a given blocklength and error probability of quantized channel. Based on this finite-blocklength channel-coding rate (FCR), we convert the optimization problem into minimizing the error probability instead of the channel coding rate. Then, we develop a cross iterative search (CIS) method and the genetic algorithm to solve this optimization problem. In our simulations, for a well-designed LDPC code, our read-voltage optimization method improves program-and-erase (PE) endurance up to about 900 and 600 cycles against the maximizing the mutual information (MMI) and entropy-based optimization methods, respectively, at a frame-error-rate (FER) of 2×10-4.