Federated Learning (FL) enables multiple clients to collaboratively train models without sharing raw data, but is vulnerable to Byzantine attacks and data heterogeneity, which can severely degrade performance. Existing Byzantine-robust approaches tackle data heterogeneity, but incur high computational overhead during gradient aggregation, thereby slowing down the training process. To address this issue, we propose a simple yet effective Federated Normalized Gradients Algorithm (Fed-NGA), which performs aggregation by merely computing the weighted mean of the normalized gradients from each client. This approach yields a favorable time complexity of $\mathcal{O}(pM)$, where $p$ is the model dimension and $M$ is the number of clients. We rigorously prove that Fed-NGA is robust to both Byzantine faults and data heterogeneity. For non-convex loss functions, Fed-NGA achieves convergence to a neighborhood of stationary points under general assumptions, and further attains zero optimality gap under some mild conditions, which is an outcome rarely achieved in existing literature. In both cases, the convergence rate is $\mathcal{O}(1/T^{\frac{1}{2} - \delta})$, where $T$ denotes the number of iterations and $\delta \in (0, 1/2)$. Experimental results on benchmark datasets confirm the superior time efficiency and convergence performance of Fed-NGA over existing methods.

In a real federated learning (FL) system, communication overhead for passing model parameters between the clients and the parameter server (PS) is often a bottleneck. Hierarchical federated learning (HFL) that poses multiple edge servers (ESs) between clients and the PS can partially alleviate communication pressure but still needs the aggregation of model parameters from multiple ESs at the PS. To further reduce communication overhead, we remove the central PS, so that each iteration only completes model training by transmitting the global model between two adjacent ES. We call this serial learning method Sequential FL (SFL). For the first time, we introduced SFL into HFL and proposed a novel algorithm adapted to this combined framework, called Fed-CHS. Convergence results are derived for strongly convex and non-convex loss functions under various data heterogeneity setups, which show comparable convergence performance with the algorithms for HFL or SFL solely. Experimental results provide evidence of the superiority of our proposed Fed-CHS on both communication overhead saving and test accuracy over baseline methods.

This paper addresses federated learning (FL) in the context of malicious Byzantine attacks and data heterogeneity. We introduce a novel Robust Average Gradient Algorithm (RAGA), which uses the geometric median for aggregation and allows flexible round number for local updates. Unlike most existing resilient approaches, which base their convergence analysis on strongly-convex loss functions or homogeneously distributed datasets, this work conducts convergence analysis for both strongly-convex and non-convex loss functions over heterogeneous datasets. The theoretical analysis indicates that as long as the fraction of the data from malicious users is less than half, RAGA can achieve convergence at a rate of$\mathcal {O}({1}/{T^{2/3- \delta }})$for non-convex loss functions, where$T$is the iteration number and$\delta \in (0, 2/3)$. For strongly-convex loss functions, the convergence rate is linear. Furthermore, the stationary point or global optimal solution is shown to be attainable as data heterogeneity diminishes. Experimental results validate the robustness of RAGA against Byzantine attacks and demonstrate its superior convergence performance compared to baselines under varying intensities of Byzantine attacks on heterogeneous datasets.

Federated learning (FL) has emerged as an appealing machine learning approach to deal with massive raw data generated at multiple mobile devices, which needs to aggregate the training parameter of every mobile device at one base station (BS) iteratively. For parameter aggregating in FL, over-the-air computation is a spectrum-efficient solution, which allows all mobile devices to transmit their parameter-mapped signals concurrently to a BS. Due to heterogeneous channel fading and noise, there exists difference between the BS’s received signal and its desired signal, measured as the mean-squared error (MSE). To minimize the MSE, we propose to jointly optimize the signal amplification factors at the BS and the mobile devices as well as the data size (the number of data samples involved in local training) at every mobile device. The formulated problem is difficult to address due to its nonconvexity. To find the optimal solution, we perform cost function simplification and variable transformation, and solve the transformed problem in a two-level structure. Optimal solution of the lower level problem is found by analyzing every candidate solution from the Karush–Kuhn–Tucker (KKT) condition. Optimal solution of the upper level problem is found by exploring its piecewise convexity. Numerical results show that our proposed method can greatly reduce the MSE and can help to enhance the training performance of FL compared with benchmark methods.

In mobile edge computing (MEC) supporting multiple mobile users (MUs), it is essential to optimize the offloading policy and communication and computation resource allocation. A main challenge is that the computation burden of a computation task may be random and even with uncertain probabilistic distribution. To address this challenge, we investigate a multiple-MU MEC system with random computation burden. For the random computation burden of an MU, only the mean and variance are known, but its distribution is unknown. Robustness is provided such that computation outage probabilities (due to uncertain distribution of computation burden) are bounded by a predefined threshold. We minimize the weighted sum of the MUs’ energy consumption. The formulated optimization problem is non-deterministic and non-convex, and thus, is hard to solve. To deal with the challenge, we transform the formulated problem into a deterministic and convex problem by applying the Chebyshev-Cantelli inequality and some mathematical manipulations. We further decompose the convex problem to a lower-level and an upper-level problem. Low-complexity algorithms are developed for the lower-level and upper-level problems. The overall complexity of our proposed method is linear with the number of MUs.