Federated learning, as a well-known framework for collaborative training among distributed local sensors and devices, has been widely used in practical learning applications. To reduce communication resource consumption and training delay, acceleration training algorithms, especially momentum-based methods, are further developed for the training process. However, it is observed that under the influence of transmission noise, existing momentum methods exhibit poor training performance due to the noise accumulation along with the momentum term. This motivates us to propose a novel acceleration algorithm to achieve an efficient trade-off between the training acceleration and noise smoothing. Specifically, to obtain clearer insights into the model update dynamics, we utilize a stochastic differential equation model to mimic the discrete-time training trajectory. Through high-order drift approximation analysis on a general momentum-based SDE model, we propose a dynamic momentum weight and gradient stepsize design for the update rule, which is adaptive to both the training state and gradient quality. Such adaptation ensures that the training algorithm can seize good update opportunities and avoid noise explosion. The corresponding discrete-time training algorithm is then derived via discretization of the proposed SDE model, which shows a superior training performance compared to state-of-the-art baselines.

The performance capabilities of models trained in a federated learning (FL) setting over wireless networks can be significantly affected by the underlying properties of the transmission channel. Even for shallow models, there can be an acute degradation in performance which necessitates the development of algorithms which are robust to transmission channel effects, such as noise and fading. In this work, we present a two-pronged approach to overcome the limitations of existing wireless machine learning (ML)-based algorithms. First, to tackle the effect of channel noise, we incorporate a novel tracking-based stochastic approximation scheme in the standard federated averaging pipeline which averages out the effect of the channel noise. In contrast to previous works on FL with a noisy channel, we provide exact convergence guarantees for our algorithm without the need to increase the transmission power gain. Second, to combat channel fading and further optimize the power consumption at the client level, we propose an adaptive transmission policy obtained by solving an optimization problem with long-term constraints. The solution is obtained in an online manner via a dual decomposition method. The superior empirical performance of the proposed scheme compared to state-of-the-art works is demonstrated on standard ML tasks.

Over-the-air gradient aggregation and data-aware scheduling have recently drawn great attention due to the outstanding performance in improving communication efficiency for federated edge learning applications. However, in this case, the estimated gradient suffers from the channel and data distortion induced by channel fading and data-aware scheduling, which introduces significant bias and harms the training performance. To solve these problems, we propose a dynamic data and channel adaptive sensor scheduling and power control algorithm combining a residual feedback mechanism. Instead of discarding the gradients not transmitted to the central server, each sensor keeps track of a local residual to store these gradients. Furthermore, by connecting the model update iterations to a dynamic evolution process, we utilize the Lyapunov drift optimization method to analyze the relationship between the training gain and resource allocation. The derived decentralized optimal solution is adaptive to both the channel state information and data importance to seize good transmission opportunity and important gradients. Theoretical analysis is provided on the convergence of the proposed algorithm in practical training scenarios. Simulation results further illustrate that under the same power cost, the proposed scheme has a much faster convergence rate and lower training loss compared to existing baselines.

Considering the wide application of multiple types of sensors with diversified data sensing and collection capabilities, we focus on the resulting hybrid data partitioning among the local data set distributed at the edge sensors, especially the practical training implementation of federated learning (FL) under such a setting, where the neural network (NN) is trained collaboratively without requiring the sensors to share their data. Different from the conventional FL schemes, since each local sensor now only has partial data samples with type-specific features, the traditional stochastic gradient descent (SGD)-based training method cannot be directly utilized due to the intertype and intratype data coupling. To address this issue, we first transform the training problem into the primal–dual domain utilizing the corresponding Lagrangian and propose a stochastic primal-descent dual-ascent training method with a two-side residual feedback mechanism. Such a method can be implemented in a scalable way and compensate for the data distortion and loss caused by the practical transmission noise. Furthermore, a decentralized joint scheduling, bandwidth allocation, and dynamic quantization policy is proposed by analyzing the performance at each training iteration and the consumed transmission resources. The proposed method is adaptive to not only the channel state information (CSI) but also the instantaneous gradient importance and dynamic gradient statistics. The closed-form convergence analysis is provided, and the simulation experiments illustrate the superior performance of the proposed scheme.

Federated learning (FL) is a machine learning framework, where multiple distributed edge Internet of Things (IoT) devices collaboratively train a model under the orchestration of a central server while keeping the training data distributed on the IoT devices. FL can mitigate the privacy risks and costs from data collection in traditional centralized machine learning. However, the deployment of standard FL is hindered by the expense of the communication of the gradients from the devices to the server. Hence, many gradient compression methods have been proposed to reduce the communication cost. However, the existing methods ignore the structural correlations of the gradients and, therefore, lead to a large compression loss which will decelerate the training convergence. Moreover, many of the existing compression schemes do not enable over-the-air aggregation and, hence, require huge communication resources. In this work, we propose a gradient compression scheme, named FedOComp, which leverages the correlations of the stochastic gradients in FL systems for efficient compression of the high-dimension gradients with over-the-air aggregation. The proposed design can achieve a smaller deceleration of the training convergence compared to other gradient compression methods since the compression kernel exploits the structural correlations of the gradients. It also directly enables over-the-air aggregation to save communication resources. The derived convergence analysis and simulation results further illustrate that under the same power cost, the proposed scheme has a much faster convergence rate and higher test accuracy compared to existing baselines.

Federated learning (FL) has been widely utilized to leverage the distributed dataset and processing capability of local sensors while preserving data privacy. Considering the utilization of multiple groups of sensors with diverse sensing functions in IoT wireless networks, we focus on a new hybrid data partitioning scenario, where each sensor can only obtain partial data samples on type-specific feature space. This results in the combined sample parallelism among same-type sensors and feature parallelism among different types of sensors, which brings challenges to designing scalable and communication-efficient training algorithms. Different from the conventional FL settings, we transform the training problem to the primal-dual domain and propose a novel hierarchical FL framework where both intra-type and inter-type over-the-air collaboration between local sensors are utilized to exploit the sample and feature diversity. Simulation results illustrate the importance of such collaborative training and the efficiency of the proposed transmission scheme.

Emerging hardware technology enables the utilization of a large number of multitype sensors with diverse sensing capabilities for data collection and the training of AI models. Each sensor collects partial data samples on a type-specific feature space, which results in hybrid data partitioning across the local datasets and brings challenges to developing a novel communication-efficient and scalable training algorithm. We propose a hierarchical federated learning framework for such hybrid data partitioning with a multitier-partitioned neural network architecture. Specifically, we adopt a primal-dual transform to decompose the training problem on both the sample and feature space. Then, a stochastic coordinate gradient descent ascent algorithm is implemented with intratype and intertype over-the-air aggregation for the update of the primal variables and dual variables, respectively. The incorporation of over-the-air aggregation for signal transmission naturally harnesses the channel perturbations and interference for lower communication complexity and preserved privacy. Despite the influence of transmission noise and channel distortion, convergence analysis is provided for general objective functions, which illustrates the robust training performance of the proposed algorithm with a theoretical guarantee.