In the domain of computer architecture, Federated Learning (FL) is a paradigm of distributed machine learning in edge systems. However, the systems’ Non-Independent and Identically Distributed (Non-IID) data negatively affect the convergence efficiency of the global model, since only a subset of these data samples is beneficial for accelerating model convergence. In pursuit of this subset, a reliable approach involves determining a measure of validity to rank the samples within the dataset. In this paper, we propose the BHerd strategy, which selects a beneficial herd of local gradients to accelerate the convergence of the FL model. Specifically, we map the distribution of the local dataset to the local gradients and use the Herding strategy to obtain a permutation of the set of gradients, where the more advanced gradients in the permutation are closer to the average of the set of gradients. These top portions of the gradients will be selected and sent to the server for global aggregation. We conduct experiments on different datasets, models, and scenarios by building a prototype system, and experimental results demonstrate that our BHerd strategy is effective in selecting beneficial local gradients to mitigate the effects brought by the Non-IID dataset.

Federated Learning (FL) is a distributed machine learning framework in parallel and distributed systems. However, the systems’ Non-Independent and Identically Distributed (Non-IID) data negatively affect the communication efficiency, since clients with different datasets may cause significant gaps to the local gradients in each communication round. In this article, we propose a Federated Vectorized Averaging (FedVeca) method to optimize the FL communication system on Non-IID data. Specifically, we set a novel objective for the global model which is related to the local gradients. The local gradient is defined as a bi-directional vector with step size and direction, where the step size is the number of local updates and the direction is divided into positive and negative according to our definition. In FedVeca, the direction is influenced by the step size, thus we average the bi-directional vectors to reduce the effect of different step sizes. Then, we theoretically analyze the relationship between the step sizes and the global objective, and obtain upper bounds on the step sizes per communication round. Based on the upper bounds, we design an algorithm for the server and the client to adaptively adjusts the step sizes that make the objective close to the optimum. Finally, we conduct experiments on different datasets, models and scenarios by building a prototype system, and the experimental results demonstrate the effectiveness and efficiency of the FedVeca method.

With the rapid growth of the coronavirus disease of 2019 (COVID-19) cases, massive amounts of relevant data are being trained on machine learning models for countering communicable infectious diseases. Federated Learning (FL) is a paradigm of distributed machine learning to deal with the individual COVID-19 data, and enable the protection of data privacy. However, FL has low efficiency in Edge-Based wireless communication systems with system heterogeneity. In this paper, we propose an “Asynchronous-Adaptive FL” (AAFL) scheme. Specifically, we allow that medical devices with different performances have a heterogeneous number of local SGD iterations in each communication round, called asynchronous iteration strategy which is balanced under adaptive control. We theoretically analyze the convergence of the AAFL scheme under a given time budget and obtain a mathematical relationship between the heterogeneous number of local SGD iterations and the optimal model parameters. Based on the mathematical relationship, we design an algorithm for parameter server and work nodes to adaptively control the heterogeneous number of local SGD iterations. Subsequently, we build a prototype heterogeneous system and conduct experiments on various scenarios for analyzing the general properties of our algorithm, and then apply our algorithm to public COVID-19 databases. The experimental results and application performance demonstrate the effectiveness and efficiency of our AAFL scheme.