Federated Learning (FL) is a promising paradigm for mining massive data while respecting users' privacy. However, the deployment of FL on resource-constrained edge devices remains elusive due to its high resource demand. In this paper, unlike existing works that use expensive dense models, we propose to utilize dynamic sparse training in FL and design a novel sparse-to-sparse FL framework, named as POP-FL. The framework can reduce both computation and communication overheads while maintaining the performance of the global model. Specifically, POP-FL partitions massive clients into groups and performs parallel parameter exploration, i.e.,Parallel Over-Parameterization, over the collaboration between these groups. This exploration can greatly improve the expressibility and generalizability of sparse training in FL (especially for extreme sparsity levels) through reliably covering sufficient parameters and dynamically updating the global sparse network's structure during the training process. Experimental results show that compared with existing sparse-to-sparse training methods in both iid and non-iid data distribution, POP-FL achieves the best inference accuracy on various representative networks.

Federated Learning (FL) is a promising paradigm for massive data mining service while protecting users’ privacy. In wireless federated learning networks (WFLNs), limited communication resources and heterogeneity of user devices have essential impacts on training efficiency of FL, hence it is critical to select clients and allocate network bandwidths among them in each learning round to improve the training efficiency. In this article, we formulate the joint client selection and bandwidth allocation optimization problem as a MDP process and design a FL framework CSBWA to solve it. CSBWA relies on DRL-based REINFORCE algorithm to automatically perform effective policy based on observed information, e.g., client states, historical bandwidths, and feedback rewards. It is able to achieve lower time cost and energy consumption with long-term FL performance guarantee by jointly optimizing the client selection and bandwidth allocation. Experimental results show the effectiveness of CSBWA in reducing time cost and energy consumption while guaranteeing model performance of wireless federated learning compared with existing state-of-art methods.

In the big data era, Federated Learning (FL), which allows multiple participants to collaboratively train a global model without sharing their raw data, emerges as a promising solution to address the challenges of isolated data silos and privacy protection. Federated learning has two main communication strategies: synchronous and asynchronous. Synchronous FL ensures stable convergence but may encounter model quality degradation and server crash risks. Asynchronous FL avoids the straggler effect and supports more participants, but unstable convergence and non-IID data could affect the model performance. In this paper, inspired by real-world FL scenarios, we propose a highly efficient K-Asynchronous FL framework, KFLBSV, which addresses the limitations of synchronous and asynchronous strategies to some extent, leading to improved model performance and convergence speed. The framework allows clients to upload updates multiple times within the same round instead of blocking after each upload, thereby enhancing training efficiency. To ensure the stability and performance of the global model, we introduce a novel aggregation method. By approximating Shapley value to assess model consistency and balancing client contribution frequency and model staleness, we allocate weights more accurately to each participating client. We extensively conducted experiments on benchmark datasets using three distinct models, and the results show that KFLBSV outperforms existing algorithms in terms of both model performance and convergence speed.

One of the primary challenges in cloud-edge environments is efficiently utilizing significant amounts of data on edge devices for machine learning tasks, enabling adaptation to increasingly complex computing and service scenarios. Federated Learning (FL) is a machine learning paradigm that enables collaborative training of models involving multiple data warehouses in a privacy-preserving manner. However, classical federated learning has poor convergence on highly heterogeneous data, which limits its performance of global model on each edge device. The emergence of Personalized Federated Learning (PFL) effectively alleviates data heterogeneity, but learning a personalized model may incur greater overheads. In this paper, we propose an efficient FL framework named as ASPFL, which uses dynamic sparse training for personalized federated learning to maintain model performance while reducing computational and communication overheads in cloud-edge environments. By adaptively allocating the dynamic sparsity from a global perspective to explore sparse network structure during training, ASPFL improves the independent parameter exploration process of local sparse training to adapt to various heterogeneous situations and solves the Non-IID challenge of FL. The abundant experimental results show that ASPFL outperforms state-of-the-art methods in performance, overheads, and convergence speed in PFL.