Federated Learning (FL) is a promising paradigm for mining massive data while respecting users' privacy. However, the deployment of FL on resource-constrained edge devices remains elusive due to its high resource demand. In this paper, unlike existing works that use expensive dense models, we propose to utilize dynamic sparse training in FL and design a novel sparse-to-sparse FL framework, named as POP-FL. The framework can reduce both computation and communication overheads while maintaining the performance of the global model. Specifically, POP-FL partitions massive clients into groups and performs parallel parameter exploration, i.e.,Parallel Over-Parameterization, over the collaboration between these groups. This exploration can greatly improve the expressibility and generalizability of sparse training in FL (especially for extreme sparsity levels) through reliably covering sufficient parameters and dynamically updating the global sparse network's structure during the training process. Experimental results show that compared with existing sparse-to-sparse training methods in both iid and non-iid data distribution, POP-FL achieves the best inference accuracy on various representative networks.

Federated Learning (FL) is a promising paradigm for massive data mining service while protecting users’ privacy. In wireless federated learning networks (WFLNs), limited communication resources and heterogeneity of user devices have essential impacts on training efficiency of FL, hence it is critical to select clients and allocate network bandwidths among them in each learning round to improve the training efficiency. In this article, we formulate the joint client selection and bandwidth allocation optimization problem as a MDP process and design a FL framework CSBWA to solve it. CSBWA relies on DRL-based REINFORCE algorithm to automatically perform effective policy based on observed information, e.g., client states, historical bandwidths, and feedback rewards. It is able to achieve lower time cost and energy consumption with long-term FL performance guarantee by jointly optimizing the client selection and bandwidth allocation. Experimental results show the effectiveness of CSBWA in reducing time cost and energy consumption while guaranteeing model performance of wireless federated learning compared with existing state-of-art methods.

One of the primary challenges in cloud-edge environments is efficiently utilizing significant amounts of data on edge devices for machine learning tasks, enabling adaptation to increasingly complex computing and service scenarios. Federated Learning (FL) is a machine learning paradigm that enables collaborative training of models involving multiple data warehouses in a privacy-preserving manner. However, classical federated learning has poor convergence on highly heterogeneous data, which limits its performance of global model on each edge device. The emergence of Personalized Federated Learning (PFL) effectively alleviates data heterogeneity, but learning a personalized model may incur greater overheads. In this paper, we propose an efficient FL framework named as ASPFL, which uses dynamic sparse training for personalized federated learning to maintain model performance while reducing computational and communication overheads in cloud-edge environments. By adaptively allocating the dynamic sparsity from a global perspective to explore sparse network structure during training, ASPFL improves the independent parameter exploration process of local sparse training to adapt to various heterogeneous situations and solves the Non-IID challenge of FL. The abundant experimental results show that ASPFL outperforms state-of-the-art methods in performance, overheads, and convergence speed in PFL.