In Federated Learning (FL), the limited accessibility of data from diverse locations and user types poses a significant challenge due to restricted user participation. Expanding client access and diversifying data enhance models by incorporating diverse perspectives, thereby improving adaptability. However, in dynamic and mobile environments, the availability of FL clients fluctuates as devices may become inaccessible, leading to inefficient client selection and reduced model performance. Current solutions often fail to adapt quickly to these changes, creating a gap in achieving real-time client availability and efficient data utilization. To address this, we propose a Deep Reinforcement Learning (DRL) On-Demand solution, deploying new clients using Docker Containers on-the-fly. Our On-Demand solution, employing DRL, targets client availability and selection while considering data shifts and container deployment complexities. It employs an autonomous end-to-end approach for handling model deployment and client selection. The DRL strategy leverages a Markov Decision Process (MDP) framework, with a Master Learner and a Joiner Learner to optimize decision-making. The designed cost functions account for the complexity of dynamic client deployment and selection, ensuring effective resource management and service reliability. Simulated tests show that our architecture can easily adapt to changes in the environment and respond to On-Demand requests while reducing the number of learning rounds used by 20-50 % compared with existing approaches. This highlights its ability to improve client availability, capability, accuracy, and learning efficiency, surpassing heuristic and traditional reinforcement learning methods.

In edge computing environments, efficient computation resource management is crucial for optimizing service allocation to hosts in the form of containers. These environments experience dynamic user demands and high mobility, making traditional static and heuristic-based methods inadequate for handling such complexity and variability. Deep Reinforcement Learning (DRL) offers a more adaptable solution, capable of responding to these dynamic conditions. However, existing DRL methods face challenges such as high reward variability, slow convergence, and difficulties in incorporating user mobility and rapidly changing environmental configurations. To overcome these challenges, we propose a novel DRL framework for computation resource optimization at the edge layer. This framework leverages a customized Markov Decision Process (MDP) and Proximal Policy Optimization (PPO), integrating a Graph Convolutional Transformer (GCT). By combining Graph Convolutional Networks (GCN) with Transformer encoders, the GCT introduces a spatio-temporal reward-shaping mechanism that enhances the agent's ability to select hosts and assign services efficiently in real time while minimizing the overload. Our approach significantly enhances the speed and accuracy of resource allocation, achieving, on average across two datasets, a 30% reduction in convergence time, a 25% increase in total accumulated rewards, and a 35% improvement in service allocation efficiency compared to standard DRL methods and existing reward-shaping techniques. Our method was validated using two real-world datasets, MOBILE DATA CHALLENGE (MDC) and Shanghai Telecom, and was compared against standard DRL models, reward-shaping baselines, and heuristic methods. • Proposing a DRL framework that integrates reward shaping for resource management. • Introducing a novel MDP design that considers the dynamic nature of the users. • Presenting a novel reward shaping mechanism, incorporating GCN and transformers.

In this paper, we increase the availability and integration of devices and models together in the learning process to enhance the convergence of federated learning (FL) models. The majority of the literature suggested client selection techniques to accelerate convergence and boost accuracy. However, none of the existing proposals have focused on the flexibility to deploy and select clients as needed, wherever and whenever that may be while serving multiple FL models. Due to the extremely dynamic surroundings, some devices are actually not available to serve as clients in FL, which affects the availability of data for learning and the applicability of the existing solution for client selection. In this paper, we address the aforementioned limitations by introducing an On-Demand-FL, a client deployment approach for FL, offering more volume and heterogeneity of data in the learning process while supporting multiple models. We make use of the containerization technology such as Docker to build efficient environments using IoT and mobile devices serving as volunteers. Furthermore, Kubernetes is used for orchestration. The performed experiments using the Mobile Data Challenge (MDC), MNIST, KDD datasets, and the Localfed framework illustrate the relevance of the proposed approach and the efficiency of the on-the-fly deployment of clients with less discarded rounds and more available data of each running FL application.

In this article, we increase the availability and integration of devices in the learning process to enhance the convergence of federated learning (FL) models. To address the issue of having all the data in one location, FL, which maintains the ability to learn over decentralized data sets, combines privacy and technology. Until the model converges, the server combines the updated weights obtained from each data set over a number of rounds. The majority of the literature suggested client selection techniques to accelerate convergence and boost accuracy. However, none of the existing proposals have focused on the flexibility to deploy and select clients as needed, wherever and whenever that may be. Due to the extremely dynamic surroundings, some devices are actually not available to serve as clients in FL, which affects the availability of data for learning and the applicability of the existing solution for client selection. In this article, we address the aforementioned limitations by introducing an On-Demand-FL, a client deployment approach for FL, offering more volume and heterogeneity of data in the learning process. We make use of the containerization technology, such as Docker, to build efficient environments using Internet of Things and mobile devices serving as volunteers. Furthermore, Kubernetes is used for orchestration. A multiobjective optimization problem representing the client and model deployment is solved using the genetic algorithm (GA) due to its evolutionary strategy. The performed experiments using the mobile data challenge (MDC) data set and the Localfed framework illustrate the relevance of the proposed approach and the efficiency of the on-the-fly deployment of clients whenever and wherever needed with less discarded rounds and more available data.