In today’s increasingly digital landscape, continuous user authentication on smartphones has become crucial for safeguarding sensitive information. Behavioral biometrics, particularly facial recognition, is emerging as a powerful tool to enhance security, leveraging advanced machine learning and deep learning models. However, traditional approaches often involve sharing personal data for training, raising significant privacy concerns. Federated Learning (FL) addresses this issue by enabling decentralized model training directly on users’ devices, thus preserving privacy. Despite its promise, FL faces unique challenges in continuous user authentication, particularly due to the non-IID (non-Independent and Identically Distributed) nature of the data where every client has access only to one label data samples. While Convolutional Neural Networks (CNNs) are commonly employed in facial recognition, they struggle with the complexities of localized features and data distribution variance. This article explores all the possible architectures in order to tackle the CNNs weaknesses, we leverage the Vision Transformers (ViTs) and MLP-Mixers as a promising alternative to CNNs in the context of facial recognition. ViTs and MLP-Mixers excel in capturing global context and hierarchical representations, making them better suited to handle the complexities of continuous user authentication in FL. Through a case study, we demonstrate how integrating ViTs and MLP-Mixers into FL frameworks for facial recognition can enhance prediction accuracy and reduce weight divergence, offering a more robust and secure solution compared to other models.

In Federated Learning (FL), the limited accessibility of data from diverse locations and user types poses a significant challenge due to restricted user participation. Expanding client access and diversifying data enhance models by incorporating diverse perspectives, thereby improving adaptability. However, in dynamic and mobile environments, the availability of FL clients fluctuates as devices may become inaccessible, leading to inefficient client selection and reduced model performance. Current solutions often fail to adapt quickly to these changes, creating a gap in achieving real-time client availability and efficient data utilization. To address this, we propose a Deep Reinforcement Learning (DRL) On-Demand solution, deploying new clients using Docker Containers on-the-fly. Our On-Demand solution, employing DRL, targets client availability and selection while considering data shifts and container deployment complexities. It employs an autonomous end-to-end approach for handling model deployment and client selection. The DRL strategy leverages a Markov Decision Process (MDP) framework, with a Master Learner and a Joiner Learner to optimize decision-making. The designed cost functions account for the complexity of dynamic client deployment and selection, ensuring effective resource management and service reliability. Simulated tests show that our architecture can easily adapt to changes in the environment and respond to On-Demand requests while reducing the number of learning rounds used by 20-50 % compared with existing approaches. This highlights its ability to improve client availability, capability, accuracy, and learning efficiency, surpassing heuristic and traditional reinforcement learning methods.

In edge computing environments, efficient computation resource management is crucial for optimizing service allocation to hosts in the form of containers. These environments experience dynamic user demands and high mobility, making traditional static and heuristic-based methods inadequate for handling such complexity and variability. Deep Reinforcement Learning (DRL) offers a more adaptable solution, capable of responding to these dynamic conditions. However, existing DRL methods face challenges such as high reward variability, slow convergence, and difficulties in incorporating user mobility and rapidly changing environmental configurations. To overcome these challenges, we propose a novel DRL framework for computation resource optimization at the edge layer. This framework leverages a customized Markov Decision Process (MDP) and Proximal Policy Optimization (PPO), integrating a Graph Convolutional Transformer (GCT). By combining Graph Convolutional Networks (GCN) with Transformer encoders, the GCT introduces a spatio-temporal reward-shaping mechanism that enhances the agent's ability to select hosts and assign services efficiently in real time while minimizing the overload. Our approach significantly enhances the speed and accuracy of resource allocation, achieving, on average across two datasets, a 30% reduction in convergence time, a 25% increase in total accumulated rewards, and a 35% improvement in service allocation efficiency compared to standard DRL methods and existing reward-shaping techniques. Our method was validated using two real-world datasets, MOBILE DATA CHALLENGE (MDC) and Shanghai Telecom, and was compared against standard DRL models, reward-shaping baselines, and heuristic methods. • Proposing a DRL framework that integrates reward shaping for resource management. • Introducing a novel MDP design that considers the dynamic nature of the users. • Presenting a novel reward shaping mechanism, incorporating GCN and transformers.

In this paper, we increase the availability and integration of devices and models together in the learning process to enhance the convergence of federated learning (FL) models. The majority of the literature suggested client selection techniques to accelerate convergence and boost accuracy. However, none of the existing proposals have focused on the flexibility to deploy and select clients as needed, wherever and whenever that may be while serving multiple FL models. Due to the extremely dynamic surroundings, some devices are actually not available to serve as clients in FL, which affects the availability of data for learning and the applicability of the existing solution for client selection. In this paper, we address the aforementioned limitations by introducing an On-Demand-FL, a client deployment approach for FL, offering more volume and heterogeneity of data in the learning process while supporting multiple models. We make use of the containerization technology such as Docker to build efficient environments using IoT and mobile devices serving as volunteers. Furthermore, Kubernetes is used for orchestration. The performed experiments using the Mobile Data Challenge (MDC), MNIST, KDD datasets, and the Localfed framework illustrate the relevance of the proposed approach and the efficiency of the on-the-fly deployment of clients with less discarded rounds and more available data of each running FL application.

In this article, we increase the availability and integration of devices in the learning process to enhance the convergence of federated learning (FL) models. To address the issue of having all the data in one location, FL, which maintains the ability to learn over decentralized data sets, combines privacy and technology. Until the model converges, the server combines the updated weights obtained from each data set over a number of rounds. The majority of the literature suggested client selection techniques to accelerate convergence and boost accuracy. However, none of the existing proposals have focused on the flexibility to deploy and select clients as needed, wherever and whenever that may be. Due to the extremely dynamic surroundings, some devices are actually not available to serve as clients in FL, which affects the availability of data for learning and the applicability of the existing solution for client selection. In this article, we address the aforementioned limitations by introducing an On-Demand-FL, a client deployment approach for FL, offering more volume and heterogeneity of data in the learning process. We make use of the containerization technology, such as Docker, to build efficient environments using Internet of Things and mobile devices serving as volunteers. Furthermore, Kubernetes is used for orchestration. A multiobjective optimization problem representing the client and model deployment is solved using the genetic algorithm (GA) due to its evolutionary strategy. The performed experiments using the mobile data challenge (MDC) data set and the Localfed framework illustrate the relevance of the proposed approach and the efficiency of the on-the-fly deployment of clients whenever and wherever needed with less discarded rounds and more available data.

Cloud computing is a reliable solution to provide distributed computation power. However, real-time response is still challenging regarding the enormous amount of data generated by the IoT devices in 5G and 6G networks. Thus, multiaccess edge computing (MEC), which consists of distributing the edge servers in the proximity of end users to have low latency besides the higher processing power, is increasingly becoming a vital factor for the success of modern applications. This article addresses the problem of minimizing both, the network delay, which is the main objective of MEC, and the number of edge servers to provide a MEC design with minimum cost. This MEC design consists of edge servers placement and base stations allocation, which makes it a joint combinatorial optimization problem (COP). Recently, reinforcement learning (RL) has shown promising results for COPs. However, modeling real-world problems using RL when the state and action spaces are large still needs investigation. We propose a novel RL framework with an efficient representation and modeling of the state space, action space, and the penalty function in the design of the underlying Markov decision process (MDP) for solving our problem. This modeling makes the temporal difference (TD) learning applicable for a large-scale real-world problem while minimizing the cost of network design. We introduce the TD$(\lambda)$with eligibility traces for minimizing the cost (TDMC) algorithm, in addition to$Q$-learning for the same problem (QMC) when$\lambda =0$. Furthermore, we discuss the impact of state representation, action space, and penalty function on the convergence of each model. Extensive experiments using real-world data sets from Shanghai Telecommunication and Citywide Public Computer Centers demonstrate that in the light of an efficient model, TDMC/QMC are able to find the actions that are the source of lower delayed penalty. The reported results show that our algorithm outperforms the other benchmarks by creating a tradeoff among multiple objectives.

In this paper, we propose Value Iteration Network for Reward Shaping (VIN-RS), a potential-based reward shaping mechanism using Convolutional Neural Network (CNN). The proposed VIN-RS embeds a CNN trained on computed labels using the message passing mechanism of the Hidden Markov Model. The CNN processes images or graphs of the environment to predict the shaping values. Recent work on reward shaping still has limitations towards training on a representation of the Markov Decision Process (MDP) and building an estimate of the transition matrix . The advantage of VIN-RS is to construct an effective potential function from an estimated MDP while automatically inferring the environment transition matrix. The proposed VIN-RS estimates the transition matrix through a self-learned convolution filter while extracting environment details from the input frames or sampled graphs. Due to (1) the previous success of using message passing for reward shaping; and (2) the CNN planning behavior, we use these messages to train the CNN of VIN-RS. Experiments are performed on tabular games, Atari 2600 and MuJoCo, for discrete and continuous action space. Our results illustrate promising improvements in the learning speed and maximum cumulative reward compared to the state-of-the-art. The improvement achieved by VIN-RS can only be observed for some of the games due to the underlying nature of some environments. In terms of the studied MuJoCo games, there is on average an increase of 30% in the maximum reward reached during early stages of learning.

Due to the current improvement in self-driving cars and the extensive focus and research on the topic of the Internet of Vehicles (IoV), the near future may behold a great revolution in the automotive industry as cars become fully autonomous. This change entails a considerable amount of data to be transferred from Internet of Things (IoT) devices, such as radars, sensors, and actuators. Consequently, overwhelming the existing infrastructure, namely cloud, and Road Side Units (RSU), reduces the quality of service (QoS) experienced by vehicular users. Accordingly, this paper contributes in proposing a new architecture for using Unmanned Ariel Vehicles (UAVs) and On-Boarding Units (OBUs) working in collaboration to achieve a significantly improved QoS. The proposed framework offers an end-to-end solution for master election, cluster management and recovery, vehicle selection, service placement, and accurate localization of vehicles. A QoS improvement is possible through an efficient cluster formation and placement solution that assigns lightweight services, as containers, to OBUs and UAVs while meeting various objectives. The efficiency of the proposed scheme originates from the use of the evolutionary Memetic Algorithm that 1) respects the mobility and energy constraints of UAVs and OBUs, 2) meets the user demands, and 3) uses machine learning for the accurate localization of vehicles. Our experiments using the Mininet-WiFi and SUMO simulators show at least 30% improvement in terms of QoS compared to a state-of-the-art solution.

The increasing number of Internet of Things (IoT) devices necessitates the need for a more substantial fog computing infrastructure to support the users’ demand for services. In this context, the placement problem consists of selecting fog resources and mapping services to these resources. This problem is particularly challenging due to the dynamic changes in both users’ demand and available fog resources. Existing solutions utilize on-demand fog formation and periodic container placement using heuristics due to the NP-hardness of the problem. Unfortunately, constant updates of services are time consuming in terms of environment setup, especially when required services and available fog nodes are changing. Therefore, due to the need for fast and proactive service updates to meet users’ demand, and the complexity of the container placement problem, we propose in this article a Deep Reinforcement Learning (DRL) solution, named Intelligent Fog and Service Placement (IFSP), to perform instantaneous placement decisions proactively. By proactively, we mean making placement decisions before demands occur. The DRL-based IFSP is developed through a scalable Markov Decision Process (MDP) design. To address the long learning time for DRL to converge, and the high volume of errors needed to explore, we also propose a novel end-to-end architecture utilizing a service scheduler and a bootstrapper. on the cloud. Our scheduler and bootstrapper perform offline learning on users’ demand recorded in server logs. Through experiments and simulations performed on the NASA server logs and Google Cluster Trace datasets, we explore the ability of IFSP to perform efficient placement and overcome the above mentioned DRL limitations. We also show the ability of IFSP to adapt to changes in the environment and improve the Quality of Service (QoS) compared to state-of-the-art-heuristic and DRL solutions.

Currently, researchers have motivated a vision of 6G for empowering the new generation of the Internet of Everything (IoE) services that are not supported by 5G. In the context of 6G, more computing resources are required, a problem that is dealt with by Mobile Edge Computing (MEC). However, due to the dynamic change of service demands from various locations, the limitation of available computing resources of MEC, and the increase in the number and complexity of IoE services, intelligent resource provisioning for multiple applications is vital. To address this challenging issue, we propose in this paper IScaler, a novel intelligent and proactive IoE resource scaling and service placement solution. IScaler is tailored for MEC and benefits from the new advancements in Deep Reinforcement Learning (DRL). Multiple requirements are considered in the design of IScaler's Markov Decision Process. These requirements include the prediction of the resource usage of scaled applications, the prediction of available resources by hosting servers, performing combined horizontal and vertical scaling, as well as making service placement decisions. The use of DRL to solve this problem raises several challenges that prevent the realization of IScaler's full potential, including exploration errors and long learning time. These challenges are tackled by proposing an architecture that embeds an Intelligent Scaling and Placement module (ISP). ISP utilizes IScaler and an optimizer based on heuristics as a bootstrapper and backup. Finally, we use the Google Cluster Usage Trace dataset to perform real-life simulations and illustrate the effectiveness of IScaler's multi-application autonomous resource provisioning.

Several studies leverage fog computing as a solution to overcome cloud delays, including computation, network, and data storage. Along with the increase in demands for computing resources in fog infrastructures, heterogeneous fog devices are used towards forming highly available clusters. Existing approaches support the use of heterogeneous fogs and enable dynamic updates and management of services through containerization and orchestration technologies. However, none of the existing works proposed a proactive solution to horizontally scale these resources based on the IoT workload fluctuations, in addition to deciding on proper placement of the scaled instances on fogs with minimal cost on the fly. An effective scaling results in improving the response time and avoid service instability on fog devices. Therefore, we propose in this work FScaler, a reinforcement learning agent that horizontally scales container's instances after studying user's demands, and schedules the placement of newly created instances based on defined cost functions after studying the change in resources availability. The environment of FScaler is modeled as an MDP to be solved by any RL algorithm. For this work, we study the efficiency of our MDP formulation by solving the problem using SARSA. Promising results are shown through testing using a real-life dataset presenting the variation of user's demands of a particular service and the change in resource availability over time.

With the increasing number of IoT devices, fog computing has emerged, providing processing resources at the edge for the tremendous amount of sensed data and IoT computation. The advantage of the fog gets eliminated if it is not present near IoT devices. Fogs nowadays are pre-configured in specific locations with pre-defined services, which limit their diverse availabilities and dynamic service update. In this paper, we address the aforementioned problem by benefiting from the containerization and micro-service technologies to build our on-demand fog framework with the help of the volunteering devices. Our approach overcomes the current limitations by providing available fog devices with the ability to have services deployed on the fly. Volunteering devices form a resource capacity for building the fog computing infrastructure. Moreover, our framework leverages intelligent container placement scheme that produces efficient volunteers' selection and distribution of services. An Evolutionary Memetic Algorithm (MA) is elaborated to solve our multi-objective container placement optimization problem. Real life and simulated experiments demonstrate various improvements over existing approaches interpreted by the relevance and efficiency of (1) forming volunteering fog devices near users with maximum time availability and shortest distance, and (2) deploying services on the fly on selected fogs with improved QoS.

Observing the headway in vehicular industry, new applications are developed demanding more resources. For instance, real-time vehicular applications require fast processing of the vast amount of generated data by vehicles in order to maintain service availability and reachability while driving. Fog devices are capable of bringing cloud intelligence near the edge, making them a suitable candidate to process vehicular requests. However, their location, processing power, and technology used to host and update services affect their availability and performance while considering the mobility patterns of vehicles. In this paper, we overcome the aforementioned limitations by taking advantage of the evolvement of On-Board Units, Kubeadm Clustering, Docker Containerization, and micro-services technologies. In this context, we propose an efficient resource and context aware approach for deploying containerized micro-services on on-demand fogs called Vehicular-OBUs-As-On-Demand-Fogs. Our proposed scheme embeds (1) a Kubeadm based approach for clustering OBUs and enabling on-demand micro-services deployment with the least costs and time using Docker containerization technology, (2) a hybrid multi-layered networking architecture to maintain reachability between the requesting user and available vehicular fog cluster, and (3) a vehicular multi-objective container placement model for producing efficient vehicles selection and services distribution. An Evolutionary Memetic Algorithm is elaborated to solve our vehicular container placement problem. Experiments and simulations demonstrate the relevance and efficiency of our approach compared to other recent techniques in the literature.

The integration of sensors and units in vehicle manufacturing is constantly increasing the data volume generated by vehicles. This requires to support services to handle these data and provide a continuous response to application requests such as collision warnings, lane changing, traffic information, routing information, multimedia streaming, and many others. Add to that the need for achieving the required quality of service is of immense importance. Fog devices deployed along the road are used as a solution to service vehicles. Vehicular ad hoc network (VANET) environment have the property of dynamically changing its cluster connectivity because of the different mobility patterns, which makes it a challenge for vehicular users to access their services. Many preliminary works proposed the use of Software Defined Wireless Network (SDWN) in VANET. In this paper, we study and analyze the use of SDWN in vehicular fog computing environment taking into consideration the suspected high delay between the SDWN controller that is located on the cloud and its switches, the high traffic coming to the controller, the high vehicle speed, and the range of RSU coverage. We have used Mininet-Wifi to implement a vehicular fog computing architecture and simulate different scenarios. Obtained results showed that using SDWN in a VANET environment might not be the best solution in many cases.