The growing demands of the Internet of Things (IoT) require a platform that supports real-time interactions and high availability of services to devices. In this context, the fog computing paradigm has emerged as an attractive solution for processing the data of IoT applications. Owing to the unpredictable traffic demands and resource heterogeneity in the fog environment, a smart workload distribution is essential to achieve high resource utilization and computing efficiency. To this end, this paper considers a joint link and server load balancing problem with multiple cooperative access points, (APs), in a combined edge-fog-cloud environment. The joint optimization problem is formulated as a stochastic game, and an actor-critic reinforcement learning framework, called FLoadNet, is proposed to optimize the joint policy of the multi-agents. FLoadNet consists of a centralized critic network, with parameter sharing and distributed individual actor networks in all the APs. Due to the learning dynamics and partially observable environment, we propose an extended critic network model, where cooperative APs learn to communicate among themselves while evaluating the value function. Unlike previous studies, the proposed critic network is designed to train both value and message functions, which is shown to significantly reduce the computational cost. The main goal of this work is to advance the development of efficient edge learning and the application of distributed learning algorithms specifically to fog network load balancing. The experimental results show that FLoadNet outperforms baseline load balancing methods.

Smart industries enabling automation and data exchange in manufacturing technologies demanding real-time processing, nearby storage, and reliability, all of which can be satisfied by the fog computing architecture. With the emergence of smart devices coupled with a diverse range of application requirements, it is essential to have an intelligent fog network where intelligence is spread across all network segments, taking network nodes self-aware and self-decision making. In fog networks, an optimal distribution decision faces challenges due to uncertainties associated with user workload and available resources at the fog nodes and also the wide range of node’s computing power. Given this challenge, a computational offloading and CPU resource scheduling method for minimizing energy consumption is proposed. To investigate the characteristics for offloading and optimizing their allocation, we consider two types of tasks, namely, offloadable and nonoffloadable tasks. The independent fog nodes adopt the same strategy without prior knowledge of the dynamic statistics and global observations, aiming to maximize a common goal with cooperative behaviors. Then, the deep recurrent$Q$-network (DRQN) is applied to deal with the partial-observability from limited information. The proposed DRQN-based method requires comparatively less computational complexity than the conventional$Q$-learning algorithm. The simulation results show that the proposed method can effectively deal with both transmission and CPU energy consumptions while guaranteeing convergence in a limited time.

As wireless services and applications become more sophisticated and require faster and higher capacity networks, there is a need for an efficient management of the execution of increasingly complex tasks based on the requirements of each application. In this regard, fog computing enables the integration of virtualized servers into networks and brings cloud services closer to end devices. In contrast to the cloud server, the computing capacity of fog nodes is limited and thus a single fog node might not be capable of computing-intensive tasks. In this context, task offloading can be particularly useful at the fog nodes by selecting the suitable nodes and proper resource management while guaranteeing the Quality-of-Service (QoS) requirements of the users. This article studies the design of a joint task offloading and resource allocation control for heterogeneous service tasks in multifog nodes systems. This problem is formulated as a partially observable stochastic game, in which each fog node cooperates to maximize the aggregated local rewards while the nodes only have access to local observations. To deal with partial observability, we apply a deep recurrent Q-network (DRQN) approach to approximate the optimal value functions. The solution is then compared to a deep Q-network (DQN) and deep convolutional Q-network (DCQN) approach to evaluate the performance of different neural networks. Moreover, to guarantee the convergence and accuracy of the neural network, an adjusted exploration-exploitation method is adopted. Provided numerical results show that the proposed algorithm can achieve a higher average success rate and lower average overflow than baseline methods.

The fifth-generation wireless technology (5G) has been developed with an aim to provide ubiquitous and scalable connectivity for Internet-of-Things (IoT) nodes. Likewise, the cloud radio access network (C-RAN) architecture can be exploited to enable efficient network access to IoT nodes. Nevertheless, the 5G C-RAN architecture is based on large data centers geographically located far apart, which introduces an inevitable overhead. Therefore, to supply real-time data services near by the data terminals, fog computing emerges as a promising solution. However, constrained physical fog resources and delay-sensitive services hinder the application of new virtualization technologies in the baseband unit (BBU) task allocation management of the fog network. To tackle these challenges, a task allocation framework for hierarchical software-defined fog virtual radio access networks (v-RANs) is proposed in this article. Precisely, we apply an enhanced ant colony optimization (ACO) in combination with a max-min algorithm to efficiently determine the optimal path for BBU task allocation management, while minimizing the transmission time for parallel task execution scheduling. Experimental results demonstrate that the queue delay in our approach is 98.38% and 98.82% lower than the round-robin (RR) algorithm and least connection technique (LCT), respectively.

The powerful paradigm of Fog computing is currently receiving major interest, as it provides the possibility to integrate virtualized servers into networks and brings cloud service closer to end devices. To support this distributed intelligent platform, Software-Defined Network (SDN) has emerged as a viable network technology in the Fog computing environment. However, uncertainties related to task demands and the different computing capacities of Fog nodes, inquire an effective load balancing algorithm. In this paper, the load balancing problem has been addressed under the constraint of achieving the minimum latency in Fog networks. To handle this problem, a reinforcement learning based decision-making process has been proposed to find the optimal offloading decision with unknown reward and transition functions. The proposed process allows Fog nodes to offload an optimal number of tasks among incoming tasks by selecting an available neighboring Fog node under their respective resource capabilities with the aim to minimize the processing time and the overall overloading probability. Compared with the traditional approaches, the proposed scheme not only simplifies the algorithmic framework without imposing any specific assumption on the network model but also guarantees convergence in polynomial time. The results show that, during average delays, the proposed reinforcement learning-based offloading method achieves significant performance improvements over the variation of service rate and traffic arrival rate. The proposed algorithm achieves 1.17%, 1.02%, and 3.21% lower overload probability relative to random, least-queue and nearest offloading selection schemes, respectively.

Coping with the limited amount of available spectrum, time division duplexing (TDD) system is considered as an attractive duplexing method due to exploiting channel reciprocity as well as flexible resource management. The conventional scheduling scheme is based on the channel quality indicator (CQI) reported from the user equipment (UE) to estimate instantaneous data rates for the scheduling metric calculation. However, CQI is insufficient to reflect the state of the channel variation in terms of frequency and time. Based on the channel reciprocity of TDD systems, we utilize uplink sounding reference signal (SRS) to estimate downlink channel status. However the received SRS power is a result of uplink power control where power control effect should be compensated to estimate channel status in downlink scheduling. In order to solve this problem, we propose the SRS path loss estimation method based on the power headroom report. By using this scheme, the base station (BS) can obtain the compensated signal-to-interference-plus-noise-ratio (SINR) and determine the scheduling metric based on its calculated SINR instead of reported CQI from UE. Simulation results show that the proposed scheduling algorithm outperforms the conventional scheme in total throughput, whereas the fairness index experienced by the proposed algorithm is lesser than of the conventional scheme based on CQI.