Distributed microgrids are being deployed into our power grids to form large-scale multimicrogrid systems for utilizing growing renewable energy sources. An effective energy management strategy is fundamental to balancing energy supply and demand alongside maintaining the stability of multimicrogrid. In this article, we propose a scalable, privacy-preserving, distributed energy management approach (SPDEM) for multimicrogrid. Specifically, we first formulate the energy management problem in multimicrogrid as a decentralized partially observable Markov decision process (Dec-POMDP). Next, we develop an intelligent energy management algorithm using mean-field multiagent recurrent reinforcement learning to efficiently solve the Dec-POMDP. This approach incorporates a novel fingerprint-based importance sampling technique to address the obsolete experiences induced by mean field approximation. Extensive experiments on real-world datasets demonstrate that SPDEM can make effective energy management decisions under variable renewable energy generation and load demand. Comparisons with five typical baselines illustrate the superb performance of SPDEM in cost reduction and scalability enhancement.

The growth of Electric Vehicles (EVs) places an increasingly heavy burden on the limited charging infrastructure, necessitating an effective charging station recommendation strategy that assists EVs in finding the most suitable charging stations. Deep reinforcement learning is a promising technology that has been applied to optimize EVs’ charging recommendations. However, existing schemes have low scalability and high communication costs as they usually require collecting real-time information on both charging requests and charger availability at various stations during policy training or execution. To address this challenge, we develop a real-time distributed charging station recommendation approach, named ReDirect, to minimize the charging duration experienced by EVs, considering dynamic charging requests of EVs and fluctuating availability at charging stations. ReDirect employs federated meta-reinforcement learning (RL) to empower distributed stations to collaboratively learn effective recommendation strategies and make decisions without sharing their local information, yielding improved scalability, reduced communication overhead, and enhanced data privacy. Furthermore, we conduct a rigorous theoretical analysis of the convergence performance of ReDirect. Extensive experimental results on real-world datasets demonstrate that ReDirect performs closely to the centralized recommendation algorithm and outperforms several state-of-the-art distributed algorithms in EV charging duration while realizing a balanced distribution of charging requests across multiple stations.

Electric Vehicle-assisted Multi-access Edge Computing (EV-MEC) is a promising paradigm where EVs share their computation resources at the network edge to perform intensive computing tasks while charging. In EV-MEC, a fundamental problem is to jointly decide the charging power of EVs and computation task allocation to EVs, for meeting both the diverse charging demands of EVs and stringent performance requirements of heterogeneous tasks. To address this challenge, we propose a new joint charging scheduling and computation offloading scheme (OCEAN) for EV-MEC. Specifically, we formulate a cooperative two-timescale optimization problem to minimize the charging load and its variance subject to the performance requirements of computation tasks. We then decompose this sophisticated optimization problem into two sub-problems: charging scheduling and computation offloading. For the former, we develop a novel safe deep reinforcement learning (DRL) algorithm, and theoretically prove the feasibility of learned charging scheduling policy. For the latter, we reformulate it as an integer non-linear programming problem to derive the optimal offloading decisions. Extensive experimental results demonstrate that OCEAN can achieve similar performances as the optimal strategy and realize up to 24% improvement in charging load variance over three state-of-the-art algorithms while satisfying the charging demands of all EVs.

Collaborative mobile edge computing (MEC) is a new paradigm that allows cooperative peer offloading among distributed MEC servers to balance their computing workloads. However, the highly dynamic workloads and wireless network conditions pose great challenges to achieving efficient task offloading in collaborative MEC. To address this challenge, digital twin (DT) has emerged as one promising solution by building a high-fidelity virtual mirror of the physical MEC to simulate its behaviors and help make optimal operational decisions. In this paper, we propose a DT-driven intelligent task offloading framework for collaborative MEC, where DT is employed to map the collaborative MEC system into a virtual space and optimize the task offloading decisions. We model the task offloading process as a Markov decision process (MDP) with the objective of maximizing the MEC system’s total income from providing computing services, and then develop a deep reinforcement learning (DRL)-based intelligent task offloading scheme (INTO) to jointly optimize the peer offloading and resource allocation decisions. An efficient action refinement method is proposed to ensure that the action selected by the DRL agent is feasible. Experimental results show that our proposed approach can effectively adapt the task offloading decisions according to the dynamic environment, and significantly improve the MEC system’s income through extensive comparison with three state-of-the-art algorithms.

With the development of 5G communication technologies and smart mobile devices, various computation-intensive and delay-sensitive tasks continue to increase. The combination of Mobile Edge Computing (MEC) and Ultra-Dense Networks (UDN) increases the network capacity and improves the computing capability of mobile devices, which effectively meets the transmission and computing demands of tasks. However, the ultra-dense deployment of network infrastructures causes energy shortage and channel interference, making it challenging to reduce the system cost. In this paper, we investigate the task offloading and resources scheduling problem in UDN with MEC. In order to minimize the total system cost including delay and energy consumption in the intensive deployment environment of edge servers and base stations (BSs) simultaneously, we design the strategy of task offloading, BS selection and resources scheduling of mobile devices. Because of the complex coupling of decision variables, the original problem is decomposed into two sub-problems. We propose Newton-IPM based Computing Resource Allocation (NICRA) algorithm and Genetic Algorithm based BS Selection and Resources Scheduling (GABSRS) algorithm to solve these two sub-problems, respectively. Then, we prove the number of iterations can be reduced effectively by the GABSRS algorithm while reaching the optimal solution through mathematical analysis. Through experiments analysis, the effectiveness of the GABSRS algorithm is validated.

With proliferation of computation-intensive Internet of Things (IoT) applications, the limited capacity of end devices can deteriorate service performance. To address this issue, computation tasks can be offloaded to the Mobile Edge Computing (MEC) for processing. However, it consumes considerable energy to transmit and process these tasks. In this paper, we study the energy efficient task offloading in MEC. Specifically, we formulate it as a stochastic optimization problem, with the objective of minimizing the energy consumption of task offloading while guaranteeing the average queue length. Solving this offloading optimization problem faces many technical challenges due to the uncertainty and dynamics of wireless channel state and task arrival process, and the large scale of solution space. To tackle these challenges, we apply stochastic optimization techniques to transform the original stochastic problem into a deterministic optimization problem, and propose an energy efficient dynamic offloading algorithm called EEDOA. EEDOA can be implemented in an online manner to make the task offloading decisions with polynomial time complexity. Theoretical analysis is provided to demonstrate that EEDOA can approximate the minimal transmission energy consumption while still bounding the queue length. Experiment results are presented which show the EEDOA’s effectiveness.

As an emerging computing paradigm, mobile edge computing (MEC) can improve users’ service experience by provisioning the cloud resources close to the mobile devices. With MEC, computation-intensive tasks can be processed on the MEC servers, which can greatly decrease the mobile devices’ energy consumption and prolong their battery lifetime. However, the highly dynamic task arrival and wireless channel states pose great challenges on the computation task allocation in MEC. This paper jointly investigates the task allocation and CPU-cycle frequency, to achieve the minimum energy consumption while guaranteeing that the queue length is upper bounded. We formulate it as a stochastic optimization problem, and with the aid of stochastic optimization methods, we decouple the original problem into two deterministic optimization subproblems. An online Task Offloading and Frequency Scaling for Energy Efficiency (TOFFEE) algorithm is proposed to obtain the optimal solutions of these subproblems concurrently. TOFFEE can obtain the close-to-optimal energy consumption while bounding the applications’ queue length. Performance evaluation is conducted which verifies TOFFEE’s effectiveness. Experiment results indicate that TOFFEE can decrease the energy consumption by about 15 percent compared with the RLE algorithm, and by about 38 percent compared with the RME algorithm.

Nowadays, driven by the rapid development of smart mobile equipments and 5G network technologies, the application scenarios of Internet of Things (IoT) technology are becoming increasingly widespread. The integration of IoT and industrial manufacturing systems forms the industrial IoT (IIoT). Because of the limitation of resources, such as the computation unit and battery capacity in the IIoT equipments (IIEs), computation-intensive tasks need to be executed in the mobile edge computing (MEC) server. However, the dynamics and continuity of task generation lead to a severe challenge to the management of limited resources in IIoT. In this article, we investigate the dynamic resource management problem of joint power control and computing resource allocation for MEC in IIoT. In order to minimize the long-term average delay of the tasks, the original problem is transformed into a Markov decision process (MDP). Considering the dynamics and continuity of task generation, we propose a deep reinforcement learning-based dynamic resource management (DDRM) algorithm to solve the formulated MDP problem. Our DDRM algorithm exploits the deep deterministic policy gradient and can deal with the high-dimensional continuity of the action and state spaces. Extensive simulation results demonstrate that the DDRM can reduce the long-term average delay of the tasks effectively.

Driven by Internet of Things (IoT) and 5G communication technologies, the paradigm of mobile computing has changed from centralized mobile cloud computing to distributed mobile edge computing (MEC). Narrowing the gap between high quality of service (QoS) requirements and limited computing resources, and improving the utilization of computing resources between IoT devices and edge servers have become key issues. In this paper, we formulate a stochastic optimization problem involving dynamic offloading and resource scheduling between the local devices, base station (BS) and the back-end cloud. The goal is to minimize the consumption of energy and computing resources in the MEC system with energy harvesting (EH) devices, while meeting the QoS requirements of IoT devices. In order to solve this stochastic optimization problem, we convert it into a deterministic optimization problem, and propose an online dynamic offloading and resource scheduling algorithm (DORS) based on Lyapunov optimization theory. It is proved that the DORS algorithm can effectively balance the relationship between scheduling cost and MEC system’s performance. The comparison experiments show the effectiveness of the DORS algorithm in reducing the energy consumption.

Summary Narrow Band Internet of Things (NB‐IoT) is a cellular‐based low power wide area network (LPWAN) radio technology, which can provide highly reliable services and wide coverage for IoT devices. Software defined networking (SDN) as an emerging network architecture can realize flexible resource allocation and network management. We introduce SDN into NB‐IoT and investigate the traffic modeling and performance evaluation of SDN‐based NB‐IoT access network. To evaluate the network performance in different environments, we introduce the Beta/D/1, Uniform/D/1, and M/D/1 queuing models, respectively. The proposed queuing models are suitable for different scenarios, in which NB‐IoT devices access the network in a highly synchronized, unsynchronized, or stochastic manner. We use the general solution to the G/G/1 and the M/G/1 queuing model to solve the proposed modeling problems. Through simulations, we investigate the influence of different network parameters. The analysis and simulation results can be used in the SDN controller to dynamically allocate resources and make network management decisions to satisfy different performance requirements of NB‐IoT applications.

To satisfy the increasingly complex demands of the Internet of Things (IoT) applications, multiclouds are a promising solution that can provide scalable, various, and abundant resources. However, in multiclouds, each cloud has its specific virtual machine (VM) type and pricing scheme. In addition, the request arrival, network bandwidth, and VM's price all vary with time and are hardly predicted. In such cases, the request scheduling and resource provisioning (RSRP) for cost efficiency becomes a highly challenging work. In this article, to capture the dynamics in the multiclouds environment, we formulate a stochastic optimization problem where the aim is to minimize the system cost and guarantee the IoT applications' queueing delay. By applying stochastic optimization theory, the original problem is transformed into a deterministic optimization problem in each slot, and then the deterministic problem is further decomposed into three independent subproblems. An online RSRP algorithm is devised to obtain these subproblems' optimal solutions. Mathematical analysis shows that RSRP can approach the optimal system cost while bounding the queueing delay, and make an arbitrary tradeoff between system cost and queueing delay as well. Moreover, trace-driven simulation results show the effectiveness of RRSP.

Mobile edge computing (MEC) has recently become a promising paradigm to meet the increasing computing requirement of mobile devices, and hybrid energy supply has been considered as an effective approach for saving the energy consumption of the MEC system and making it environmentally friendly. In particular, the joint task scheduling and energy management (TSEM) scheme plays a crucial role in reaping the benefits of MEC with hybrid energy supply. In this article, we focus on jointly optimizing the TSEM decisions to maximize the utility of the MEC system which accounts for both the computation throughput and the fairness among different cells, by formulating a stochastic optimization problem subject to the constraints of queue stability and energy budget. We transform the formulated problem into a deterministic problem and then decouple it into four independent subproblems, which can be solved in a distributed manner without future system statistical information. An online TSEM algorithm is developed to derive the optimal solutions to these subproblems. Mathematical analysis shows that TSEM can achieve a close-to-optimal system utility and realize the utility-queue tradeoff. The experimental results validate the advantages of TSEM in improving the system utility and stabilizing the queue length.

Limited capacities in computation and battery of Internet of things (IoT) devices are two main bottlenecks for quality of service. Emerging mobile edge computing (MEC) and radio frequency based wireless power transfer (WPT) can help alleviate the issues. Incorporating WPT into MEC, IoT devices can get sustainable energy supply by WPT, and offload computation tasks to MEC to improve the computing ability. In this paper, we jointly consider the radio resource and task allocation for the wireless powered MEC system. To capture the high dynamics in task arrival and wireless network, a stochastic optimization problem which minimizes the energy consumption while guaranteeing queue stability is formulated. By exploiting the stochastic optimization theory, we transform the original problem into a deterministic optimization problem. A radio resource and task allocation (RRTA) algorithm is designed to acquire the optimal solutions of this problem. Theoretical analysis shows that RRTA can achieve arbitrary tradeoff between the energy consumption and queue length. Moreover, the close-to-optimal energy consumption can be reached by RRTA while bounding the queueing length. Experiment results reveal that RRTA can effectively decrease the energy consumption and maintain a small queue length.

Nowadays, billions of Internet of Things (IoT) devices arise around us running complex and computation-intensive applications. Due to the limited resources of the IoT devices, it is appealing to offload the application tasks from IoT devices to the remote cloud data centers. However, offloading all the tasks to the cloud can put a significant burden on the network. One promising way to solve this issue is edge computing, where edge servers are provisioned at the network edge. In edge computing for IoT, as the task generating process is highly dynamic and the statistical information can hardly be obtained or precisely predicted, it is of great importance yet very challenging to effectively offload application tasks to achieve the tradeoff between offloading cost and performance. In this paper, we formulate the computation offloading as an optimization problem to minimize offloading cost while providing performance guarantees. Based on stochastic optimization, we propose a dynamic computation offloading algorithm (DCOA), which decomposes the optimization problem into a series of subproblems, and solves these subproblems concurrently in an online and distributed way. Theoretical analysis is presented which demonstrates that DCOA can achieve the tradeoff between offloading cost and performance. Experiments are also carried out to evaluate the effectiveness of DCOA.

Nowadays, mobile services (applications) running on terminal devices are becoming more and more computation‐intensive. Offloading the service requests from terminal devices to cloud computing can be a good solution, but it would put a high burden on the network. Edge computing is an emerging technology to solve this problem, which places servers at the edge of the network. Dynamic scheduling of offloaded service requests in mobile edge computing systems is a key issue. It faces challenges due to the dynamic nature and uncertainty of service request patterns. In this article, we propose a Dynamic Service Request Scheduling (DSRS) algorithm, which makes request scheduling decisions to optimize scheduling cost while providing performance guarantees. The DSRS algorithm can be implemented in an online and distributed way. We present mathematical analysis which shows that the DSRS algorithm can achieve arbitrary tradeoff between scheduling cost and performance. Experiments are also carried out to show the effectiveness of the DSRS algorithm.