Mobile edge computing (MEC), along with device-to-device (D2D) assisted MEC (D-MEC), are promising technologies that could improve the quality-of-experience for mobile devices (MDs) by offloading their tasks to edge servers or nearby idle MDs. There is a popular trend to develop distributed task offloading algorithms using multi-agent reinforcement learning (MARL), whose adoption of central critics during training makes the offloading still size-sensitive. Therefore, this paper proposes a Size-Insensitive Task Offloading (SITOff) algorithm for D-MEC based on fully-distributed offloading without maintaining any central venue. Specifically, taking advantage of the inherent graph-like structure of D-MEC, SITOff adopts graphs to represent MDs’ states and relationships and form each MD's local knowledge about D-MEC through graph computation. Furthermore, considering the limitation of local knowledge in performing whole performance-oriented offloading, each MD utilizes D2D-transmitting to exchange knowledge with its neighbors and form a comprehensive knowledge about D-MEC to enhance the coordination of distributed offloading. Additionally, regarding the different impacts of neighbors’ knowledge, each MD leverages attention mechanisms to selectively learn its neighbors’ knowledge during knowledge-exchange. Extensive experimental results show the superiority of SITOff over state-of-the-art MARL-based offloading algorithms in D-MEC with various MDs, and the easy collaboration of SITOff with curriculum-learning for large-scale D-MEC offloading.

Computation offloading is one of the key issues in mobile edge computing (MEC) that alleviates the tension between user equipment's limited capabilities and mobile application's high requirements. To achieve model-free computation offloading when reliable MEC dynamics are unavailable, deep reinforcement learning (DRL) has become a popular methodology. However, most existing DRL-based offloading approaches are developed for a single MEC environment, with invariant system bandwidth, edge capability, task types, etc., while realistic MEC scenarios tend to be of high diversity. Unfortunately, in multi-MEC environments, DRL-based offloading faces at least two challenges, learning inefficiency and interference of offloading experiences. To address the challenges, we propose a DRL-based Multi-environmental Module-compositional Modelfree computation OFFloading (M3OFF) framework. M3OFF generates offloading policies using module composition instead of a single DRL network so that learning efficiency could be improved by reusing the same modules and learning interference could be reduced by composing different modules. Furthermore, we design multiple module composition-specific training methods for M3OFF, including alternate modules-and-composer updates to improve training stability, loss-regularization to avoid module degeneration, and module-dropout to mitigate overfitting. Extensive experimental results on both simulation and testbed demonstrate that M3OFF outperforms the performances of most state-of-the-arts in multi-MEC and reaches close to single-MEC.

One of the key issues in mobile edge computing (MEC) is computation offloading, most policies of which are developed based on mathematical programming (MP). Due to the high computational complexity of iterative programming in MP-based policies, recent years have seen a popular trend to develop offloading policies based on deep reinforcement learning (DRL). However, on account of the poor generalization ability of DRL models in MEC environments with different network sizes and settings, it is difficult to directly apply DRL-based offloading policies in unseen MEC environments. Motivated by this, we propose a DRL-based environment-adaptive offloading framework (DEAT), including a size-adaptive scheme (SIED) and setting-adaptive component (SEAL). SIED leverages the idea of ‘time division multiplexing’ to adapt to varying MEC network sizes and order-unaware feature extraction to mitigate impacts of different size-changing orders. SEAL adopts system dynamics embedding and offloading policy embedding, which guide the finding of the closest pre-training MEC environment and offloading policy, respectively, to achieve fast setting-adaptation with only few exploring interactions in unseen MEC environments. Extensive experiments are conducted via both simulation and testbed to demonstrate the adaptation performance advantages of DEAT in unseen MEC environments compared to the state-of-the-art offloading approaches.

Mobile edge computing (MEC) has been proposed as a promising paradigm to provide mobile devices with both satisfactory computing capacity and task latency. One key issue in MEC is computation offloading (CompOff), which has attracted numerous research interests. Most existing CompOff approaches are developed based on iterative programming (IterProg), that calculates a CompOff action based on system dynamics each time mobile tasks arrive. Due to the heavy dependency of IterProg on reliable system dynamics, as well as the online computational burden, recent years have seen a popular trend to develop CompOff approaches based on deep reinforcement learning (DRL), which could generate real-time model-free CompOff actions. However, due to the intrinsic poor generalization of DRL, it is hard to directly apply DRL-based policies in new MEC environments, and long-time fine-tuning is often required. To address the challenge, this paper proposes a fast transferable CompOff framework (named TransOff), based on the idea of embedded reinforcement learning. Specifically, TransOff is composed of multiple primitive CompOff policies (pCOPs) and a multiplicative composition function (MCF). The pCOPs and MCF are pre-trained in a diverse variety of MEC environments. When encountering new MEC environments, pCOPs are kept fixed to prevent catastrophic forgetting of pre-trained CompOff skills, while only MCF is fine-tuned to produce new compositions of pCOPs to achieve fast transfer. We conduct extensive experiments via both numerical simulation and real testbed, indicating the fast transfer ability of TransOff compared to the state-of-the-art DRL-based and meta learning-based CompOff approaches.

Due to the high maneuverability and flexibility, unmanned aerial vehicles (UAVs) have been considered as a promising paradigm to assist mobile edge computing (MEC) in many scenarios including disaster rescue and field operation. Most existing research focuses on the study of trajectory and computation-offloading scheduling for UAV-assisted MEC in stationary environments, and could face challenges in dynamic environments where the locations of UAVs and mobile devices (MDs) vary significantly. Some latest research attempts to develop scheduling policies for dynamic environments by means of reinforcement learning (RL). However, as these need to explore in high-dimensional state and action space, they may fail to cover in large-scale networks where multiple UAVs serve numerous MDs. To address this challenge, we leverage the idea of “divide-and-conquer” and propose HT3O, a scalable scheduling approach for large-scale UAV-assisted MEC. First, HT3O is built with neural networks via deep RL to obtain real-time scheduling policies for MEC in dynamic environments. More importantly, to make HT3O more scalable, we decompose the scheduling problem into two-layered subproblems and optimize them alternately via hierarchical RL. This not only substantially reduces the complexity of each subproblem, but also improves the convergence efficiency. Experimental results show that HT3O can achieve promising performance improvements over state-of-the-art approaches.

Mobile edge computing (MEC) has been envisioned as a promising paradigm that could effectively enhance the computational capacity of wireless user devices (WUDs) and quality of experience of mobile applications. One of the most crucial issues of MEC is computation offloading, which decides how to offload WUDs’ tasks to edge severs for further intensive computation. Conventional mathematical programming-based offloading approaches could face troubles in dynamic MEC environments due to the time-varying channel conditions (caused primarily by WUD mobility). To address the problem, reinforcement learning (RL) based offloading approaches have been proposed, which develop offloading policies by mapping MEC states to offloading actions. However, these approaches could fail to converge in large-scale MEC due to the exponentially-growing state and action spaces. In this article, we propose a novel online computation offloading approach that could effectively reduce task latency and energy consumption in dynamic MEC with large-scale WUDs. First, a RL-based computation offloading and energy transmission algorithm is proposed to accelerate the learning process. Then, a joint optimization method is adopted to develop the allocating algorithm, which obtains near-optimal solutions for energy and computation resources allocation. Simulation results show that the proposed approach can converge efficiently and achieve significant performance improvements over baseline approaches.

Recent years have witnessed the increasing popularity of mobile applications, e.g., virtual reality, unmanned driving, which are generally computation-intensive and latency-sensitive, posing a major challenge for resource-limited user equipment (UE). Mobile edge computing (MEC) has been proposed as a promising approach to alleviate the problem, by offloading mobile tasks to the edge server (ES) deployed in close proximity to UE. However, most existing task offloading algorithms are primarily based on centralized scheduling, which could suffer from the ‘curse of dimensionality’ in large MEC environments. To address this issue, this paper proposes a fully distributed task offloading approach based on multi-agent deep reinforcement learning, whose critic and actor neural networks are trained under the assistance of global and local network states, respectively. In addition, we design a model parameter aggregation mechanism, along with a normalized fine-tuned reward function, to further improve the learning efficiency of the training process. Simulation results show that our proposed approach could achieve substantial performance improvements over baseline approaches.