Decentralized task offloading among cooperative edge nodes has been a promising solution to enhance resource utilization and improve users’ Quality of Experience (QoE) in edge computing. However, current decentralized methods, such as heuristics and game theory-based methods, either optimize greedily or depend on rigid assumptions, failing to adapt to the dynamic edge environment. Existing DRL-based approaches train the model in a simulation and then apply it in practical systems. These methods may perform poorly because of the divergence between the practical system and the simulated environment. Other methods that train and deploy the model directly in real-world systems face a cold-start problem, which will reduce the users’ QoE before the model converges. This paper proposes a novel offline-to-online DRL called (O2O-DRL). It uses the heuristic task logs to warm-start the DRL model offline. However, offline and online data have different distributions, so using offline methods for online fine-tuning will ruin the policy learned offline. To avoid this problem, we use on-policy DRL to fine-tune the model and prevent value overestimation. We evaluate O2O-DRL with other approaches in a simulation and a Kubernetes-based testbed. The performance results show that O2O-DRL outperforms other methods and solves the cold-start problem.

As a key enabler of future 5G network, Service Function Chain (SFC) forwards the traffic flow along a chain of Virtual Network Functions (VNFs) to provide network services flexibility. One of the most important problems in SFC is to deploy the VNFs and schedule arriving requests among computing nodes to achieve low latency and high reliability. Existing works consider a static network and assume that all SFC requests are known in advance, which is impractical. In this paper, we focus on the dynamic 5G network environment where the SFC request arrives randomly following a certain distribution. Computing nodes can redeploy all types of VNF with a time cost. We formulate the problem of SFC scheduling in NFV-enabled 5G network as a mixed integer non-linear programing. The objective is to maximize the number of requests satisfying the latency and reliability constraints. To solve the problem, we propose an efficient algorithm to decide the redundancy of the VNFs while minimizing delay. Then we present a state-of-art Reinforcement Learning (RL) to learn SFC scheduling policy to increase the success rate of SFC requests. The effectiveness of our method is evaluated through extensive simulations. The result shows that our proposed RL solution can increase the success rate by 18.7% over the benchmark algorithms.