The emerging intelligent services, spurred by the rise of the intelligent Internet, are placing multidimensional requirements on the network to collaboratively guarantee computing and networking resources. In this article, we propose a unified end-to-end intelligent resource scheduling method for converging networks [e.g., Internet of Things (IoT)], which can always globally abstract the available resources from different networks with a unified model description, and jointly planning the resources from end-to-end by deep reinforcement learning (DRL) algorithms supporting both discrete and continuous variable decisions. The method proposes a three-layer architecture, including service layer, network layer, and adaption layer, which aims at optimizing the flow transmission performance. Through the general Markov decision process (MDP) transformation from the model, the DRL-assisted algorithm can further solve the optimization problem. We categorize heterogeneous network resource scheduling into horizontal and vertical scenarios, applying the proposed architecture to both. Compared with the existing diverse learning (DiLearn) and naive (DiNaive) approaches, the proposed approach is not only time-saving but also can schedule 28.4% and$8\times $more flows in horizontal scheduling scenarios, and improve 54.2% and$3.5\times $flows in vertical scheduling scenarios, respectively.

Deterministic network services play a vital role for supporting emerging real-time applications with bounded low latency, jitter, and high reliability. The deterministic guarantee is penetrated into various types of networks, such as 5G, WiFi, satellite, and edge computing networks. From the user’s perspective, the real-time applications require end-to-end deterministic guarantee across the converged network. In this paper, we investigate the end-to-end deterministic guarantee problem across the whole converged network, aiming to provide a scalable method for different kinds of converged networks to meet the bounded end-to-end latency, jitter, and high reliability demands of each flow, while improving the network scheduling QoS. Particularly, we set up the global end-to-end control plane to abstract the deterministic-related resources from converged network, and model the deterministic flow transmission by using the abstracted resources. With the resource abstraction, our model can work well for different underlying technologies. Given large amounts of abstracted resources in our model, it is difficult for traditional algorithms to fully utilize the resources. Thus, we propose a deep reinforcement learning based end-to-end deterministic-related resource scheduling (E2eDRS) algorithm to schedule the network resources from end to end. By setting the action groups, the E2eDRS can support varying network dimensions both in horizontal and vertical end-to-end deterministic-related network architectures. Experimental results show that E2eDRS can averagely increase 1.33x and 6.01x schedulable flow number for horizontal scheduling compared with MultiDRS and MultiNaive algorithms, respectively. The E2eDRS can also optimize 2.65x and 3.87x server load balance than MultiDRS and MultiNaive algorithms, respectively. For vertical scheduling, the E2eDRS can still perform better on schedulable flow number and server load balance.

Deterministic transmission guarantee in time-sensitive networks (TSN) relies on queue models (such as CQF, TAS, ATS) and resource scheduling algorithms. Thanks to its ease of use, the CQF queue model has been widely adopted. However, the existing resource scheduling algorithms of CQF model only focus on periodic time-triggered (TT) flows without consideration of bursting flows. Considering that the bursting flows often carry high-priority data in real systems, in this paper we investigate the mixed-flow (i.e., TT and bursting flows) scheduling problem in CQF-based TSN aiming to maximize the number of schedulable flows and system load balance while satisfying the deterministic demands of delay, jitter, and reliability for both TT and bursting flows. Unfortunately, it is challenging to schedule the mixed flows with the original CQF model because of the huge difference between TT and bursting flows. To resolve this problem, we firstly design an enhanced Multi-CQF model to satisfy the basic demands of bursting flows sent at any time without affecting the deterministic transmission of TT flows. Given the complexity of mixed-flow scheduling and the proposed queue model, it is difficult for traditional algorithms to fully utilize network resources. Thus, we further propose a uline time-correlated uline DRL uline resource uline scheduling (TimeDRS) algorithm to optimize the resource allocation. TimeDRS can be extended to other time-related resource scheduling scenarios, such as TDMA-based scheduling. Experimental results demonstrate that our proposed approaches can greatly reduce frame loss and end-to-end latency for bursting flows, and well balance runtime and schedulability compared with state-of-the-art benchmarks.