Deterministic Networking (DetNet) is a highly predictable and controllable network technology. It provides low packet loss rate and bounded latency data transmission for applications through resource reservation and scheduling mechanisms. However, DetNet is a hybrid traffic system, and the resource reservation mechanism cannot guarantee the deterministic requirements as the number of diverse deterministic applications increases. As a result, there is an urgent need for an efficient and fine-grained scheduling mechanism to meet the deterministic and bounded latency requirements. In this paper, we propose a novel end-to-end multi-policy deep reinforcement learning framework for automatically learning multiple policies and addressing the problem of multi-objective joint routing and scheduling. Specifically, we formulate the multi-action problem in joint routing and scheduling as a Multi-Markov Decision Process (MMDP) and design a new reward function to optimize multiple objectives. When optimizing the learning agent, we introduce an A3C-based multi-strategy optimization algorithm (A3C-MSO) to train two sub-policies, including the queue operation policy and the node operation policy for assigning queue operations to nodes. Furthermore, we integrate a graph convolutional network (GCN) into the learning framework to capture the spatial characteristics of irregular network topologies and enhance the algorithm’s generalization ability. Extensive experimental results in different scenarios indicate that compared to the existing state-of-the-art mechanisms, the proposed mechanism has shown a 13% improvement in schedulability and an 18% enhancement in resource utilization. Particularly in high-load scenarios, the time cost of the proposed mechanism can be reduced by up to 40.5%. Furthermore, results obtained on real industrial network topology instances indicate that the proposed learning strategies exhibit good generalization and effectiveness in large-scale scheduling instances.

The ever-growing carbon emission of information infrastructure accounts for a significant proportion of the global carbon emissions. Existing studies reduce carbon consumption mainly by improving power efficiency on specific facilities or energy source structures. However, these methods do not jointly consider the impact of computation and network resource distribution on carbon emission. In this paper, we propose a data-driven scheme named EcoNet using reinforcement learning to reduce carbon emissions by jointly scheduling computation and network resources. We dynamically monitor the status of the computation and network facilities using cloud-edge collaboration and software-defined networking. Based on the collected status information, we formulate the resource scheduling problem as an optimization problem, which comprehensively considers the carbon emission, electricity price, and quality of service. The problem has high computation complexity, and we solve the problem with the proposed EcoNet to achieve efficient scheduling and near-optimal performance based on the collected network status information. The evaluation results show that EcoNet can maintain good Quality of Service and save at least 17% of the overall cost considering the electricity bills and carbon emissions.

Traditional routing schemes usually use fixed models for routing policies and thus are not good at handling complicated and dynamic traffic, leading to performance degradation (e.g., poor quality of service). Emerging Deep Reinforcement Learning (DRL) coupled with Software-Defined Networking (SDN) provides new opportunities to improve network performance with automatic traffic analysis and policy generation. However, existing DRL-based routing solutions usually rely on all node information to make routing decisions for the network and hence are both hard to converge in large networks and vulnerable to topology changes. In this paper, we propose ScaleDeep, a scalable DRL-based routing scheme for SDN, which improves the routing performance and is resilient to topology changes. Essentially, ScaleDeep takes advantage of partial control on network nodes and DRL. We select a set of critical nodes from a network as driver nodes, which can simulate the entire network operation, based on the control theory. By observing the traffic variation on the driver nodes, DRL dynamically adjusts some link weights for a weighted shortest path algorithm to change the routing paths and improve the routing performance. Limiting the control on driver nodes improves the convergence ability of DRL and reduces the dependency of the DRL agent on the fixed network topology. To validate the performance of ScaleDeep, we conduct packet-level simulations on different topologies. The results show that ScaleDeep outperforms existing DRL-based schemes by reducing the average flow completion time by up to 36% and exhibiting better robustness against minor topology changes.

Machine learning and deep learning methods have been widely used in network intrusion detection, most of which are supervised intrusion detection methods, which need to train a lot of marked data. However, in some cases, a small amount of exception data is hidden in a large amount of exception data, making methods that require a large amount of the same markup data to learn features invalid. In order to solve this problem, this paper proposes an innovative method of small sample network intrusion detection. The innovation point is that network data is modeled as graph structure to effectively mine the correlation features between data samples, and by comparing the distance similarity, the triplet network structure is used to detect anomalies. The triplet network is composed of triplet graph convolutional neural network which shares the same parameters and is trained by providing triplet samples to the network. Experiments on network traffic datasets CSE-CIC-IDS2018 and UNSW-NB15 as well as system status monitoring datasets verify the effectiveness of the proposed method in network intrusion detection of small samples.

When traditional machine learning methods are applied to network intrusion detection, they need to rely on expert knowledge to extract feature vectors in advance, which incurs lack of flexibility and versatility. Recently, deep learning methods have shown superior performance compared with traditional machine learning methods. Deep learning methods can learn the raw data directly, but they are faced with expensive computing cost. To solve this problem, a preprocessing method based on multipacket input unit and compression is proposed, which takes m data packets as the input unit to maximize the retention of information and greatly compresses the raw traffic to shorten the data learning and training time. In our proposed method, the CNN network structure is optimized and the weights of some convolution layers are assigned directly by using the Gabor filter. Experimental results on the benchmark data set show that compared with the existing models, the proposed method improves the detection accuracy by 2.49% and reduces the training time by 62.1%. In addition, the experiments show that the proposed compression method has obvious advantages in detection accuracy and computational efficiency compared with the existing compression methods.

Reducing the power consumption and maintaining the Flow Completion Time (FCT) for the Quality of Service (QoS) of applications in Data Center Networks (DCNs) are two major concerns for data center operators. However, existing works either fail in guaranteeing the QoS due to the neglect of the FCT constraints or achieve a less satisfying power efficiency. In this paper, we propose SmartFCT, which employs Software-Defined Networking (SDN) coupled with the Deep Reinforcement Learning (DRL) to improve the power efficiency of DCNs and guarantee the FCT. The DRL agent can generate a dynamic policy to consolidate traffic flows into fewer active switches in the DCN for power efficiency, and the policy also leaves different margins in different active links and switches to avoid FCT violation of unexpected short bursts of flows. Simulation results show that with similar FCT guarantee, SmartFCT can save 8% more of the power consumption compared to the state-of-the-art solutions.

Machine Learning (ML)-based routing optimization has been proposed to optimize the performance of flow routing for future networks, such as Software-Defined Networks (SDNs). However, existing studies are either hard to converge for large networks or vulnerable to topology changes. In this paper, we propose SINET, a scalable and intelligent network control framework for routing optimization. To improve the robustness and scalability, SINET selects several critical routing nodes to be directly controlled by a Deep Reinforcement Learning (DRL) agent, which dynamically generates routing policy to optimize network performance. Simulation results show that SINET can reduce the average flow completion time by at least 32% for a network with 82 nodes and exhibit better robustness against minor topology changes, compared to other DRL-based schemes.

Network Function Virtualization (NFV) enables flexible deployment of network services as applications. Network operators expect to use a limited number of Network Function (NF) instances to handle the fluctuating traffic load and provide network services. However, it is a big challenge to guarantee the Quality of Service (QoS) under the unpredictable network traffic while minimizing the processing resources. One typical solution is to realize NF scale-out, scale-in and load balancing by elastically migrating the related traffic flows with SoftwareDefined Networking (SDN). However, it is difficult to optimally migrate flows since many real-time statuses of NF instances should be considered to make accurate decisions. In this paper, we propose DeepMigration to solve the problem by efficiently and dynamically migrating traffic flows among different NF instances. DeepMigration is a Deep Reinforcement Learning (DRL)-based solution coupled with Graph Neural Network (GNN). By taking advantages of the graph-based relationship deduction ability from our customized GNN and the self-evolution ability from the experience training of DRL, DeepMigration can accurately model the cost (e.g., migration latency) and the benefit (e.g., reducing the number of NF instances) of flow migration among different NF instances and generate dynamic and effective flow migration policies to improve the QoS. Experiment results show that DeepMigration requires less migration cost and saves up to 71.6{%} of the computation time than existing solutions.

To improve the processing efficiency of jobs in distributed computing, the concept of coflow is proposed. A coflow is a collection of flows that are semantically correlated in a multi-stage computation task. A job consists of multiple coflows and can be usually formulated as a Directed-Acyclic Graph (DAG). A proper scheduling of coflows can significantly reduce the completion time of jobs in distributed computing. However, this scheduling problem is proved to be NP-hard. Different from existing schemes that use hand-crafted heuristic algorithms to solve this problem, in this paper, we propose a Deep Reinforcement Learning (DRL) framework named DeepWeave to generate coflow scheduling policies. To improve the inter-coflow scheduling ability in the job DAG, DeepWeave employs a Graph Neural Network (GNN) to process the DAG information. DeepWeave learns from the history workload trace to train the neural networks of the DRL agent and encodes the scheduling policy in the neural networks, which make coflow scheduling decisions without expert knowledge or a pre-assumed model. The proposed scheme is evaluated with a simulator using real-life traces. Simulation results show that DeepWeave completes jobs at least 1.7X faster than the state-of-the-art solutions.

Multi-controller is a promising control plane solution for the large-scale Software-Defined Networks (SDN). Some existing works (e.g., OpenFlow 1.2) propose to use backup controllers named slave controllers to achieve fault- tolerance in the control plane. In this paper, we identify the unreasonable slave controller assignment could cause the controller chain failure and eventually crash the entire network. We consider some important factors for designing fault-tolerant control plane and formulating Slave Controller Assignment (SCA) problem. SCA is an NP-complete problem, and we solve it with Adaptive Slave Controller Assignment (ASCA) scheme, which adaptively assigns slave controller according to load variance difference. The numerical results validate the efficiency of ASCA.

In large-scale Internet applications running on geographically distributed datacenters, such as video streaming, it is important to efficiently allocate requests among datacenters. To the best of our knowledge, existing approaches, however, either solely focus on minimizing total cost for provider, or guaranteeing QoS for end-users. In this paper, we apply the software defined network (SDN) controller to enable the central control of the entire network, and propose a joint optimization model to consider high bandwidth utilization for provider and low delay for users. We present the Nash bargaining solution (NBS) based method to model both requirements of provider's high bandwidth utilization and end-users' low delay. Specifically, we formulate the design of request allocation under those requirements as an optimization problem, which is NP-hard. To solve such hard optimization problem, we develop an efficient algorithm blending the advantages of Logarithmic Smoothing technique and the auxiliary variable method. According to the theoretical analysis, we verify the existence and uniqueness of our solution and the convergence of our algorithm. We conduct a large amount of experiments based on real-world workload traces and demonstrate the efficiency of our algorithm compared to both greedy and locality algorithms.

Despite the critical role that middleboxes play in introducing new network functionality, management and innovation of them are still severe challenges for network operators, since traditional middleboxes based on hardware lack service flexibility and scalability. Recently, though new networking technologies, such as network function virtualization (NFV) and software-defined networking (SDN), are considered as very promising drivers to design cost-efficient middlebox service architectures, how to guarantee transmission efficiency has drawn little attention under the condition of adding virtual service process for traffic. Therefore, we focus on the service deployment problem to reduce the transport delay in the network with a combination of NFV and SDN. First, a framework is designed for service placement decision, and an integer linear programming model is proposed to resolve the service placement and minimize the network transport delay. Then a heuristic solution is designed based on the improved quantum genetic algorithm. Experimental results show that our proposed method can calculate automatically the optimal placement schemes. Our scheme can achieve lower overall transport delay for a network compared with other schemes and reduce 30% of the average traffic transport delay compared with the random placement scheme.

Conference key management requires relatively heavy-weight modular exponentiation operations and additional communications among group members. So time efficiency of key update for conference key management is the key problem many literatures strived to address. In this paper, we propose a Huffman-based join-exit-tree (HJET) key agreement to achieve better time efficiency in key update. Compared with JET scheme of Mao et al., HJET has two major improvements. First, the join algorithm inserting the new node into the root of the join tree makes the join cost minimal and constant. Second, Huffman coding is used to form the exit tree with the information of users' withdrawal probabilities, and therefore has optimal average leave cost. Performance analysis and simulation results demonstrate that HJET is efficient in key update and achieves the asymptotic time cost of O(1) for join event and nearly O(1) for leave events.

With emergence of group-oriented applications needing content confidentiality, secure group communications have drawn more attention. To provide this service in large groups with highly dynamic memberships, a secure group key management efficient in key establishment and update is the foundation. In this paper, we present a join-tree-based contributory group key management (JDH) to achieve better time efficiency, and propose using the notion "sequential exponentiations" as the evaluation metric for time efficiency. First, a new key tree topology comprised of main tree and join tree is put forward, with the join tree locating close to the root of the key tree and serving as the temporary buffer for sequential joining users. Then, a new join algorithm in the join tree is presented to reduce the time complexity. Last, optimal capacity of the join tree is selected through an optimization method. Theoretical analysis shows that the asymptotic average join time is sharply reduced to from previous, where is the group size. Our analytical comparison with existing managements and experiments demonstrate that JDH is time and communication efficient in group key establishment and update.

As Internet grows exponentially, scalable routers on backbone are required to provide more number of ports, higher line-rates, and larger capacity under acceptable complexity. Until now, most routers are implemented on the centralized single crossbar as the switched backplane fabric. In terms of crosspoint number, however, the complexity of a single Crossbar is unacceptable with large number of ports, which is increased with O(N2). Distributed multiple-stage Clos network and Parallel Packet Switching fabric were proposed to provide large number of ports and high line-card rate, respectively. To obtain both goals simultaneously, we study a novel multiple-plane and multiple-stage (MPMS) switching fabric in this paper. We first bring out a graphic model for the MPMS fabric based on its topological architecture. Then we study the internal connectivity of the MPMS fabric through the concepts of vertex in-degree, vertex out-degree and vertex mux degree. Lastly, we analyze the performance of the MPMS fabric including its maximum number of ports, line-rate, switching capacity and complexity of crosspoints by comparison to that of the single-stage crossbar fabric.

The multiple-plane and multiple-stage (MPMS) switching fabrics are the next step in scaling current crossbar fabrics to many hundreds or few thousands of ports. However, scheduling cells in the MPMS fabric is complex. With the recent blooming of bandwidth sensitive Internet traffic, scheduling cells with guaranteed bandwidth is becoming an urgent demand. The CRRD algorithm delivers high throughput under uniform traffic pattern, but it does not work well under nonuniform traffic and does not provide any bandwidth guarantees. In this paper, we analyze the graphic model of the MPMS fabric, and propose a novel bandwidth-guaranteed scheduling algorithm based on CRRD. Simulation results show that it delivers 100% throughput under uniform traffic, and achieves much higher throughput than that of CRRD under nonuniform traffic, and keeps its implementation complexity low without internal expansion and allocates the output-link bandwidth fairly for the reserved flows in the overloaded case.

Protocol independent multicast-sparse mode can use either a shared tree or a shortest path tree to deliver IPv6 multicast packets, consequently the multicast IP lookup engine requires, in some cases, two searches to get a correct forwarding decision, and this will lead to a new requirement of doubling the lookup speed. The ordinary method to satisfy this requirement in TCAM-based (ternary content addressable memory) lookup engines is to exploit parallelism among multiple TCAMs, however, parallel methods always incur more resources and higher design difficulty. We propose in this paper a approach to solve this problem. By arranging multicast forwarding table in class sequence in TCAM together with using the intrinsic characteristic of the TCAM, our approach can use just one search and a single TCAM to get the right lookup result, while keeping the hardware of lookup engine unchanged. Experimental results have shown that the approach can make it possible for just one TCAM to satisfy forwarding IPv6 multicast packets at the full link rate of 20 Gb/s with the current TCAM chip level.

The ever increasing line speed and the continuous growing demands of various functions support(for example QoS, multicast and security) have interact- tively made it harder for forwarding engines to process packets at line speed, and this will increasingly make the forwarding engines call for additional buffers to accommodate the burst transmission and decrease the packet loss rate. In this paper, a high-speed pipeline designed for TCAM-based forwarding engines is presented, and its buffer analysis model is also given, then, the buffer requirement of the forwarding engine is analyzed under two conditions: the forwarding rate is not less than and less than the input rate. Our analysis results and experiments both show that, the proposed forwarding pipeline is of high performance, and just one pipeline can easily deal with the data transfer rate of 30 Gb/s or even higher; the pipelined forwarding engine only need to buffer a several packets, then the loss rate will be an acceptable value or even zero, further increasing the buffer size will have little effect on reducing the loss rate.

Centralized parallel packet switch algorithm and distributed parallel packet switch algorithm are two typical scheduling algorithms for parallel packet switch. This paper analyzes the two algorithms in detail, addresses several key problems in their implementation and finally presents several available methods and suggestions to make the parallel packet switch more practical