Congestion Control (CC) plays a fundamental role in optimizing traffic in Datacenter Networks (DCNs). Currently, DCNs implement two main CC protocols: DCTCP and DCQCN. Both protocols are based on Explicit Congestion Notification (ECN), where switches mark packets when they detect congestion. Nowadays, network experts carefully set ECN parameters to optimize the average network performance. However, today’s DCNs experience rapid and abrupt changes that severely affect the network state (e.g., dynamic workloads, incasts), which leads to under-utilization and sub-optimal performance. In this paper we present GraphCC , a framework for in-network CC optimization. GraphCC relies on Multi-agent Reinforcement Learning (MARL) and Graph Neural Networks (GNN), and is compatible with widely deployed ECN-based CC protocols. The proposed solution deploys distributed agents on switches that communicate with their neighbors to cooperate and optimize the global ECN configuration. In our evaluation, we test GraphCC with three real-world traffic workloads, focusing on its capability to accommodate scenarios unseen during training (e.g., traffic changes, failures). We compare GraphCC with a state-of-the-art MARL solution for ECN tuning, and observe that our method outperforms the state-of-the-art baseline in all evaluation scenarios, with improvements up to 20% in average Flow Completion Time, similar mean throughput (within 1%), and significant reductions in buffer occupancy (38.0–85.7%).

We seek the best traffic allocation scheme for the edge–cloud networking subject to SD-WAN architecture and burstable billing. First, we formulate a family of quantile-based integer programming problems for a fixed network topology with random parameters describing the traffic demands. Then, to overcome the difficulty caused by the discrete feature, we generalize the Gumbel-softmax reparameterization method to induce an unconstrained continuous optimization problem as a regularized continuation of the discrete problem. Finally, we introduce the Gumbel-softmax sampling neural network to solve optimization problems via unsupervised learning. The neural network structure reflects the edge–cloud networking topology and is trained to minimize the expectation of the cost function for unconstrained continuous optimization problems. The trained network works as an efficient traffic allocation scheme sampler, outperforming the random strategy in feasibility and cost value. Besides testing the quality of the output allocation scheme, we examine the generalization property of the network by increasing the time steps and the number of users. We also feed the solution to existing integer optimization solvers as initial conditions and verify the warm-starts can accelerate the short-time iteration process. The framework is general, and the decoupled feature of the random neural networks is adequate for practical implementations.

Network models are an essential block of modern networks. For example, they are widely used in network planning and optimization. However, as networks increase in scale and complexity, some models present limitations, such as the assumption of Markovian traffic in queuing theory models, or the high computational cost of network simulators. Recent advances in machine learning, such as Graph Neural Networks (GNN), are enabling a new generation of network models that are data-driven and can learn complex non-linear behaviors. In this paper, we present RouteNet-Fermi, a custom GNN model that shares the same goals as Queuing Theory, while being considerably more accurate in the presence of realistic traffic models. The proposed model predicts accurately the delay, jitter, and packet loss of a network. We have tested RouteNet-Fermi in networks of increasing size (up to 300 nodes), including samples with mixed traffic profiles — e.g., with complex non-Markovian models — and arbitrary routing and queue scheduling configurations. Our experimental results show that RouteNet-Fermi achieves similar accuracy as computationally-expensive packet-level simulators and scales accurately to larger networks. Our model produces delay estimates with a mean relative error of 6.24% when applied to a test dataset of 1,000 samples, including network topologies one order of magnitude larger than those seen during training. Finally, we have also evaluated RouteNet-Fermi with measurements from a physical testbed and packet traces from a real-life network.

Network modeling is an essential tool for network planning and management. It allows network administrators to explore the performance of new protocols, mechanisms, or optimal configurations without the need for testing them in real production networks. Recently, Graph Neural Networks (GNNs) have emerged as a practical solution to produce network models that can learn and extract complex patterns from real data without making any assumptions. However, state-of-the-art GNN-based network models only work with traffic matrices, this is a very coarse and simplified representation of network traffic. Although this assumption has shown to work well in certain use-cases, it is a limiting factor because, in practice, networks operate with flows. In this paper, we present FlowDT a new DL-based solution designed to model computer networks at the fine-grained flow level. In our evaluation, we show how FlowDT can accurately predict relevant per-flow performance metrics with an error of 3.5%, FlowDT’s performance is also benchmarked against vanilla DL models as well as with Queuing Theory.

Network modeling is a fundamental tool in network research, design, and operation. Arguably the most popular method for modeling is Queuing Theory (QT). Its main limitation is that it imposes strong assumptions on the packet arrival process, which typically do not hold in real networks. In the field of Deep Learning, Graph Neural Networks (GNN) have emerged as a new technique to build data-driven models that can learn complex and non-linear behavior. In this paper, we present RouteNet-Erlang, a pioneering GNN architecture designed to model computer networks. RouteNet-Erlang supports complex traffic models, multi-queue scheduling policies, routing policies and can provide accurate estimates in networks not seen in the training phase. We benchmark RouteNet-Erlang against a state-of-the-art QT model, and our results show that it outperforms QT in all the network scenarios.

The recent growth of emergent network applications (e.g., satellite networks, vehicular networks) is increasing the complexity of managing modern communication networks. As a result, the community proposed the Digital Twin Networks (DTN) as a key enabler of efficient network management. Network operators can leverage the DTN to perform different optimization tasks (e.g., Traffic Engineering, Network Planning).Deep Reinforcement Learning (DRL) showed a high performance when applied to solve network optimization problems. In the context of DTN, DRL can be leveraged to solve optimization problems without directly impacting the real-world network behavior. However, DRL scales poorly with the problem size and complexity. In this paper, we explore the use of Evolutionary Strategies (ES) to train DRL agents for solving a routing optimization problem. The experimental results show that ES achieved a training time speed-up of 128 and 6 for the NSFNET and GEANT2 topologies respectively.

Wide Area Networks (WAN) are a key infrastructure in today’s society. During the last years, WANs have seen a considerable increase in network’s traffic and network applications, imposing new requirements on existing network technologies (e.g., low latency and high throughput). Consequently, Internet Service Providers (ISP) are under pressure to ensure the customer’s Quality of Service and fulfill Service Level Agreements. Network operators leverage Traffic Engineering (TE) techniques to efficiently manage the network’s resources. However, WAN’s traffic can drastically change during time and the connectivity can be affected due to external factors (e.g., link failures). Therefore, TE solutions must be able to adapt to dynamic scenarios in real-time. In this paper we propose Enero, an efficient real-time TE solution based on a two-stage optimization process. In the first one, Enero leverages Deep Reinforcement Learning (DRL) to optimize the routing configuration by generating a long-term TE strategy. To enable efficient operation over dynamic network scenarios (e.g., when link failures occur), we integrated a Graph Neural Network into the DRL agent. In the second stage, Enero uses a Local Search algorithm to improve DRL’s solution without adding computational overhead to the optimization process. The experimental results indicate that Enero is able to operate in real-world dynamic network topologies in 4.5 s on average for topologies up to 100 links.

Network modeling is a critical component of Quality of Service (QoS) optimization. Current networks implement Service Level Agreements (SLA) by careful configuration of both routing and queue scheduling policies. However, existing modeling techniques are not able to produce accurate estimates of relevant SLA metrics, such as delay or jitter, in networks with complex QoS-aware queueing policies (e.g., strict priority, Weighted Fair Queueing, Deficit Round Robin). Recently, Graph Neural Networks (GNNs) have become a powerful tool to model networks since they are specifically designed to work with graph-structured data. In this paper, we propose a GNN-based network model able to understand the complex relationship between (i) the queueing policy (scheduling algorithm and queue sizes), (ii) the network topology, (iii) the routing configuration, and (iv) the input traffic matrix. We call our model TwinNet, a Digital Twin that can accurately estimate relevant SLA metrics for network optimization. TwinNet can generalize to its input parameters, operating successfully in topologies, routing, and queueing configurations never seen during training. We evaluate TwinNet over a wide variety of scenarios with synthetic traffic and validate it with real traffic traces. Our results show that TwinNet can provide accurate estimates of end-to-end path delays in 106 unseen real-world topologies, under different queuing configurations with a Mean Absolute Percentage Error (MAPE) of 3.8%, as well as a MAPE of 6.3% error when evaluated with a real testbed. We also showcase the potential of the proposed model for SLA-driven network optimization and what-if analysis.

Traffic Engineering (TE) is a basic building block of the Internet. In this paper, we analyze whether modern Machine Learning (ML) methods are ready to be used for TE optimization. We address this open question through a comparative analysis between the state of the art in ML and the state of the art in TE. To this end, we first present a novel distributed system for TE that leverages the latest advancements in ML. Our system implements a novel architecture that combines Multi-Agent Reinforcement Learning (MARL) and Graph Neural Networks (GNN) to minimize network congestion. In our evaluation, we compare our MARL+GNN system with DEFO, a network optimizer based on Constraint Programming that represents the state of the art in TE. Our experimental results show that the proposed MARL+GNN solution achieves equivalent performance to DEFO in a wide variety of network scenarios including three real-world network topologies. At the same time, we show that MARL+GNN can achieve significant reductions in execution time (from the scale of minutes with DEFO to a few seconds with our solution).

Network slicing, as a key 5G enabling technology, is promising to support with more flexibility, agility, and intelligence towards the provisioned services and infrastructure management. Fulfilling these tasks is challenging, as nowadays networks are increasingly heterogeneous, dynamic and large-dimensioned. This contradicts the dominant network slicing solutions that only customize immediate performance over one snapshot of the system in the literature. Instead, this paper first presents a two-stage slicing optimization model with time-averaged metrics to safeguard the network slicing in the dynamical networks, where prior environmental knowledge is absent but can be partially observed at runtime. Directly solving an off-line solution to this problem is intractable since the future system realizations are unknown before decisions. Therefore, we propose a learning augmented optimization approach with deep learning and Lyapunov stability theories. This enables the system to learn a safe slicing solution from both historical records and run-time observations. We prove that the proposed solution is always feasible and nearly optimal, up to a constant additive factor. Finally, we demonstrate up to 2.6× improvement in the simulation when compared with three state-of-the-art algorithms.

As a key enabling technology for 5G network softwarization, network function virtualization (NFV) provides an efficient paradigm to optimize network resource utility for the benefits of both network providers and users. However, the inherent network dynamics and uncertainties from 5G infrastructure, resources, and applications are slowing down the further adoption of NFV in many emerging networking applications. Motivated by this, in this paper, we investigate the issues of network utility degradation when implementing NFV in dynamic networks, and design a proactive NFV solution from a fully stochastic perspective. Unlike existing deterministic NFV solutions, which assume given network capacities and/or static service quality demands, this paper explicitly integrates the knowledge of influential network variations into a two-stage stochastic resource utilization model. By exploiting the hierarchical decision structures in this problem, a distributed computing framework with two-level decomposition is designed to facilitate a distributed implementation of the proposed model in large-scale networks. The experimental results demonstrate that the proposed solution not only improves 3~5 folds of network performance, but also effectively reduces the risk of service quality violation.

Previously, we have proposed a C/U-plane (Control/User plane) decoupled railway wireless network in which higher frequency bands are adopted by small cells to provide wider available spectra for the U-plane of passengers' services. In order to guarantee reliable connectivity to wayside eNodeBs (eNBs), an onboard mobile relay (MR), consisting of two components, namely, MR-UE (User Equipment) and MR-AP (Access Point), is employed to forward passengers' services over backhaul links between MR-UE and wayside eNBs. The remaining problem here is how to utilize spectra effectively and efficiently under this new configuration. Since a given wayside eNB hosts only one single accessed user most of the time, we design an additional uplink scheduler for the MR-UE to self-manage the usage of uplink resources, avoiding the complicated uplink grant procedure commonly used in the conventional cellular systems. Moreover, we develop eNB schedulers to coordinate the spectra in small cells. To deal with occasional multi-user scenarios, we propose an uplink scheduler switcher for the macro cell to judge which uplink scheduler should be activated. Furthermore, an uplink resource allocation scheme with high spectrum efficiency is deliberately designed for the new dual-scheduler configuration. Finally, we carry out theoretical analysis and numerical simulations to demonstrate the effectiveness of our proposed scheme.