In 5G wireless networks, the User Plane Function (UPF) plays a crucial role in efficiently transferring users’ traffic — a series of data packets — to manage internet communications. Setting the server’s processor frequency excessively high can easily meet the packet drop requirements but may lead to unnecessary power consumption. Therefore, as user traffic fluctuates, selecting the optimal processor frequency is essential for minimizing power consumption while satisfying packet drop constraints. This challenge motivates us to address the dynamic resource (frequency) allocation problem, where deep reinforcement learning (RL) has shown significant potential. Most existing studies train and evaluate the RL model in the same environment with consistent traffic patterns. However, frequent variations in user traffic can cause the policy trained on the outdated traffic to fail catastrophically on unseen traffic.To address such traffic distribution shifts, we propose a two-phase RL approach augmented with Automatic Domain Randomization (RL-ADR). This method includes a training phase that utilizes domain randomization to create a library of policy candidates, and an inference phase that selects the optimal frequency using this policy library alongside a safe data buffer. The proposed RL-ADR achieves zero packet drops on two unseen long-horizon traffics (3 hours) after being trained on 25 synthetic traffics that only span for 18 seconds. Compared to static resource allocation baselines, RL-ADR reduces power consumption by at least 14.5% and performs comparably to the oracle solution.

The recent development of reinforcement learning (RL) has boosted the adoption of online RL for wireless radio resource management (RRM). However, online RL algorithms require direct interactions with the environment, which may be undesirable given the potential performance loss due to the unavoidable exploration in RL. In this work, we first explore the use ofofflineRL algorithms in solving the RRM problem. We evaluate several state-of-the-art offline RL algorithms for a practical RRM problem that aims at maximizing a linear combination of total rates and 5-percentile rates via user scheduling. Our findings indicate that the performance of offline RL for the RRM problem is heavily contingent upon the behavior policy deployed for data collection. We propose an innovative offline RL approach utilizing heterogeneous datasets from various behavior policies. This method demonstrates that a strategic mixture of datasets enables near-optimal RL policy generation, even with suboptimal behavior policies. Additionally, we introduce two enhancements: an ensemble-based policy to augment dataset mixture training efficiency, and a novel offline-to-online strategy for seamless adaptation to new environments. Our data mixture approach achieves over 95% efficiency of an online RL agent in the absence of expert data. The ensemble algorithm notably reduces training duration by half compared to the data mixture method. Furthermore, our model, when applied with offline-to-online fine-tuning, surpasses existing benchmarks by approximately 5% in our user scheduling problem.

Network Function Virtualization (NFV), the replacement of purpose-built network appliances with software functions running on general purpose compute servers, is ubiquitous in today's telecommunication networks. The 5G User Plane Function (UPF) is an important example of an NFV workload, which enables 5G and internet communications. The UPF has strict packet drop requirements and because user traffic load can vary dramatically throughout the day, the selection of a single static configuration leads to over-provisioning of server resources. To reduce the cost of ownership, network operators can reduce power consumption during periods of low traffic load, but to do so they must ensure that packet drop requirements are met. In this paper we present DPM-NFV, a machine learning based framework that enables dynamic tuning of a real NFV system. Our methodology is composed of two phases: (1) Offline, targeted automated studies use Bayesian Optimization to infer the best configurations for various load levels; (2) Online, a run-time classifier dynamically selects the best configuration for the current load. Our results obtained on a real system demonstrate that the UPF can meet strict packet drop requirements while reducing power consumption by up to 52% with smooth traffic and up to 46% with bursty traffic.

We propose a mechanism for distributed resource management and interference mitigation in wireless networks using multi-agent deep reinforcement learning (RL). We equip each transmitter in the network with a deep RL agent that receives delayed observations from its associated users, while also exchanging observations with its neighboring agents, and decides on which user to serve and what transmit power to use at each scheduling interval. Our proposed framework enables agents to make decisions simultaneously and in a distributed manner, unaware of the concurrent decisions of other agents. Moreover, our design of the agents' observation and action spaces is scalable, in the sense that an agent trained on a scenario with a specific number of transmitters and users can be applied to scenarios with different numbers of transmitters and/or users. Simulation results demonstrate the superiority of our proposed approach compared to decentralized baselines in terms of the tradeoff between average and 5thpercentile user rates, while achieving performance close to, and even in certain cases outperforming, that of a centralized information-theoretic baseline. We also show that our trained agents are robust and maintain their performance gains when experiencing mismatches between train and test deployments.

We study the performance of TCP/ABR and TCP/UBR as a function of the number of bottlenecks in an IP/ATM inter-networking system. We define an efficiency metric that captures the amount of badput generated per unit of goodput. We define a gain metric to be the ratio of the efficiencies of these two services. With these new metrics, we demonstrate that the bandwidth efficiency of TCP/ABR is scalable in the number of bottlenecks, whereas TCP/UBR is scalable only if there are no greedy sources in the traffic mix. We examine the influence of ABR and UBR on TCP factors such as packet loss, round trip time delays (RTTs), and the fraction of lost packets detected via fast retransmit events. We show that TCP/ABR is more efficient than TCP/UBR because the ABR control loop has favorable effects on the TCP control loop via its its influence on RTTs and loss behavior. We demonstrate that fairness has far reaching consequences beyond throughput fairness because the improvements to TCP in efficiency and scalability are a ramification of fairness.

In this paper we describe the implementation of our end-to-end QoS-driven resource management scheme, called ERDoS, within a CORBA-compliant ORB that we call the ERDoS QoS ORB. Unlike other real-time CORBA implementations that focus on real-time support for simple client-server applications, our ERDoS QoS ORB provides end-to-end QoS support (i.e., QoS spanning computer network, and storage resources) to applications, while retaining the benefits of an open distributed object system. Specifically we present three contributions. First, we present a model for describing end-to-end applications as a combination of client-server interactions between CORBA objects. Second, we define a model for relating the user/application level QoS requirements to the corresponding resource demand requirements of these individual CORBA objects. Third, we suggest a framework for performing distributed resource management within the CORBA environment.

Because the ATM Forum does not standardize the ABR flow control algorithm that an ATM switch should run, some ATM networks are likely to contain switches that run different ABR flow control algorithms. Even if all the switches within a "cloud" of switches use the same algorithm, individual clouds (each with it own algorithm) will be interconnected by virtual circuits. Virtual circuits which traverse multiple clouds will therefore be controlled by two different flow control algorithms concurrently. We explore some of the ramifications of mixing different flow control algorithms in the same ATM network. We identify the rate mismatch problem, which arises when a nonbottleneck switch (that uses one algorithm) interferes with the control of the bottleneck switch (that uses another algorithm). We formulate a hypothesis that states the conditions that lead to the rate mismatch problem. These conditions identify a specific class of problematic topologies. We validate the hypothesis formally and prove that rate mismatch causes unfairness. Using four different algorithms, in combinations of two at a time, we illustrate the interoperability of ABR flow control algorithms.