The virtualization of Radio Access Networks (vRAN) is rapidly becoming a reality, driven by the increasing need for flexible, scalable, and cost-effective mobile network solutions. To mitigate energy efficiency concerns in vRAN deployments, two approaches are gaining attention: ($i$) sharing computing infrastructure among multiple virtualized base stations (vBSs); and ($ii$) relying upon general-purpose, low-cost CPUs. However, effectively realizing these approaches poses several challenges. In this paper, we first conduct a comprehensive experimental campaign on a vRAN platform to characterize the impact of computing and radio resource allocation on energy consumption and performance across various network contexts. This analysis reveals several key issues. First, determining the optimal allocation of computing resources is difficult because it depends on the context of each vBS (e.g., traffic load, channel quality) in a non-trivial and non-linear manner. Second, suboptimal resource assignment can lead to increased energy consumption or, even worse, degradation of users' Quality of Service. Third, the high dimensionality of the solution space hinders the effectiveness of traditional optimization or learning methods. To tackle these challenges, we propose AegisRAN, a framework for optimizing computing resource allocation in vRAN. AegisRAN addresses the dual objective of minimizing energy consumption while maintaining high system reliability. Moreover, when computing resources are overbooked, our solution ensures a fair resource partition based on vBS performance. AegisRAN leverages a discrete soft actor-critic algorithm combined with several techniques, including multi-step decision-making, action masking, digital twin-based training, and a tailored reward signal that mitigates feedback sparsity. Our evaluations demonstrate that AegisRAN achieves near-optimal performance and offers high flexibility across diverse network contexts and varying numbers of vBSs, with up to 25% improvement in energy savings compared to baseline solutions in medium-scale scenarios.

Recent advances in learning techniques have garnered attention for their applicability to a diverse range of real-world sequential decision-making problems. Yet, many practical applications have critical constraints for operation in real environments. Most learning solutions often neglect the risk of failing to meet these constraints, hindering their implementation in real-world contexts. In this paper, we propose a risk-aware decision-making framework for contextual bandit problems, accommodating constraints and continuous action spaces. Our approach employs an actor multi-critic architecture, with each critic characterizing the distribution of performance and constraint metrics. Our framework is designed to cater to various risk levels, effectively balancing constraint satisfaction against performance. To demonstrate the effectiveness of our approach, we first compare it against state-of-the-art baseline methods in a synthetic environment, highlighting the impact of intrinsic environmental noise across different risk configurations. Finally, we evaluate our framework in a real-world use case involving a 5G mobile network where only our approach satisfies consistently the system constraint (a signal processing reliability target) with a small performance toll (8.5% increase in power consumption).

Radio Access Network (RAN) virtualization, key for new-generation mobile networks, requires Hardware Accelerators (HAs) that swiftly process wireless signals from Base Stations (BSs) to meet stringent reliability targets. However, HAs are expensive and energy-hungry, which increases costs and has serious environmental implications. To address this problem, we gather data from our experimental platform and compare the performance and energy consumption of a HA (NVIDIA GPU V100) vs. a CPU (Intel Xeon Gold 6240R, 16 cores) for energy-friendly software processing. Based on the insights obtained from this data, we devise a strategy to offload workloads to HAs opportunistically to save energy while preserving reliability. This offloading strategy, however, needs to be configured in near-real-time for every BS sharing common computational resources. This renders a challenging multi-agent collaborative problem in which the number of involved agents (BSs) can be arbitrarily large and can change over time. Thus, we propose an efficient multi-agent contextual bandit algorithm called ECORAN1, which applies concepts from mean field theory to be fully scalable. Using a real platform and traces from a production mobile network, we show that ECORAN can provide up to 40% energy savings with respect to the approach used today by the industry.

RAN virtualization is revolutionizing the telco industry, enabling 5G Distributed Units to run using general-purpose platforms equipped with Hardware Accelerators (HAs). Recently, GPUs have been proposed as HAs, hinging on their unique capability to execute 5G PHY operations efficiently while also processing Machine Learning (ML) workloads. While this ambivalence makes GPUs attractive for cost-effective deployments, we experimentally demonstrate that multiplexing 5G and ML workloads in GPUs is in fact challenging, and that using conventional GPU-sharing methods can severely disrupt 5G operations. We then introduce YinYangRAN, an innovative O-RAN-compliant solution that supervises GPU-based HAs so as to ensure reliability in the 5G processing pipeline while maximizing the throughput of concurrent ML services. YinYangRAN performs GPU resource allocation decisions via a computationally-efficient approximate dynamic programming technique, which is informed by a neural network trained on real-world measurements. Using workloads collected in real RANs, we demonstrate that YinYangRAN can achieve over 50% higher 5G processing reliability than conventional GPU sharing models with minimal impact on co-located ML workloads. To our knowledge, this is the first work identifying and addressing the complex problem of HA management in emerging GPU-accelerated vRANs, and represents a promising step towards multiplexing PHY and ML workloads in mobile networks.

Virtualized base stations (vBS) can be implemented in diverse commodity platforms and are expected to bring unprecedented operational flexibility and cost efficiency to the next generation of cellular networks. However, their widespread adoption is hampered by their complex configuration options that affect in a non-traditional fashion both their performance and their power consumption requirements. Following an in-depth experimental analysis in a bespoke testbed, we characterize the vBS power cost profile and reveal previously unknown couplings between their various control knobs. Motivated by these findings, we develop a Bayesian learning framework for the orchestration of vBSs and design two novel algorithms: (i) BP-vRAN, which employs online learning to balance the vBS performance and energy consumption, and (ii) SBP-vRAN, which augments our optimization approach with safe controls that maximize performance while respecting hard power constraints. We show that our approaches are data-efficient, i.e., converge an order of magnitude faster than state-of-the-art Deep Reinforcement Learning methods, and achieve optimal performance. We demonstrate the efficacy of these solutions in an experimental prototype using real traffic traces.

Future mobile networks need to support intelligent services which collect and process data streams at the network edge, so as to offer real-time and accurate inferences to users. However, the widespread deployment of these services is hindered by the unprecedented energy cost they induce to the network, and by the difficulties in optimizing their end-to-end operation. To address these challenges, we propose a Bayesian learning framework for jointly configuring the service and the Radio Access Network (RAN), aiming to minimize the total energy consumption while respecting accuracy and latency service requirements. Using a fully-fledged prototype with a software-defined base station (vBS) and a GPU-enabled edge server, we profile a typical video analytics service and identify new performance trade-offs and optimization opportunities. Accordingly, we tailor the proposed learning framework to account for the (possibly varying) network conditions, user needs, and service metrics, and apply it to a range of experiments with real traces. Our findings suggest that this approach effectively adapts to different hardware platforms and service requirements, and outperforms state-of-the-art benchmarks based on neural networks.

Bidding optimization is one of the most important problems in online advertising. Auto-bidding tools are designed to address this problem and are offered by most advertising platforms for advertisers to allocate their budgets. In this work, we present a Knowledge Graph-enriched Multi-Agent Reinforcement Learning Advertising Framework (KRAF). It combines Knowledge Graph (KG) techniques with a Multi-Agent Reinforcement Learning (MARL) algorithm for bidding optimization with the goal of maximizing advertisers' return on ad spend (ROAS) and user-ad interactions, which correlates to the ad platform revenue. In addition, this proposal is flexible enough to support different levels of user privacy and the advent of new advertising markets with more heterogeneous data. In contrast to most of the current advertising platforms that are based on click-through rate models using a fixed input format and rely on user tracking, KRAF integrates the heterogeneous available data (e.g., contextual features, interest-based attributes, information about ads) as graph nodes to generate their dense representation (embeddings). Then, our MARL algorithm leverages the embeddings of the entities to learn efficient budget allocation strategies. To that end, we propose a novel coordination strategy based on a mean-field style to coordinate the learning agents and avoid the curse of dimensionality when the number of agents grows. Our proposal is evaluated on three real-world datasets to assess its performance and the contribution of each of its components, outperforming several baseline methods in terms of ROAS and number of ad clicks.

The virtualization of radio access networks (vRAN) is the last milestone in the NFV revolution. However, the complex dependencies between computing and radio resources make vRAN resource control particularly daunting. We present vrAIn, a dynamic resource orchestrator for vRANs based on deep reinforcement learning. First, we use an autoencoder to project high-dimensional context data (traffic and channel quality patterns) into a latent representation. Then, we use a deep deterministic policy gradient (DDPG) algorithm based on an actor-critic neural network structure and a classifier to map contexts into resource control decisions. We have evaluated vrAIn experimentally, using an open-source LTE stack over different platforms, and via simulations over a production RAN. Our results show that: (i) vrAIn provides savings in computing capacity of up to 30% over CPU-agnostic methods; (ii) it improves the probability of meeting QoS targets by 25% over static policies; (iii) upon computing capacity under-provisioning, vrAIn improves throughput by 25% over state-of-the-art schemes; and (iv) it performs close to an optimal offline oracle. To our knowledge, this is the first work that thoroughly studies the computational behavior of vRANs and the first approach to a model-free solution that does not need to assume any particular platform or context.

Supporting Edge AI services is one of the most exciting features of future mobile networks. These services involve the collection and processing of voluminous data streams, right at the network edge, so as to offer real-time and accurate inferences to users. However, their widespread deployment is hampered by the energy cost they induce to the network. To overcome this obstacle, we propose a Bayesian learning framework for jointly configuring the service and the Radio Access Network (RAN), aiming to minimize the total energy consumption while respecting desirable accuracy and latency thresholds. Using a fully-fledged prototype with a software-defined base station (BS) and a GPU-enabled edge server, we profile a state-of-the-art video analytics AI service and identify new performance trade-offs. Accordingly, we tailor the optimization framework to account for the network context, the user needs, and the service metrics. The efficacy of our proposal is verified in a series of experiments and comparisons with neural network-based benchmarks.

Network virtualization is intended to be a key element of new generation networks. However, it is no clear how the implantation of this new paradigm will affect the power consumption of the network. To shed light on this relatively unexplored topic, we evaluate and analyze the power consumption of virtualized Base Station (vBS) experimentally. In particular, we measure the power consumption associated with uplink transmissions as a function of different variables such as traffic load, channel quality, modulation selection, and bandwidth. We find interesting tradeoffs between power savings and performance and propose two linear mixed-effect models to approximate the experimental data. These models allow us to understand the power behavior of the vBS and select power-efficient configurations. We release our experimental dataset hoping to foster further efforts in this research area.

Radio Access Network Virtualization (vRAN) will spearhead the quest towards supple radio stacks that adapt to heterogeneous infrastructure: from energy-constrained platforms deploying cells-on-wheels (e.g., drones) or battery-powered cells to green edge clouds. We perform an in-depth experimental analysis of the energy consumption of virtualized Base Stations (vBSs) and render two conclusions: (i) characterizing performance and power consumption is intricate as it depends on human behavior such as network load or user mobility; and (ii) there are many control policies and some of them have non-linear and monotonic relations with power and throughput. Driven by our experimental insights, we argue that machine learning holds the key for vBS control. We formulate two problems and two algorithms: (i) BP-vRAN, which uses Bayesian online learning to balance performance and energy consumption, and (ii) SBP-vRAN, which augments our Bayesian optimization approach with safe controls that maximize performance while respecting hard power constraints. We show that our approaches are data-efficient and have provably performance, which is paramount for carrier-grade vRANs. We demonstrate the convergence and flexibility of our approach and assess its performance using an experimental prototype.

Video analytics constitute a core component of many wireless services that require processing of voluminous data streams emanating from handheld devices. Multi-Access Edge Computing (MEC) is a promising solution for supporting such resource-hungry services, but there is a plethora of configuration parameters affecting their performance in an unknown and possibly time-varying fashion. To overcome this obstacle, we propose an Automated Machine Learning (AutoML) framework for jointly configuring the service and wireless network parameters, towards maximizing the analytics' accuracy subject to minimum frame rate constraints. Our experiments with a bespoke prototype reveal the volatile and system/data-dependent performance of the service, and motivate the development of a Bayesian online learning algorithm which optimizes on-the-fly the service performance. We prove that our solution is guaranteed to find a near-optimal configuration using safe exploration, i.e., without ever violating the set frame rate thresholds. We use our testbed to further evaluate this AutoML framework in a variety of scenarios, using real datasets.

Virtualized radio access networks (vRAN) are emerging as a key component of wireless cellular networks, and it is therefore imperative to optimize their architecture. vRANs are decentralized systems where the Base Station (BS) functions can be split between the edge Distributed Units (DUs) and Cloud computing Units (CUs); hence they have many degrees of design freedom. We propose a framework for optimizing the number and location of CUs, the function split for each BS, and the association and routing for each DU-CU pair. We combine a linearization technique with a cutting-planes method to expedite theexactproblem solution. The goal is to minimize the network costs and balance them with the criterion of centralization, i.e., the number of functions placed at CUs. Using data-driven simulations we find that multi-CU vRANs achieve cost savings up to 28% and improve centralization by 77%, compared to single-CU vRANs. Interestingly, we see non-trivial trade-offs among centralization and cost, which can be aligned or conflicting based on the traffic and network parameters. Our work sheds light on the vRAN design problem from a new angle, highlights the importance of deploying multiple CUs, and offers a rigorous optimization tool for balancing costs and performance.

Real-time object recognition is becoming an essential part of many emerging services, such as augmented reality, which require accurate inference in a timely fashion with low delay. We consider an edge-assisted object recognition system that can be configured in ways that have diverse impacts on these key performance criteria. Our goal is to design an online algorithm that learns the optimal configuration of the system by observing the outcomes of configurations applied in the past. We leverage the structure of the problem and combine a Gaussian process with a multi-armed bandit framework to efficiently solve the problem at hand. Our results indicate that our solution makes better configuration choices compared to other bandit algorithms, resulting in lower regret.

The virtualization of radio access networks (vRAN) is the last milestone in the NFV revolution. However, the complex dependencies between computing and radio resources make vRAN resource control particularly daunting. We present vrAIn, a dynamic resource controller for vRANs based on deep reinforcement learning. First, we use an autoencoder to project high-dimensional context data (traffic and signal quality patterns) into a latent representation. Then, we use a deep deterministic policy gradient (DDPG) algorithm based on an actor-critic neural network structure and a classifier to map (encoded) contexts into resource control decisions. We have implemented vrAIn using an open-source LTE stack over different platforms. Our results show that vrAIn successfully derives appropriate compute and radio control actions irrespective of the platform and context: (i) it provides savings in computational capacity of up to 30% over CPU-unaware methods; (ii) it improves the probability of meeting QoS targets by 25% over static allocation policies using similar CPU resources in average; (iii) upon CPU capacity shortage, it improves throughput performance by 25% over state-of-the-art schemes; and (iv) it performs close to optimal policies resulting from an offline oracle. To the best of our knowledge, this is the first work that thoroughly studies the computational behavior of vRANs, and the first approach to a model-free solution that does not need to assume any particular vRAN platform or system conditions.

While the application of the NFV paradigm into the network is proceeding full steam ahead, there is still one last mile- stone to be achieved in this context: the virtualization of the radio access network (vRAN). Due to the very complex de- pendency between the radio conditions and the computing resources needed to provide the baseband processing func- tionality, attaining an efficient resource control is particularly challenging. In this demonstration, we will showcase vrAIn, a vRAN dynamic resource controller that employs deep re- inforcement learning to perform resource assignment deci- sions. vrAIn, which is implemented using an open-source LTE stack over a Linux platform, can achieve substantial sav- ings in the used CPU resources while maintaining the target QoS for the attached terminals and maximize throughput when there is a deficit of computational capacity.

In heterogeneous cellular networks (HetNets), switching OFF small cells under low user traffic periods has been proved to be an effective energy saving strategy. However, this strategy has strong interactions with interference coordination (IC) mechanisms, making it convenient to address both tasks simultaneously. The motivation of this paper is to develop a self-optimization algorithm capable of jointly controlling energy saving and IC mechanisms using an online learning approach. Our proposal is based on a contextual bandit formulation that, among other challenges, implies discovering the most energy-efficient control actions while satisfying a predefined level of Quality of Service (QoS) for the users. We propose a two-level framework comprising a global controller, in charge of a group of macro cells, and multiple local controllers, one per macro cell. The global controller implements a novel algorithm, referred to as the Bayesian Response Estimation and Threshold Search (BRETS), that is capable of learning, for each control action, its feasibility boundaries in terms of QoS and its energy consumption as a function of the aggregated user traffic. The algorithm comes with a bound on its expected convergence time. The local controllers translate the control actions learned by the global controller into local decisions. Our numerical results show that BRETS is only 1% less efficient than an ideal oracle policy, clearly outperforming other benchmark algorithms.

This paper addresses the joint problem of energy saving and interference coordination in heterogeneous networks (HetNets) using a contextual bandit formulation. We propose a semi-distributed scheme consisting of a learning agent and local controllers. The learning agent comprises a neural network (NN) classifier and a Multi-Armed Bandit (MAB) algorithm. The NN classifier is dynamically trained to choose a subset of configurations (i.e., feasible configurations in terms of QoS) based on the context information (network state). Then, the MAB algorithm picks one control (i.e., global configuration parameters) among those selected by the NN classifier, with the aim of improving the energy efficiency. These global configurations are interpreted by the local controllers on each network sector. This scheme allows the learning agent to progressively learn the best policy by observing the network state and the performance of the chosen configurations in terms of energy consumption and QoS. Our numerical results show an energy saving close to 20% with respect to a default policy and an improvement of 13% with respect to addressing energy saving and interference coordination separately.

This paper focuses on interference coordination between the small cell and macro cell tiers of a wireless access network. We propose a new perspective based on a modelfree learning strategy, not requiring any previous knowledge about the network (e.g., topology, interference graph, scheduling algorithms). Our approach is based on a stochastic optimization algorithm known as Response Surface Methodology, that we use to learn the optimal parameter configuration during network operation (online learning) adapting to changes on network conditions (e.g., traffic, user positions). The result is a simple, effective and flexible mechanism that outperforms previous proposals. As a case study we apply our scheme to the dynamic adjustment of LTE-A eICIC parameters (CRE bias and ABS ratio).

This paper focuses on interference coordination between the small cell and macro cell tiers of a wireless access network. We present a self-optimization mechanism for LTE-A eICIC parameters (CRE bias and ABS ratio) following a novel approach based on a model-free learning strategy, not requiring any previous knowledge about the network (e.g., topology, interference graph, and scheduling algorithms). Our proposal is built upon a stochastic optimization algorithm known as response surface methodology (RSM), that we use to find efficient eICIC configurations during network operation (online learning), adapting to changing network conditions, such as traffic or user distribution. The objective consists of optimizing a performance metric for which, in general, mathematical expression is unavailable. In particular, we consider the fifth percentile throughput defined by the 3GPP. By means of RSM, our mechanism obtains local approximations of the objective function to perform steepest ascent iterations with an adjustable level of statistical accuracy. The algorithm can be extended to account for stochastic constraints, allowing the network to optimize one performance metric while maintaining other metrics above a desired level.

As cellular network technology evolves, the operators deploy new generation networks while maintaining their legacy networks, since not all users upgrade their terminals at the same pace. Therefore, the spectrum associated to these legacy networks becomes gradually underused. By means of cognitive radio techniques, the operator can allow its new generation terminals to reuse this spectrum. We propose a semi-decentralized scheme in which the operator guides the secondary access by broadcasting some operational parameters of the access strategy. The mechanism dynamically learns the optimal parameters by means of a response surface methodology (RSM), implying a very small signaling overhead. Our results show a notable capacity improvement compared to the classical approaches that exploit either spatial or temporal opportunities.