Link adaptation in 5G wireless networks faces fundamental challenges due to the stochastic nature of the radio environment. Deep reinforcement learning (DRL) has shown potential for automating adaptive control, but it often suffers from instability when user mobility, fading, and interference introduce significant variability in link performance. This inconsistency in action outcomes, particularly when adjusting the downlink target block error rate (BLER), complicates the learning process and slows convergence. To address this, we propose a probabilistic representation learning framework in which the variance of each user’s latent state is conditioned on the selected BLER. This structure enables the DRL agent to reason about action-dependent uncertainty and learn more robust, risk-aware policies. We implement this representation within DRL agents and evaluate the approach using a commercial-grade 5G system-level simulator based on ns-3. Experimental results show that the proposed method improves downlink throughput by up to 19.0% over fixed-rule baselines, achieves up to 11.3% gain over heuristic eOLLA algorithms, and reduces convergence time by 46.2%, consistently demonstrating superior performance under diverse mobility scenarios.

Digital twins are expected to play an important role in the widespread adaptation of AI-based networking solutions in the real world. The calibration of these virtual replicas is critical to ensure a trustworthy replication of the real environment. This work focuses on the input parameter calibration of radio access network (RAN) simulators using real network performance metrics as supervision signals. Usually, the RAN digital twin is considered a black-box function and each calibration problem is viewed as a standalone search problem. RAN simulators are slow and non-differentiable, often posing as the bottleneck in the execution time for these search problems. In this work, we aim to accelerate the search process by reducing the number of interactions with the simulator by leveraging RAN interactions from previous problems. We present a sequential Bayesian optimization framework that uses information from the past to warm-start the calibration process. Assuming that the network performance exhibits gradual and periodic changes, the stored information can be reused in future calibrations. We test our method across multiple physical sites over one week and show that using the proposed framework, we can obtain better calibration with a smaller number of interactions with the simulator during the search phase.

With the global aim of reducing carbon emissions, energy saving for communication systems has gained tremendous attention. Efficient energy-saving solutions are not only required to accommodate the fast growth in communication demand but solutions are also challenged by the complex nature of the load dynamics. Recent reinforcement learning (RL)-based methods have shown promising performance for network optimization problems, such as base station energy saving. However, a major limitation of these methods is the requirement of online exploration of potential solutions using a high-fidelity simulator or the need to perform exploration in a real-world environment. We circumvent this issue by proposing an offline reinforcement learning energy saving (ORES) framework that allows us to learn an efficient control policy using previously collected data. We first deploy a behavior energy-saving policy on base stations and generate a set of interaction experiences. Then, using a robust deep offline reinforcement learning algorithm, we learn an energy-saving control policy based on the collected experiences. Results from experiments conducted on a diverse collection of communication scenarios with different behavior policies showcase the effectiveness of the proposed energy-saving algorithms.

As technologies for 4G and 5G networks become increasingly complex, user equipment's (UE) energy consumption also increased significantly. The 4G and 5G systems employ connected-mode discontinuous reception (C-DRX) to save UEs energy consumption by intermittently suspending network connections. However, optimizing the C-DRX operation is complex that requires considering various network conditions, including traffic patterns of each UE and the scheduling algorithms of a base station (BS). In this paper, we introduce a novel approach, a Multi-Agent Deep Reinforcement Learning (MADRL) algorithm with an integrated self-attention mechanism. This approach empowers the model to optimize C-DRX parameters independent of BSs' scheduling algorithms, enhancing its adaptability to dynamically changing network conditions. Simulation results illustrate that our MADRL-based C-DRX optimization algorithm consistently outperforms traditional methods under varying network conditions, affirming its robustness and efficacy for real- world network optimization.

With the increasing use of data-intensive mobile applications and the number of mobile users, the demand for wireless data services has been increasing exponentially in recent years. In order to address this demand, a large number of new cellular base stations are being deployed around the world, leading to a significant increase in energy consumption and greenhouse gas emission. Consequently, energy consumption has emerged as a key concern in the fifth-generation (5G) network era and beyond. Reinforcement learning (RL), which aims to learn a control policy via interacting with the environment, has been shown to be effective in addressing network optimization problems. However, for reinforcement learning, especially deep reinforcement learning, a large number of interactions with the environment are required. This often limits its applicability in the real world. In this work, to better deal with dynamic traffic scenarios and improve real-world applicability, we propose a transfer deep reinforcement learning framework for energy optimization in cellular communication networks. Specifically, we first pre-train a set of RL-based energy-saving policies on source base stations and then transfer the most suitable policy to the given target base station in an unsupervised learning manner. Experimental results demonstrate that base station energy consumption can be reduced significantly using this approach.

The association of mobile devices with network resources (e.g., base stations, frequency bands/channels), known as load balancing, is critical to reduce communication traffic congestion and network performance. Reinforcement learning (RL) has shown to be effective for communication load balancing and achieves better performance than currently used rule-based methods, especially when the traffic load changes quickly. However, RL-based methods usually need to interact with the environment for a large number of time steps to learn an effective policy and can be difficult to tune. In this work, we aim to improve the data efficiency of RL-based solutions to make them more suitable and applicable for real-world applications. Specifically, we propose a simple, yet efficient and effective deep RL-based wireless network load balancing framework. In this solution, a set of good initialization values for control actions are selected with some cost-efficient approach to center the training of the RL agent. Then, a deep RL-based agent is trained to find offsets from the initialization values that optimize the load balancing problem. Experimental evaluation on a set of dynamic traffic scenarios demonstrates the effectiveness and efficiency of the proposed method.

Communication load balancing aims to balance the load between different available resources, and thus improve the quality of service for network systems. After formulating the load balancing (LB) as a Markov decision process problem, reinforcement learning (RL) has recently proven effective in addressing the LB problem. To leverage the benefits of classical RL for load balancing, however, we need an explicit reward definition. Engineering this reward function is challenging, because it involves the need for expert knowledge and there lacks a general consensus on the form of an optimal reward function. In this work, we tackle the communication load balancing problem from an inverse reinforcement learning (IRL) approach. To the best of our knowledge, this is the first time IRL has been successfully applied in the field of communication load balancing. Specifically, first, we infer a reward function from a set of demonstrations, and then learn a reinforcement learning load balancing policy with the inferred reward function. Compared to classical RL-based solution, the proposed solution can be more general and more suitable for real-world scenarios. Experimental evaluations implemented on different simulated traffic scenarios have shown our method to be effective and better than other baselines by a considerable margin.

With the continuous growth in communication network complexity and traffic volume, communication load balancing solutions are receiving increasing attention. Specifically, reinforcement learning (RL)-based methods have shown impressive performance compared with traditional rule-based methods. However, standard RL methods generally require an enormous amount of data to train, and generalize poorly to scenarios that are not encountered during training. We propose a policy reuse framework in which a policy selector chooses the most suitable pre-trained RL policy to execute based on the current traffic condition. Our method hinges on a policy bank composed of policies trained on a diverse set of traffic scenarios. When deploying to an unknown traffic scenario, we select a policy from the policy bank based on the similarity between the previous-day traffic of the current scenario and the traffic observed during training. Experiments demonstrate that this framework can outperform classical and adaptive rule-based methods by a large margin.

Digital twins have shown a great potential in supporting the development of wireless networks. They are virtual representations of 5G/6G systems enabling the design of machine learning and optimization-based techniques. Field data replication is one of the critical aspects of building a simulation-based twin, where the objective is to calibrate the simulation to match field performance measurements. Since wireless networks involve a variety of key performance indicators (KPIs), the replication process becomes a multi-objective optimization problem in which the purpose is to minimize the error between the simulated and field data KPIs. Unlike previous works, we focus on designing a data-driven search method to calibrate the simulator and achieve accurate and reliable reproduction of field performance. This work proposes a search-based algorithm based on mixed-variable particle swarm optimization (PSO) to find the optimal simulation parameters. Furthermore, we extend this solution to account for potential conflicts between the KPIs using a-fairness concept to adjust the importance attributed to each KPI during the search. Experiments on field data showcase the effectiveness of our approach to (i) improve the accuracy of the replication, (ii) enhance the fairness between the different KPIs, and (iii) guarantee faster convergence compared to other methods.

Accurate traffic volume estimation and prediction are essential for advanced communication network functions, such as automatic operations and predictive resource allocation. Although machine learning (ML)-based approaches achieve great success in accomplishing this goal, existing approaches suffer from two drawbacks that limit their real-world applications. First, the ML-based prediction models developed in the past might be obsolete now, since the communication traffic patterns and volumes keep changing in the real world, leading to prediction errors. Second, most Base Stations (BSs) can only save a small amount of data due to the limited storage capacity and high storage costs, which prevents from training an accurate prediction model. In this paper, we propose a novel framework that adapts the prediction model to the constantly changing traffic with only a few current traffic data. Specifically, the framework first learns the knowledge of historical traffic data as much as possible by using a proposed two-branch neural network design, which includes a prediction and a reconstruction module. Then, the framework transfers the knowledge from an old (past) prediction model to a new (current) model for the model update by using a proposed continual knowledge distillation technique. Evaluations on a real-world dataset show that the proposed framework reduces the Mean Absolute Error (MAE) of traffic prediction by up to 9.62% compared to the state-of-the-art prediction methods.

Due to the rapid increase in wireless communication traffic in recent years, load balancing is becoming increasingly important for ensuring the quality of service. However, variations in traffic patterns near different serving base stations make this task challenging. On one hand, crafting a single control policy that performs well across all base station sectors is often difficult. On the other hand, maintaining separate controllers for every sector introduces overhead, and leads to redundancy if some of the sectors experience similar traffic patterns. In this paper, we propose to construct a concise set of controllers that cover a wide range of traffic scenarios, allowing the operator to select a suitable controller for each sector based on local traffic conditions. To construct these controllers, we present a method that clusters similar scenarios and learns a general control policy for each cluster. We use deep reinforcement learning (RL) to first train separate control policies on diverse traffic scenarios, and then incrementally merge together similar RL policies via knowledge distillation. Experimental results show that our concise policy set reduces redundancy with very minor performance degradation compared to policies trained separately on each traffic scenario. Our method also outperforms handcrafted control parameters, joint learning on all tasks, and two popular clustering methods.

Mobile edge computing (MEC) networks have been recently adopted to accommodate the fast-growing number of mobile devices performing complicated tasks with limited hardware capability. Recently, edge nodes with communication, computation, and caching capacities are starting to be deployed in MEC networks. Due to the physical separation of these resources, efficient coordination and scheduling are important for efficient resource utilization and optimal network performance. In this paper, we study mobility load balancing for communication, computation, and caching-enabled heterogeneous MEC networks. Specifically, we propose to tackle this problem via a multi-agent deep reinforcement learning-based framework. Users served by overloaded edge nodes are handed over to less loaded ones, to minimize the load in the most loaded base station in the network. In this framework, the handover decision for each user is made based on the user’s own observation which comprises the user’s task at hand and the load status of the MEC network. Simulation results show that our proposed multi-agent deep reinforcement learning-based approach can reduce the time-average maximum load by up to 30% and the end-to-end delay by 50% compared to baseline algorithms.

Load balancing has become a key technique to handle the increasing traffic demand and improve the user experience. It evenly distributes the traffic across network resources by offloading users from overloaded base stations or channels to less crowded ones. Load balancing is a multi-objective optimization problem involving the automatic adjustment of several parameters to simultaneously maximize multiple network performance indicators. However, the existing methods mostly rely on single-objective approaches which lead to sub-optimal solutions. In this paper, we introduce the first multi-objective reinforcement learning (MORL) framework for load balancing. Specifically, we propose a solution based on meta-reinforcement learning (meta-RL) to learn a general policy capable of quickly adapting to new trade-offs between the objectives. We further enhance the generalization of our proposed solution using policy distillation techniques. To showcase the effectiveness of our framework, experiments are conducted based on real-world traffic scenarios. Our results show that our load balancing framework can (i) significantly outperform the existing rule-based and single-objective solutions, (ii) compute better Pareto front approximations compared to MORL baselines, and (iii) quickly adapt to new objective trade-offs.

Prediction of key system characteristics, such as the communication load, is required to overcome the delays in wireless communication systems. State-of-The-Art (SOTA) approaches mostly apply existing Neural Network (NN) structures, and extract latent features purely based on their sensitivity to the forecasting accuracy. This way of feature extraction may neglect some non-obvious yet informative dimensions in the model input, leading to inaccurate forecasting results. In this paper, we present an Adaptive Feature Boosting (AFB) approach, which integrates multiple AutoEncoders (AEs) to automatically extract robust and comprehensive latent features for communication load forecasting. The recurrent and residual connections among the AEs make sure that the extracted latent features are representative for all input dimensions. With more comprehensive information extracted from the history, the forecasting accuracy is thus improved. We evaluate AFB against existing approaches on a real-world dataset that contains Call Detail Records (CDRs) of the Milan city over a period of two months. The evaluation shows that our AFB-based approach achieves 35.2% more accurate load forecasting results than the SOTA deep approaches.

The new 802.11ac standard aims at achieving Gbps data throughput for individual users by exploiting enhanced physical-layer features, such as higher modulation levels, Multiple Input Multiple Output (MIMO), and wider bandwidths. However, the heterogeneity of bandwidth in a network can cause asymmetric interferences in which certain transmissions cannot be sensed by some other nodes. As a result, the conventional Carrier Sense Multiple Access with Collision Avoidance (CSMA/CA) may not work well in 802.11ac networks. We call this the Hidden Channel (HC) problem, which is shown to be real via experiments with USRP and WARP boards. To solve this problem, we propose bandwidth adaptation based on post-CCA, which is a clear channel assessment (CCA) procedure performed aftercompleting a transmission. Post-CCA in wireless networks helps mimic the CSMA with Collision Detection (CSMA/CD) mechanism in the wired Ethernet, thus enhancing channel assessment capability. Using post-CCA, we propose Post-CCA based Bandwidth Adaptation (PoBA) that alters bandwidth and channel configuration dynamically by applying a reinforcement learning mechanism. Post-CCA and PoBA do not require any hardware modification and are also compliant with the 802.11 standards. PoBA is shown via simulation to increase network-wide throughput, channel utilization and fairness, and also lower packet error probability.

Mobile gadgets including smartphones and laptops nowadays support both Wi-Fi and Bluetooth together. The two protocols coexist in the 2.4GHz ISM band while sharing a single antenna in order to meet constraints on form-factor. We call this kind of devices "Dual- stack devices" in this paper. Since Wi-Fi and Bluetooth on dual-stack devices work in TDM (Time Division Multiplexing) manner, the performance of Wi- Fi, specifically throughput, degrades inevitably. We propose a novel scheme, Opportunistic Antenna Utilization (OAU), which aims to improve the throughput of Wi-Fi by utilizing time assigned to yet not used for a Bluetooth streaming service. We first present measurement results showing the degradation of Wi-Fi throughput on dual-stack devices followed by a simple analysis to anticipate the expected gain of OAU. Then we verify our analysis by showing that simulation results coincide well with the analysis.

The major goal of IEEE 802.11ac is to provide very high throughput (VHT) performance while at the same time guaranteeing backward compatibility. To achieve this goal, 802.11ac adopts the channel bonding technique that makes use of multiple 20 MHz channels in 5 GHz band. Due to the heterogeneity of bandwidth that each device exploits, and the fixed total transmission power in the standards, a problem called `Hidden Channel' arises. In this paper, we first analyze the problem and show how the contention parameters and transmission time affect collision probability and fairness in some deployment scenarios. Then, we propose a heuristic channel allocation algorithm that aims to avoid such problematic situations effectively. Through simulations, we demonstrate that our proposed channel allocation algorithm lowers the packet error rate (PER) compared to uncoordinated and received signal strength indicator(RSSI) based allocation schemes and increases the network-wide throughput as well as the throughput of a station that experiences poor performance. This implies improved fairness performance among transmission pairs with various channel bandwidths.

The goal of IEEE 802.11ac is to provide very high throughput~(VHT) performance while at the same time guaranteeing backward compatibility. For the goal, 802.11ac adopts the channel bonding technique that makes use of multiple 20MHz channels in 5GHz band. Due to the heterogeneity of bandwidth that each device exploits, and the fixed total transmission power, a problem called `Hidden Channel' arises. In this paper, we propose a heuristic channel allocation algorithm that aims to avoid such problematic situations. Through simulations, we demonstrate that our proposed channel allocation algorithm lowers the packet error rate~(PER) compared to uncoordinated and RSSI~(Received Signal Strength Indicator) based allocation schemes and increases the throughput of a station that experiences poor performance.

As the number of wireless service providers increases, competition among them is becoming stronger in wireless access networks. On the other hand, users actively change their behaviors toward the networks to get more network resources such as service time, bandwidth, capacity, etc. That is, each user will actively choose a cell or a network that offers the largest amount of resources with the lowest cost. In these environments, service providers have to consider not only technical factors but also economical factors such as revenue and user price. By controlling the pricing policy, a service provider can recruit or refuse users that are trying to associate with. In this paper, we first model the resource purchasing and pricing game scheme that takes not only revenue of a service provider but also user satisfaction into account. Assuming selfish behaviors, solution is derived using game theoretic approach. The solution produces the integrated purchasing and pricing scheme that shows cell breathing effect. We extend the model to multicell environments where a user has freedom to choose its service provider. As a user actively changes its weight of the utility function and chooses a cell to associate with, overall performance can be improved. We demonstrate the effect of load balancing with the pricing policy, and the performance improvement compared to a conventional method of association via simulation.