The open radio access network (O-RAN) architecture supports intelligent network control algorithms as one of its core capabilities. Data-driven applications incorporate such algorithms to optimize radio access network (RAN) functions via RAN intelligent controllers (RICs). Deep reinforcement learning (DRL) algorithms are among the main approaches adopted in the O-RAN literature to solve dynamic radio resource management problems. However, despite the benefits introduced by the O-RAN RICs, the practical adoption of DRL algorithms in real network deployments falls behind. This is primarily due to the slow convergence and unstable performance exhibited by DRL agents upon deployment and when encountering previously unseen network conditions. In this paper, we address these challenges by proposing transfer learning (TL) as a core component of the training and deployment workflows for the DRL-based closed-loop control of O-RAN functionalities. To this end, we propose and design a hybrid TL-aided approach that leverages the advantages of both policy reuse and distillation TL methods to provide safe and accelerated convergence in DRL-based O-RAN slicing. We conduct a thorough experiment that accommodates multiple services, including real VR gaming traffic to reflect practical scenarios of O-RAN slicing. We also propose and implement policy reuse and distillation-aided DRL and non-TL-aided DRL as three separate baselines. The proposed hybrid approach shows at least: 7.7% and 20.7% improvements in the average initial reward value and the percentage of converged scenarios, and a 64.6% decrease in reward variance while maintaining fast convergence and enhancing the generalizability compared with the baselines.

The success of immersive applications such as virtual reality (VR) gaming and metaverse services depends on low latency and reliable connectivity. To provide seamless user experiences, the open radio access network (O-RAN) architecture and 6G networks are expected to play a crucial role. RAN slicing, a critical component of the O-RAN paradigm, enables network resources to be allocated based on the needs of immersive services, creating multiple virtual networks on a single physical infrastructure. In the O-RAN literature, deep reinforcement learning (DRL) algorithms are commonly used to optimize resource allocation. However, the practical adoption of DRL in live deployments has been sluggish. This is primarily due to the slow convergence and performance instabilities suffered by the DRL agents both upon initial deployment and when there are significant changes in network conditions. In this paper, we investigate the impact of time series forecasting of traffic demands on the convergence of the DRL-based slicing agents. For that, we conduct an exhaustive experiment that supports multiple services including real VR gaming traffic. We then propose a novel forecasting-aided DRL approach and its respective O-RAN practical deployment workflow to enhance DRL convergence. Our approach shows up to 22.8%, 86.3%, and 300% improvements in the average initial reward value, convergence rate, and number of converged scenarios respectively, enhancing the generalizability of the DRL agents compared with the implemented baselines. The results also indicate that our approach is robust against forecasting errors and that forecasting models do not have to be ideal.

Reinforcement Learning (RL) algorithms have recently been proposed to solve dynamic radio resource management (RRM) problems in beyond 5G networks. However, RL-based solutions are still not widely adopted in commercial cellular networks. One of the primary reasons for this is the slow convergence of RL agents when they are deployed in a live network and when the network’s context changes significantly. Concurrently, the open radio access network (O-RAN) paradigm promises to give mobile network operators (MNOs) more control over their networks, furthering the need for intelligent and RL-based network management. O-RAN’s standardized interfaces will allow MNOs to make real-time custom changes to intelligently control various RRM functionalities. We consider a RAN slicing scenario in which MNOs can modify the weights of the RL reward function. This enables MNOs to change the priorities of fulfilling the service level agreements of the slices. However, this results in a practical challenge since the RL agent needs to adapt promptly to the changes made by the MNO. This challenge is addressed in this paper, where we first present and discuss the results from an exhaustive experiment to examine the efficiency of using transfer learning (TL) to accelerate the convergence of RL-based RAN slicing in the considered scenario. We then propose a novelpredictiveapproach to enhance the TL-based acceleration by selecting the best-saved policy for reuse. By adopting the proposed policy transfer approach, RL agents are able to converge up to 14000 learning steps faster than their non-accelerated counterparts. The proposed machine learning (ML)-basedpredictiveapproach also shows up to a 96.5% accuracy in selecting the best expert policy to reuse for acceleration.

The ability to accurately forecast network resource utilization is vital in next-generation wireless networks. Based on the predicted load, telecom operators can proactively allocate network resources in an efficient way. In this paper, we perform a thorough analysis of a cellular network downlink load dataset collected at millisecond resolution. We first evaluate various statistical metrics of the physical resource block (PRB) utilization data to investigate its predictability. Then, we develop deep learning-based models to forecast PRB utilization in radio access networks (RANs). In particular, we propose univariate and multivariate long short-term memory (LSTM) network-based architectures for the forecasting task and investigate the impact of various prediction horizons and history lengths. When predicting PRB utilization, our approach showed up to 49% improvement in the Coefficient of Determination (r2score) and 19.5% decrease in the Root Mean Square Error (RMSE) compared with the baseline methods used.

Deep Reinforcement Learning (DRL) algorithms have been recently proposed to solve dynamic Radio Resource Management (RRM) problems in 5G networks. However, the slow convergence experienced by traditional DRL agents puts many doubts on their practical adoption in cellular networks. In this paper, we first discuss the need to have accelerated DRL algorithms. We then analyze the exploration behavior of various state-of-the-art DRL algorithms for slice resource allocation, and compare it with the traditional 5G Radio Access Network (RAN) slicing baselines. Finally, we propose a transfer learning-accelerated DRL-based solution for slice resource allocation. In particular, we tackle the challenge of slow convergence by transferring the policy learned by a DRL agent at an expert base station (BS) to newly deployed agents at target learner BSs. Our approach shows a remarkable reduction in convergence time and a significant performance improvement compared with its non-accelerated counterparts when tested against multiple traffic load variations.

With the advancements in wireless network technologies over the past few decades and the deployment of 4G LTE networks, the capabilities and services provided to end-users have become seemingly endless. Users of smartphones utilize high-speed network services while commuting on public transit and hope to have a consistent, high-quality connection for the duration of their trip. Due to the massive load demand on cellular networks and frequent changes in the underlying radio channel, users often experience sudden unexpected variations in the connection quality. To overcome such variations and maintain a consistent connection, these variations need to be predicted before they occur. This can be accomplished by the spatio-temporal analysis of the different network quality parameters and the investigation of the main factors that affect the network's performance and QoS. To this end, we conducted a network survey via Kingston Transit in Kingston, Ontario, Canada. We used the Android network monitoring application G-NetTrack Pro to build a dataset of various client-side wireless network quality parameters. The dataset consists of 30 repeated public transit bus trips at three different times of the day, each lasting around one hour. In this paper, we describe the data collection process, present an analysis of the collected data, and investigate the effects of time and location on the network's measured throughput and signal strength. We made the collected data, including more than 190 thousand unique records, publicly available to researchers in a domain where open data is rare.