Advanced wireless networks must be highly dynamic and capable of managing heterogeneous service demands. A key feature of these networks is network slicing, which is supported in the next-generation radio access network (RAN) architecture, such as open RAN (O-RAN), by leveraging artificial intelligence (AI) and machine learning (ML) approaches within the ran intelligent controller (RIC) modules. Deep reinforcement learning (DRL) has much potential for managing dynamic networks but often needs to improve when dealing with unstructured, multi-modal data like RF signals and QoS metrics. This kind of data makes it challenging for DRL to grasp the high-level context fully. To tackle this, we're using large language models (LLMs) to give DRL a richer, more meaningful state representation. LLMs add semantic layers to raw data, helping the agent grasp deeper context and think more strategically over the long term. This way, the DRL agent can make smarter decisions even as the environment becomes more complex and changes over time. This paper proposes a task-related state representation that employs LLM to augment multi-agent DRL (MARL) approaches. Simulation results demonstrate that the proposed approach significantly outperforms related baselines.

As wireless networks grow to support more complex applications, the Open Radio Access Network (O-RAN) architecture, with its smart RAN Intelligent Controller (RIC) modules, becomes a crucial solution for real-time network data collection, analysis, and dynamic management of network resources including radio resource blocks and downlink power allocation. Utilizing artificial intelligence (AI) and machine learning (ML), O-RAN addresses the variable demands of modern networks with unprecedented efficiency and adaptability. Despite progress in using ML-based strategies for network optimization, challenges remain, particularly in the dynamic allocation of resources in unpredictable environments. This paper proposes a novel Meta Deep Reinforcement Learning (Meta-DRL) strategy, inspired by Model-Agnostic Meta-Learning (MAML), to advance resource block and downlink power allocation in O-RAN. Our approach leverages O-RAN's disaggregated architecture with virtual distributed units (DUs) and meta-DRL strategies, enabling adaptive and localized decision-making that significantly enhances network efficiency. By integrating meta-learning, our system quickly adapts to new network conditions, optimizing resource allocation in real-time. This results in a 19.8% improvement in network management performance over traditional methods, advancing the capabilities of next-generation wireless networks.

This paper investigates an interference-aware joint path planning and power allocation mechanism for a cellular-connected unmanned aerial vehicle (UAV) in a sparse suburban environment. The UAV’s goal is to fly from an initial point and reach a destination point by moving along the cells to guarantee the required quality of service (QoS). In particular, the UAV aims to maximize its uplink throughput and minimize interference to the ground user equipment (UEs) connected to neighboring cellular base stations (BSs), considering both the shortest path and limitations on flight resources. Expert knowledge is used to experience the scenario and define the desired behavior for the sake of the agent (i.e., UAV) training. To solve the problem, an apprenticeship learning method is utilized via inverse reinforcement learning (IRL) based on both Q-learning and deep reinforcement learning (DRL). The performance of this method is compared to learning from a demonstration technique called behavioral cloning (BC) using a supervised learning approach . Simulation and numerical results show that the proposed approach can achieve expert-level performance. We also demonstrate that, unlike the BC technique, the performance of our proposed approach does not degrade in unseen situations.

As emerging networks such as Open Radio Access Networks (O-RAN) and 5G continue to grow, the demand for various services with different requirements is increasing. Network slicing has emerged as a potential solution to address the different service requirements. However, managing network slices while maintaining quality of services (QoS) in dynamic environments is a challenging task. Utilizing machine learning (ML) approaches for optimal control of dynamic networks can enhance network performance by preventing Service Level Agreement (SLA) violations. This is critical for dependable decision-making and satisfying the needs of emerging networks. Although RL-based control methods are effective for real-time monitoring and controlling network QoS, generalization is necessary to improve decision-making reliability. This paper introduces an innovative attention-based deep RL (ADRL) technique that leverages the O-RAN disaggregated modules and distributed agent cooperation to achieve better performance through effective information extraction and implementing generalization. The proposed method introduces a value-attention network between distributed agents to enable reliable and optimal decision-making. Simulation results demonstrate significant improvements in network performance compared to other DRL baseline methods.

Collaborative deep reinforcement learning (CDRL) algorithms in which multiple agents can coordinate over a wireless network is a promising approach to enable future intelligent and autonomous systems that rely on real-time decision making in complex dynamic environments. Nonetheless, in practical scenarios, CDRL face many challenges due to heterogeneity of agents and their learning tasks, different environments, time constraints of the learning, and resource limitations of wireless networks. To address these challenges, in this paper, a novel semantic-aware CDRL method is proposed to enable a group of heterogeneous untrained agents with semantically-linked DRL tasks to collaborate efficiently across a resource-constrained wireless cellular network. To this end, a new heterogeneous federated DRL (HFDRL) algorithm is proposed to select the best subset of semantically relevant DRL agents for collaboration. The proposed approach then jointly optimizes the training loss and wireless bandwidth allocation for the cooperating selected agents in order to train each agent within the time limitation of its real-time task. Simulation results show the superior performance of the proposed algorithm compared to state-of-the-art baselines.

Future wireless cellular networks must support both enhanced mobile broadband (eMBB) and ultra-reliable low-latency communications (URLLC) to manage heterogeneous data traffic for emerging wireless services. To achieve this goal, a promising technique is to enable flexible frame structure by dynamically changing the data frame's numerology according to the channel information as well as traffic quality-of-service requirements. However, due to non-orthogonal subcarriers, this technique can result in an interference, known as inter numerology interference (INI), thus, degrading the network performance. In this work, a novel framework is proposed to analyze the INI in the uplink cellular communications. In particular, a closed-form expression is derived for the INI power in the uplink with a flexible frame structure and a new resource allocation problem is formulated to maximize the network spectral efficiency (SE) by jointly optimizing the power allocation and numerology selection in a multi-user uplink scenario. The simulation results validate the derived theoretical INI analyses and provide guidelines for the power allocation and numerology selection.

This letter focuses on adaptive target detection in the presence of multiple interference sources, which comprise clutter, thermal noise, noise-like jammers, and fully-correlated (or coherent) signals. In order to account for different operating scenarios, we formulate the problem at hand in terms of a multiple hypothesis test with several alternative hypotheses representative of each considered scenario. In this context, we devise a family of two-stage detection architectures capable of classifying the specific scenario and, hence, of working under different operating conditions. The performance analysis shows the effectiveness of the detector based upon the Generalized Information Criterion also in comparison with traditional adaptive decision schemes.