Deep reinforcement learning (DRL) is a powerful method for Dynamic Power Allocation (DPA) due to its adaptability to changing wireless environments. Crafting proficient DRL agents necessitates a well-structured problem representation, learning scheme, and agent interaction framework. Additionally, the environment, as the second pivotal component in DRL, must mirror realistic channel dynamics faithfully. This research tackles these often-neglected aspects. We introduce Het-DRL, a heterogeneous multi-agent DRL framework with a competitive interaction paradigm for DPA, moving away from conventional collaborative and centralized learning schemes. The independent learning scheme eliminates information exchange among agents, empowering autonomous decision-making. Moreover, we present our CELLUCARLA wireless simulator, an extension of the CARLA (Car Learning to Act) simulator with a cellular network layer, enabling realistic urban mobility modeling. We leverage CELLUCARLA to evaluate the proposed Het-DRL, highlighting its effectiveness for DPA in Orthogonal Frequency-Division Multiplexing (OFDM) systems, with up to 153% sum rate relative to the Weighted Minimum Mean Square Error (WMMSE) algorithm.

Dynamic Power Allocation (DPA) in wireless networks has been tackled utilizing model-driven optimization, reinforcement learning, and deep learning methods. However, these methods still face some limitations, including dependence on task-specific designs and limited adaptability to diverse network configurations. In this paper, we optimize three adaptation variants of a pre-trained 3.82-billion-parameter Large Language Model (LLM) for DPA. These strategies include LLM with Prompt Engineering (LLM-PE), which employs DPA-specific prompt engineering with detailed network and channel information; LLM with Retrieval Augmented Generation (LLM-RAG), which enhances prompts with retrieved optimal power allocation examples; and LLM with Fine-tuning (LLM-FT), which fine-tunes the model on a small DPA-specific dataset. Our results show that LLM-FT excels in adaptability across various network configurations and resilience to channel estimation errors, maintaining a sum rate performance above 99% compared to the optimal water-filling algorithm, using only 5000 fine-tuning examples.

Despite recent advancements in millimeter-wave (mm Wave) communication systems, the practical validation of research ideas in this field is still challenging. This is mostly due to the simplifying assumptions in their system modeling and overlooking hardware limitations in the simulation environment. In this paper, we propose a high-speed and programmable mm Wave testbed with multi-user (MU) multiple-input multiple-output (MIMO) beamforming capability. The baseband signal processing is preformed fully on a general-purpose processor. The radio frequency (RF) frontend holds a hybrid MIMO architecture, where each RF chain is connected to one phased-array antenna. The testbed allows to plug and play different physical layer signal processing algorithms or radio resource allocation techniques, including machine learning models, to assess their performance under real-life conditions.

Deep reinforcement learning has emerged as a pow-erful tool for dynamic power allocation, as it allows continuous learning and realtime responsiveness to environmental changes through its core element, experience replay. Experience replay involves two critical hyperparameters: the replay buffer and mini-batch sizes. While state-of-the-art solutions primarily concentrate on designing input features and reward-shaping methods, the impact of experience replay parameters on system performance has often been overlooked. This paper aims to address this gap by exploring the effects of experience replay parameters in the context of dynamic power allocation in multi-carrier wireless systems. To address the power allocation problem, we propose a multi-agent cooperative deep reinforcement learning framework. The results show that a minimum of 2000 experiences in the replay buffer is necessary for the proposed solution to outperform conventional approaches. Moreover, many obsolete experiences within a larger replay buffer slightly decrease system performance. Interestingly, the increase in batch size does not significantly affect the learning models' training time due to parallel execution, yet, it improves performance.

Multi-agent deep reinforcement learning has pre-dominantly been applied to tackle the challenges of power allocation within complex and dynamic wireless networks. These intelligent agents rely on exploration as a fundamental tool to gather crucial information about their environment, reducing uncertainty and facilitating more informed power allocation decisions. Importantly, as a wireless channel is inherently dynamic, exploration is an indispensable mechanism for agents to continually adapt by acquiring fresh environment insights. Traditionally, most research has leaned towards employing semi-uniform distributed exploration for discrete action spaces and noise perturbation exploration for continuous action spaces. In this paper, we set out to question the validity of this default choice. We modify both exploration methods to work in both action spaces and with different exploration speeds. We demonstrate that semi-uniform distributed exploration yields comparable results to noise perturbation exploration in a continuous action space while significantly simplifying the exploration factor's fine-tuning. In contrast, noise perturbation exploration excels primarily in continuous action space.

Despite the success of Deep Reinforcement Learning (DRL) in radio-resource management within multi-cell wireless networks, applying it to power allocation in ultra-dense 5G and beyond networks poses challenges. While existing multi-agent DRL-based methods often adopt a fully centralized approach, they often overlook communication overhead costs. In this paper, we model a multi-cell network as a collaborative multi-agent DRL system, implementing a centralized training-decentralized execution approach for accurate and real-time decision-making, thereby eliminating communication overhead during execution. We carefully design the DRL agents’ input observations, actions, and rewards to address potential impractical power allocation policies in multi-carrier systems and ensure strict compliance with transmit power constraints. Through extensive simulations, we assess the sensitivity of the proposed DRL-based power allocation to various exploration methods and system parameters. Results indicate superior performance of DRL-based power allocation with continuous action space in complex network environments. Conversely, simpler network settings with fewer subcarriers and users require fewer power allocation actions, ensuring rapid convergence. By leveraging a fast exploration rate, DRL-based power allocation with discrete action space outperforms conventional algorithms, achieving a 36% relative sum rate increase within 60,000 training episodes.