Deep reinforcement learning (DRL) is a powerful method for Dynamic Power Allocation (DPA) due to its adaptability to changing wireless environments. Crafting proficient DRL agents necessitates a well-structured problem representation, learning scheme, and agent interaction framework. Additionally, the environment, as the second pivotal component in DRL, must mirror realistic channel dynamics faithfully. This research tackles these often-neglected aspects. We introduce Het-DRL, a heterogeneous multi-agent DRL framework with a competitive interaction paradigm for DPA, moving away from conventional collaborative and centralized learning schemes. The independent learning scheme eliminates information exchange among agents, empowering autonomous decision-making. Moreover, we present our CELLUCARLA wireless simulator, an extension of the CARLA (Car Learning to Act) simulator with a cellular network layer, enabling realistic urban mobility modeling. We leverage CELLUCARLA to evaluate the proposed Het-DRL, highlighting its effectiveness for DPA in Orthogonal Frequency-Division Multiplexing (OFDM) systems, with up to 153% sum rate relative to the Weighted Minimum Mean Square Error (WMMSE) algorithm.

Dynamic Power Allocation (DPA) in wireless networks has been tackled utilizing model-driven optimization, reinforcement learning, and deep learning methods. However, these methods still face some limitations, including dependence on task-specific designs and limited adaptability to diverse network configurations. In this paper, we optimize three adaptation variants of a pre-trained 3.82-billion-parameter Large Language Model (LLM) for DPA. These strategies include LLM with Prompt Engineering (LLM-PE), which employs DPA-specific prompt engineering with detailed network and channel information; LLM with Retrieval Augmented Generation (LLM-RAG), which enhances prompts with retrieved optimal power allocation examples; and LLM with Fine-tuning (LLM-FT), which fine-tunes the model on a small DPA-specific dataset. Our results show that LLM-FT excels in adaptability across various network configurations and resilience to channel estimation errors, maintaining a sum rate performance above 99% compared to the optimal water-filling algorithm, using only 5000 fine-tuning examples.

In modern phased-array systems, limited resolution of digital attenuators and phase shifters can degrade beamforming accuracy, increase interference, and disrupt beam tracking, impacting overall performance. While existing methods like rounding, dithering, and optimization-based approaches have addressed phase quantization, most have overlooked amplitude errors and are often limited to specific configurations, such as linear arrays. This paper introduces MISOCP-QBeam, a unified optimization framework for joint amplitude and phase quantization, applicable to both 1D and 2D arrays. Using a Mixed-Integer Second-Order Cone Program (MISOCP), the method provides deterministic, hardware-compliant solutions that enhance main-lobe accuracy and suppress sidelobes. Numerical results demonstrate significant improvements, with up to 5.8x lower pointing error and 3.4 dB reduction in sidelobe level compared to traditional rounding methods.

Deep reinforcement learning has emerged as a pow-erful tool for dynamic power allocation, as it allows continuous learning and realtime responsiveness to environmental changes through its core element, experience replay. Experience replay involves two critical hyperparameters: the replay buffer and mini-batch sizes. While state-of-the-art solutions primarily concentrate on designing input features and reward-shaping methods, the impact of experience replay parameters on system performance has often been overlooked. This paper aims to address this gap by exploring the effects of experience replay parameters in the context of dynamic power allocation in multi-carrier wireless systems. To address the power allocation problem, we propose a multi-agent cooperative deep reinforcement learning framework. The results show that a minimum of 2000 experiences in the replay buffer is necessary for the proposed solution to outperform conventional approaches. Moreover, many obsolete experiences within a larger replay buffer slightly decrease system performance. Interestingly, the increase in batch size does not significantly affect the learning models' training time due to parallel execution, yet, it improves performance.

Multi-agent deep reinforcement learning has pre-dominantly been applied to tackle the challenges of power allocation within complex and dynamic wireless networks. These intelligent agents rely on exploration as a fundamental tool to gather crucial information about their environment, reducing uncertainty and facilitating more informed power allocation decisions. Importantly, as a wireless channel is inherently dynamic, exploration is an indispensable mechanism for agents to continually adapt by acquiring fresh environment insights. Traditionally, most research has leaned towards employing semi-uniform distributed exploration for discrete action spaces and noise perturbation exploration for continuous action spaces. In this paper, we set out to question the validity of this default choice. We modify both exploration methods to work in both action spaces and with different exploration speeds. We demonstrate that semi-uniform distributed exploration yields comparable results to noise perturbation exploration in a continuous action space while significantly simplifying the exploration factor's fine-tuning. In contrast, noise perturbation exploration excels primarily in continuous action space.

Despite the success of Deep Reinforcement Learning (DRL) in radio-resource management within multi-cell wireless networks, applying it to power allocation in ultra-dense 5G and beyond networks poses challenges. While existing multi-agent DRL-based methods often adopt a fully centralized approach, they often overlook communication overhead costs. In this paper, we model a multi-cell network as a collaborative multi-agent DRL system, implementing a centralized training-decentralized execution approach for accurate and real-time decision-making, thereby eliminating communication overhead during execution. We carefully design the DRL agents’ input observations, actions, and rewards to address potential impractical power allocation policies in multi-carrier systems and ensure strict compliance with transmit power constraints. Through extensive simulations, we assess the sensitivity of the proposed DRL-based power allocation to various exploration methods and system parameters. Results indicate superior performance of DRL-based power allocation with continuous action space in complex network environments. Conversely, simpler network settings with fewer subcarriers and users require fewer power allocation actions, ensuring rapid convergence. By leveraging a fast exploration rate, DRL-based power allocation with discrete action space outperforms conventional algorithms, achieving a 36% relative sum rate increase within 60,000 training episodes.

Integrating distributed machine learning tools with sensing, communication, and decision-making operations in networked and cooperative intelligent machines has opened up novel avenues for research. The cross-fertilization of these components is essential for enabling collaborative task management that requires safety, reliability, scalability, and low latency. Data privacy preservation and high spectral efficiency are required for various Industrial Internet of Things (IIoT) applications. Most previous works have focused on joint optimization of task-offloading and resource allocation while neglecting the data privacy and data overhead of the decision-making process in task-offloading. Considering the robos as agents, this paper proposes Prioritized Spectral efficient Federated Reinforcement Learning (PSFRL)-based partial task-offloading in a heterogeneous industrial robotic Mobile Edge Computing (MEC) system. Due to its definition, PSFRL keeps data private while achieving high spectrum efficiency. Additionally, we define a Value of Information (VoI) metric so that the PSFRL agents allocate their resources more wisely for more valuable data. As robots have limited battery and computing resources for task processing, multiple edge computing-enabled access points assist the robots in accomplishing tasks. Simulation results show that our proposed PSFRL outperforms other learning-based task-offloading methods concerning energy consumption, spectral efficiency, and VoI. Moreover, we demonstrate the superiority of the proposed cooperative approach over its non-cooperative counterpart.

Despite the substantial success of deep learning for Automatic Modulation Classification (AMC), models trained on a specific transmitter configuration and channel model often fail to generalize well to other scenarios with different transmitter configurations, wireless fading channels, or receiver impairments such as clock offset. This paper proposes Contrastive Learning with Self-Reconstruction called CLSR-AMC to learn good representations of signals resilient to channel changes. While contrastive loss focuses on the differences between individual modulations, the reconstruction loss captures representative features of the signal. Additionally, we develop three data augmentation operators to emulate the impact of channel and hardware impairments without exhaustive modeling of different channel profiles. We perform extensive experimentation with commonly used realistic datasets. We show that CLSR-AMC outperforms its counterpart based on contrastive learning for the same amount of labeled data by significant average accuracy gains of 24.29%, 17.01%, and 15.97% in the Additive White Gaussian Noise (AWGN), Rayleigh, and Rician channels, respectively.

Automatic Modulation Classification (AMC) is significant for the practical support of a plethora of emerging spectrum applications, such as Dynamic Spectrum Access (DSA) in 5G and beyond, resource allocation, jammer identification, intruder detection, and in general, automated interference analysis. Although a well-known problem, most of the existing AMC work has been done under the assumption that the classifier has prior knowledge about the signal and channel parameters. This paper shows that unknown signal and channel parameters significantly degrade the performance of two of the most popular research streams in modulation classification: expert feature-based and data-driven. By understanding why and where those methods fail, in such unknown scenarios, we propose two possible directions to make AMC more robust to signal shape transformations introduced by unknown signal and channel parameters. We show that Spatial Transformer Networks (STN) and Transfer Learning (TL) embedded into a light ResNeXt-based classifier can improve average classification accuracy up to 10-30% for specific unseen scenarios with only 5% labeled data for a large dataset of 20 complex higher-order modulations.

Given the availability of lightweight radio and processing technology, it becomes feasible to imagine spectrum sensing systems using weather balloons. Such balloons navigate the airspace up to 40 km, and can provide a bird's eye and clear view of terrestrial, as well as aerial spectrum use. In this paper, we present SkySense, which is an extension of the Electrosense sensing framework with mobile GPS-located sensors and local data logging. In addition, we present 6 different sensing campaigns, targeting multiple terrestrial or aerial technologies such as ADS-B, AIS or LTE. For instance, for ADS-B, we can clearly conclude that the number of airplanes that are detected is the same for each balloon altitude, but the message reception rate decreases strongly with altitude because of collisions. For each sensing campaign, the dataset is described, and some example spectrum analysis results are presented. In addition, we analyse and quantify important trends visible when sensing from the sky, such as temperature and hardware variations, increased ambient interference levels, as well as hardware limitations of the lightweight system. A key challenge is the automatic gain control and dynamic range of the system, as a radio navigating over 30km, sees a very wide range of possible signal levels. All data is publicly available through the Electrosense framework, to encourage the spectrum sensing community to further analyse the data or motivate further measurement campaigns using weather balloons.