The coordinated deployment of multiple Base Stations (BS) and tuning of antenna configuration plays a crucial role in ensuring high-quality communication services, especially in the context of dense 5G BS deployment in megacities. However, traditional optimization methods, such as heuristics and Reinforcement Learning (RL), face challenges in addressing such problems involving the coordination of hundreds of BSs due to their limitations in handling the complexity and scale of large-scale scenarios. To address these challenges, this article proposes the Hierarchical Multi-Agent Proximal Policy Optimization with Representation Learning (HMAPPO-RL). By employing a hierarchical structure, we effectively decouple the optimization problem into two sub-problems: BS deployment and antenna parameter tuning. Different from the step-by-step method of optimizing the BS location and antenna, HMAPPO-RL achieves joint optimization of the two problems through an ingenious interactive mechanism, fully considering the mutual influence of the BS location and antenna. To address the large-scale challenge posed by hundreds of BSs, we utilize the upsampling and downsampling mechanisms of the UNet network to integrate global and local information from large-scale state information for performance enhancement. Since complex environmental information will cause great difficulties for the agent to evaluate the state value in large-scale scenarios, we add a representation learning module to enhance the accuracy of the agent’s state value estimation. The experiments using a precise mobile network simulator demonstrate the superiority of the proposed HMAPPO-RL, offering a comparative analysis with existing state-of-the-art methods. HMAPPO-RL achieves a coverage rate of 91.66% and an average throughput of 4,983,537 bit/s. These results represent improvements of 3.62% and 6.75% in coverage rate and throughput, respectively, when compared with the MAPPO algorithm.

To address the surge in wireless traffic, multiple-input multiple-output (MIMO) technology has become essential for advancing communication systems. Within MIMO cellular networks, coordinated beamforming (CBF), achieved through the collaborative design of beamformers across multiple base stations (BSs), presents a promising strategy for enhancing network performance. However, in large-scale multi-cell, multi-user MIMO environments, optimizing beamforming coordination remains challenging due to high-dimensional spaces and dynamic conditions. While centralized, optimization-based CBF algorithms provide near-optimal solutions, their dependence on real-time global channel state information (CSI) and high computational complexity renders them impractical for dynamic networks. To overcome these limitations, we introduce a diffusion model-enhanced multi-agent reinforcement learning (MARL) coordinated beamforming framework, termed Diffusion-enhanced MACBF, which enables BSs to determine optimal beamformers independently based on partial observations. Key innovations of this framework include: firstly, a novel limited-information exchange protocol that facilitates effective coordination among BSs with minimal communication overhead; secondly, the introduction of a diffusion model-enhanced Multi-agent Soft Actor-Critic (MASAC) algorithm that enables BSs to efficiently manage spatial and temporal variations in large-scale scenarios; and thirdly, an optimized network architecture combining an Encoder, Convolutional Neural Network (CNN), and Bidirectional Long Short-Term Memory (Bi-LSTM) layers to accurately map partial observations to optimal actions. Extensive simulations demonstrate the effectiveness and efficiency of the proposed algorithm, achieving an average rate of 7.5502 bps/Hz with up to 10.66% improvement over existing methods while significantly reducing information exchange requirements.

The rapid advancement of 5G networks has significantly increased the demand for high throughput and low latency, particularly for enhanced Mobile Broadband (eMBB) services. Optimizing user rates in large-scale base station (BS) collaborative scenarios is crucial, ensuring high throughput and low energy consumption while meeting the needs of users. The optimization process faces two key challenges: how to efficiently explore and utilize the vast action space for collaborative decision-making in large-scale networks with numerous BSs, and how to ensure optimal decision-making across numerous parameters while satisfying constraints. To tackle these challenges, we propose a novel solution that integrates a generative AI-powered Digital Twin platform with a multi-agent reinforcement learning (MARL) decision algorithm. An important tool is the high-fidelity DT-SimNet platform, enabling seamless integration with AI optimizers. Moreover, we propose a novel algorithm called Safe Hierarchical Multi-Agent Proximal Policy Optimization with Adaptive Prediction mechanism (Safe HMAPPO-AP). This algorithm integrates Safe MARL to ensure decision-making remains within constraint boundaries and incorporates a hierarchical structure to decompose the action space. Moreover, it leverages Graph Neural Networks (GNNs) for efficient feature extraction and employs the Adaptive Prediction mechanism to accelerate learning and improve network performance. Comprehensive experiments using the DT-SimNet platform demonstrate the superior performance of the proposed algorithm compared to competing methods. Within the target area, Safe HMAPPO-AP achieved a 6.73% improvement in average throughput for regular users and an 11.48% increase in system utility, while meeting demands of newly added eMBB users.

With the rapid advancement of communication technology and the exponential growth of mobile users, improving network coverage quality and throughput has become increasingly important. In particular, large-scale Base Station (BS) cooperative optimization has become a highly significant topic. BSs can adjust various parameters for high-quality communication, but automating this optimization remains challenging due to environmental sensitivity and interdependencies. Traditional methods for network optimization are constrained by the intricate nature of real-world environments. Further, Reinforcement Learning (RL) techniques, which are effective for configuration policies, encounter difficulties in intricate, high-dimensional wireless communication networks, especially in multi-agent cooperative optimization. To overcome these challenges, this article proposes the Enhanced Multi-Agent Proximal Policy Optimization (EMAPPO), which utilizes the capabilities of the UNet network to extract multi-spatial relationships among a massive number of network elements and employs the DiffPool network to efficiently depict the impact of large-scale action coordination among massive agents on coverage performance. To facilitate evaluation in communication optimization, we further introduce a high-fidelity digital twin-driven mobile network. Extensive experiments validate the effectiveness and superior performance of EMAPPO by utilizing the network digital twin. The results demonstrate significant improvements in signal coverage rate and network throughput compared to the competing methods.

With the exponential growth of mobile users, ensuring high-quality network coverage has become paramount. Large-scale mobile networks consist of numerous base stations (BSs), each with adjustable parameters such as angles and beam widths. Automatically optimizing network coverage can be difficult due to environmental factors and the interdependence of the adjustable parameters. Due to the inherent uncertainties and unpredictable nature of large-scale wireless networks, traditional methods such as heuristics and meta-heuristics lack the adaptability and scalability required to cope with their dynamic environment. To address these challenges, we propose utilizing digital twin and reinforcement learning (RL) techniques within mobile networks characterized by multiple collaborating agents. We initially introduce DT-SimNet, a digital twin-enabled mobile network simulator to facilitate optimization evaluation. DT-SimNet can efficiently simulate communication behaviors of network elements within a complex environment while revealing user mobility patterns. Moreover, to address challenges arising from multifaceted relationships among users, BSs, and the parameters across BSs, we introduce an innovative strategy named Optimized Multi-Agent Proximal Policy Optimization with Self-supervised Prediction (OMAPPO-SSP). Compared to MAPPO, which leads to limited applicability and inferior performance due to the dynamic characteristics of 5G networks, this approach leverages network structure optimization and a self-supervised prediction mechanism, employing multi-agent reinforcement learning (MARL) principles to enhance efficiency. By harnessing collaborative neural networks, OMAPPO-SSP facilitates the explicit learning of behavioral interactions among all BSs, enabling effective decision-making in environments characterized by intricate spatial relationships, dynamic user behaviors, and diverse interactions. Extensive experiments are conducted to validate the efficiency and effectiveness of the OMAPPO-SSP. Within the target area, OMAPPO-SSP achieves a coverage ratio of 94.66% and an average throughput of 89746 bits per second (bps), demonstrating significant improvements compared to competing methods.

Digital Twin brings a new realization approach to the modeling of mobile networks. Mobile networks, as complex systems comprising multiple components, such as mobile users, base stations, and wireless environments, have intricate interactions and relationships with each other. By creating a virtual replica of each physical mobile network entity in a virtual space, we build a scalable digital twin system for mobile networks with generative AI. The system can interact with multiple optimizers to evaluate and display real-time simulation results. A companion video can be accessed using the link below. https://youtu.be/xtcBIXPzvkc

With the development of 5G and the Internet, the explosion of new mobile applications has led to an increasing number of computation-intensive and latency-sensitive tasks, which poses a severe challenge to resource-limited vehicles. Mobile Edge Computing (MEC) is recognized as an encouraging paradigm for providing offloading services from vehicles to the edge of wireless access networks. Computation offloading is the key technology in MEC that determines which tasks should be offloaded, for achieving the minimal time and energy consumption. However, in highway scenarios, statically deployed lightweight edge servers cannot meet the high resource requirements of vehicles with dependencies among subtasks. To alleviate this issue, this paper explores the utility of vehicles with idle resources as vehicular MEC servers. Considering the problem of dependency-aware task offloading, we utilize the directed acyclic graph (DAG) to analyze the task topology. Furthermore, due to privacy preservation, a distributed deep reinforcement learning-based algorithm with an optimized structure for offloading strategy is proposed. Convolutional neural networks and transformers are employed in the structure to extract rich state information efficiently. Experimental results reveal that, the proposed scheme achieves superior performance compared to the benchmark algorithms.