Multi-beam low earth orbit (LEO) satellites, with their wide coverage, high communication rates, low latency, and flexibility, are essential components in the 5G and 6G eras. However, challenges such as uneven ground user distribution, multi-dimensional resource coupling, inter-beam interference, and dynamic network topology complicate efficient resource management. This paper presents a joint beam hopping and resource allocation (BHRA) scheme within a Digital Twin (DT)empowered multi-beam LEO satellites network. The complex resource management problem is divided into two sub-problems: traffic-satellite allocation and joint multi-satellite BHRA. First, cell traffic is allocated among satellites to balance load using deep reinforcement learning (DRL). Then, Hierarchical Proximal Policy Optimization (HPPO) optimizes multi-satellite beam hopping and resource allocation to meet demand. Extensive simulation results demonstrate that the proposed method reduces satellite load imbalance by approximately 83%, achieves the highest throughput compared to other algorithms, and generalizes well as traffic demand increases.

Low-Earth orbit (LEO) satellites utilizing beam hopping (BH) technology offer extensive coverage, low latency, high bandwidth, and significant flexibility. However, the uneven geographical distribution and temporal variability of ground traffic demands, combined with the high mobility of LEO satellites, present significant challenges for efficient beam resource utilization. Traditional BH methods based on GEO satellites fail to address issues such as satellite interference, overlapping coverage, and mobility. This paper explores a Digital Twin (DT)-based collaborative resource allocation network for multiple LEO satellites with overlapping coverage areas. A two-tier optimization problem, focusing on load balancing and cell service fairness, is proposed to maximize throughput and minimize inter-cell service delay. The DT layer optimizes the allocation of overlapping coverage cells by designing BH patterns for each satellite, while the LEO layer optimizes power allocation for each selected service cell. At the DT layer, an Actor-Critic network is deployed on each agent, with a global critic network in the cloud center. The A3C algorithm is employed to optimize the DT layer. Concurrently, the LEO layer optimization is performed using a Multi-Agent Reinforcement Learning algorithm, where each beam functions as an independent agent. The simulation results show that this method reduces satellite load disparity by about 72.5% and decreases the average delay to 12ms. Additionally, our approach outperforms other benchmarks in terms of throughput, ensuring a better alignment between offered and requested data.

As the Low Earth Orbit (LEO) satellite has extremely high moving speed and limited networking resources, designing dynamic routing has become a promising approach to improve satellite communication performance. Due to the hundreds of satellites within a constellation and the complex attributes of each satellite, traditional routing strategies based on centralized paradigm derivation face increasingly complex challenges. To address these issues, this paper jointly optimizes queuing delay and propagation delay by proposing a fully distributed routing algorithm based on deep reinforcement learning. Each satellite builds a partially observable Markov decision process (POMDP) model based on the spatial location and queue length of surrounding nodes and adaptively selects the next hop by calculating the estimated residual propagation delay between the neighboring satellites and the destination satellite. Simulation analysis shows that our proposed method has tremendous advantages and effectiveness.

Video services such as short video sharing have exploded due to the rapid development of Internet social media platforms. Caching video segments on satellites effectively shortens service delay and speeds up video sharing, especially for users without terrestrial Internet access. However, where to place what video and how to replace it in time is by no means an easy task, requiring careful consideration of many factors, e.g., satellite coverage, video popularity, and limited caching resource. In this paper, we propose a coverage-aware cooperative video caching algorithm (CACVC) that considers the prevalence of video in the coverage area and the collaboration between adjacent satellites. In CACVC, we model the cache placement problem of video as a Partially Observable Markov Decision Process (POMDP) to optimize the service delay of video provided by access satellites, neighboring satellites, or ground stations. We derive the optimal cache strategy by utilizing Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm with a centralized training and distributed execution paradigm. Simulation results show that the cache hit ratio can be improved by 4%~18%, and the average service delay can be reduced by 1%~14%.

Low Earth Orbit (LEO) satellite constellations are promising to provide global coverage and low latency communication by deploying a large number of small satellites and widely establishing Inter-Satellite Links (ISLs). However, due to the high motion of the LEO satellites, fixed inter-plane ISLs cannot provide long-time continuous connectivities and guarantee high-throughput communication performance. The existing dynamic planning approaches almost only consider part of the constellation information and cannot derive the optimal inter-plane ISLs. This paper proposes a dynamic Inter-plane Inter-satellite Links Planning method based on Multi-Agent deep reinforcement learning (MA-IILP) to optimize the total throughput and inter-plane ISL switching rate. We formulate a Partially Observable Markov Decision Process (POMDP) model with taking into account the Euclidean distance, communication rate and link switching cost. We derive the optimal strategy by utilizing Multi-Agent Deep Deterministic Policy Gradient (MADDPG) algorithm with a centralized training and distributed execution paradigm. Finally, extensive experiments are carried out and the results illustrate that our proposed approach can increase the total throughput of the target constellation by 2.8%∼7.2%, and decrease the inter-plane ISL switching rate by 30.7%∼68.4% compared to the state-of-the-art baseline algorithms.