The rapid development of low Earth orbit (LEO) satellite constellations has enabled global broadband coverage and low-latency communication, but also brings new challenges to connection continuity and network stability. Due to the ultra-dense deployment and high orbital velocity of LEO satellites, users frequently experience mandatory handovers and are prone to triggering “ping-pong” effects in overlapping coverage areas. To address these issues, we propose a hierarchical reinforcement learning (HRL)-based inter-satellite handover framework that decouples decision-making into two subproblems: when to initiate a handover at the temporal level and which satellite to handover to at the spatial level. To achieve long-term planning, the temporal agent utilizes the proximal policy optimization (PPO) algorithm to determine the handover timing by predicting future satellite load and interference levels. For short-term adaptability, the spatial agent employs the deep Q-network (DQN) to select the target satellite based on a utility function incorporating satellite load, signal quality, and service duration. We implement the proposed method in a Starlink constellation environment. Simulation results show that the proposed HRL-based scheme outperforms traditional and non-hierarchical RL methods in terms of user handover frequency and handover success rate. Furthermore, we validate that the proposed approach significantly improves transmission rate and load balancing, demonstrating its effectiveness in highly dynamic LEO satellite environments.

With the continuous advancement of onboard digital payload technology, high-throughput satellites (HTSs) now possess the capability to dynamically reconfigure their onboard resources in response to user traffic demands. This flexibility is primarily enabled by the efficient design of the satellite’s frequency plan and the beam power allocation scheme. However, the joint optimization of frequency planning and power allocation is a typical mixed-integer nonlinear programming (MINLP) problem, which can not be directly solved using traditional optimization methods. To address this issue, we propose a two-stage optimization framework, aimed at bridging the gap between the beam capacity and the user traffic request, while simultaneously avoiding co-channel interference among different beams. In the first stage, we propose a multi-agent deep reinforcement learning framework based on proximal policy optimization. By modeling each beam as an agent, each beam can select its reuse group, starting frequency, bandwidth, and transmit power based on the current system state. To mitigate the impact of neural network randomness on the results, we further introduce a second-stage optimization algorithm using the difference of convex (DC) programming to refine the beam power allocation. Simulation results demonstrate that the proposed algorithm significantly reduces execution time compared to benchmark algorithms, such as genetic algorithms and particle swarm optimization, while also demonstrating considerable performance improvements.