Satellite networks have long been regarded as a vital component of space communication systems, which provide integrated satellite-terrestrial broadband access in seamless coverage and cost-effective manner. The inter-satellite routing design for low earth orbit (LEO) satellite constellations is critical for achieving low-latency and high-reliability communication in the space communication systems. However, the inherent dynamic nature of LEO satellites, coupled with the variability in inter-satellite connectivity, imposes significant challenges for routing efficiency and network dependability. Existing routing schemes cannot handle such topological fluctuations due to their insensitivity to real-time network changes, thus suffering from performance degradations in highly dynamic space environments. This paper presents Iris, an intelligent reliable routing scheme for inter-satellite communication, aiming at increasing efficiency and reliability of the packet transmission process. Specifically, we propose a comprehensive deep reinforcement learning (DRL) framework that learns a policy to select routing paths automatically under the emerging software-defined satellite networking (SDSN) architecture. To strengthen fault-tolerance in fluctuating environments, we train an agent in an incremental manner by gradually increasing scenario complexity. Simulation results indicate that our solution significantly outperforms baselines and exhibits advances in adaptability and reliability, especially under dynamic environments with frequent topology changes.

Recent efforts have explored adaptive routing via deep reinforcement learning (DRL) techniques without handcrafted parameter engineering. Intrinsically, routing decision-making is essentially a process used to find a subgraph in a graph-structured network. However, previous works seldom took topological relationships into consideration when providing adaptive routing algorithms, causing them to suffer from suboptimal routes in dynamic network environments involving both varying traffic loads and burst traffic. In this paper, we presentGrace, a novel graph embedding-based Deep Reinforcement Learning framework tailored for distributed routing algorithm optimization within the Software-Defined Networking (SDN) paradigm. Specifically,Graceleverages graph embedding to translate graph-structured entities into low-dimensional vectors, thereby enabling multiple DRL agents to learn optimal routing paths under dynamic network environments. Unfortunately, training multiple agents encounters inherent challenges in complicated and dynamic network scenarios. In response, we design an adaptive incremental training method forGracethat makes the model adapt to task complexity in a gradual manner, while speeding up its retraining efforts when environments change. To further accelerate convergence, we integrate intrinsic curiosity intoGraceto tackle large environments with sparse rewards. Extensive experiments conducted on two real-world topologies demonstrate the rationality and effectiveness ofGrace, and the results show throughput improvements of up to 40.1% compared to other state-of-the-art DRL routing algorithms under bursty traffic conditions.

The emerging space-terrestrial integrated networks (STIN) are envisioned to provide seamless connectivity for global coverage. As a promising key component for STIN, the Low Earth Orbiting (LEO) satellite network still faces significant technological challenges in routing for ensuring both reachability and efficiency, due to frequent topology changes and uneven load distribution. Load-balancing routing is a feasible solution to improve efficiency by alleviating regional overload, however, it may encounter either slow convergences or local perceptions. In this paper, we propose a Neighbor Load Rank (NLR) based load-balancing routing for LEO satellite networks, where potential congestion at the queue buffer of a satellite node is characterized by load scores of its neighboring satellites, so as to reduce the perspective limitation of local load-balancing routing. To accelerate routing convergence and reduce computational complexity, we design region delineation, boundary penalty and directional incentive strategies to obtain the approximate minimum hop count path. Meanwhile, we employ the topology-stabilizing model (TSM) to convert the frequent satellite-ground interconnection changes into traffic fluctuations. Simulations demonstrate that NLR can maintain low-latency capacity with lower transmission overhead and effectively balance the overloaded traffic.

The low earth orbit (LEO) satellite constellation is regarded as an effective complement to the terrestrial communication system due to its seamless coverage and ultra-low latency. Unfortunately, the highly dynamic traffic volume, as well as the inherent nature of dynamic topology changes caused by frequent link handover and uncertain hardware failures, pose severe challenges in the design of reliable routing. However, most existing reliable routing approaches with distributed schemes only focus on information exchange between adjacent nodes, which makes them fail to perceive real-time global network changes and make optimal decisions. In this paper, we propose a software defined networking (SDN) based intelligent satellite routing (SISR) method to increase the adaptivity and reliability during the packet transmission process. With the facilitation of SDN, we manage the network in a hierarchical and centralized paradigm, and further implement a more refined form of reinforcement learning (RL) to enhance the fault-tolerant ability of satellite network routing. Experimental results show that our solution can reduce latency and packet loss ratio by more than 42% and 29% compared to baselines.

Forwarding path selection for data traffic is one of the most fundamental operations in computer networks, whose performance drastically impacts both transmission efficiency and reliability in network domains. Although deep reinforcement learning (DRL) has attracted considerable attention for path selection instead of hand-tuned heuristics, few works have considered how to exploit graph-structured information in networks to improve routing and forwarding efficiency. In fact, generating routes is essentially a process for finding a subgraph in a graph-structured network. To this end, this paper proposes an effective and novel graph embedding-based DRL framework for adaptive path selection (termed GRL-PS), aiming at reducing end-to-end (E2E) latency and promoting network throughput while maintaining stability in dynamically changing environments. Specifically, graph representation learning (GRL) is deployed as an effective enabler for the DRL agent to learn the relational knowledge of interacting entities for route decisions in networks. However, training such an agent in a dynamically changing environment encounters a knowledge acquisition bottleneck, since the DRL agent is always forced to learn every task from scratch. To improve the adaptation of behaviors and acquire skills beyond what the source policy can teach, we introduce potential-based reward shaping as a means of knowledge transfer to guide the agent in unfamiliar conditions with sparse rewards. Experimental results show that compared with baseline methods, our solution can achieve nearly-optimal performance with both latency and throughput, especially in large-scale dynamic networks.