Low Earth orbit satellite networks (LSNs) are envisioned as key enablers of 6 G by offering ubiquitous, low-latency connectivity. Their mesh topology enables multipath differential routing, which improves bandwidth utilization and reduces transmission delay. However, the growing demand for data and the dynamic, self-organizing nature of LSNs pose significant challenges for joint multipath routing and traffic scheduling under strict latency and energy constraints. To address these challenges, this paper proposes a multipath routing optimization (MRO) and traffic scheduling method tailored for multipath differential routing. Specifically, a dynamic multi-attribute graph model is developed to precisely capture the dynamic properties of LSNs. Building on this model, a MRO algorithm, integrated with a Stackelberg game framework, is introduced. The MRO algorithm employs a decomposition-based approach to identify multiple optimal paths that minimize delay and energy consumption, while the Stackelberg game framework ensures efficient traffic distribution across these paths. Numerical results demonstrate that the proposed approach significantly outperforms existing baseline methods, achieving cumulative reward improvements of 26.77% to 43.8% across four real-world network topologies and exhibiting better Pareto front coverage. Furthermore, by leveraging the rapid convergence properties of the Stackelberg game model, the proposed method enhances network throughput by 12% to 43% and reduces transmission time by 14% to 49%.

In this article, we consider the design of dynamic transmission scheduling policies for the industrial network systems sharing scarce communication resources. Only a few subsystems can obtain channel access for information updates to close their control loops at each time step, which highlights the necessity of designing optimal transmission scheduling schemes to achieve a minimum average linear quadratic cost of the industrial network systems. We first propose a greedy state-error-dependent scheduling (SES) policy based on the one-step expected profit and discuss its stability employing the Lyapunov function method. After formulating the scheduling optimization as a Markov decision process problem and relaxing with a soft constraint, we develop a heuristic near-optimal solution that guarantees the optimality of certainty equivalent controllers, namely, Whittle’s index-inspired error-dependent scheduling (WIES). A stochastic stability condition of WIES is further given based on f-ergodicity. Due to low computational complexity and ease of implementation, the proposed schemes are suitable for large-scale heterogeneous industrial network systems. Finally, simulation results show that the proposed policies outperform the existing round-robin, holding-time-prioritized, and error-aware scheduling schemes.

This article studies the dynamic transmission-scheduling problem of rate-limited networked control systems with multiple loops. It is critical to develop an optimal-scheduling policy because, due to limited spectrum resources and energy budgets, only partial subsystems may have access to shared channels at each time step to update the plant states affected by stochastic disturbances. An adaptive event-triggered stochastic scheduling policy is proposed based on the one-step-ahead comparison error of state estimation between the sensor and controller. By formulating the scheduling problem as a constrained multiagent partially observable Markov decision process, a multiagent reinforcement learning algorithm is proposed to search for the optimal parameters of the scheduling policy. And an edge-assisted learning architecture is introduced to facilitate its implementation. An explicit performance index of the optimal scheduling is further obtained, revealing the effects of system disturbances, observation noises, and the communication rate constraint. Moreover, two numerical examples are given to validate the proposed scheduling policy, showing that it outperforms several competitive scheduling schemes. Due to the low computational complexity, the proposed scheduling may have an advantage in large-scale heterogeneous control systems, e.g., the flow and pressure control in fluid transport pipelines.

In this article, we investigate the routing problem of packet networks through multiagent reinforcement learning (RL), which is a very challenging topic in distributed and autonomous networked systems. In specific, the routing problem is modeled as a networked multiagent partially observable Markov decision process (MDP). Since the MDP of a network node is not only affected by its neighboring nodes' policies but also the network traffic demand, it becomes a multitask learning problem. Inspired by recent success of RL and metalearning, we propose two novel model-free multiagent RL algorithms, named multiagent proximal policy optimization (MAPPO) and multiagent metaproximal policy optimization (meta-MAPPO), to optimize the network performances under fixed and time-varying traffic demand, respectively. A practicable distributed implementation framework is designed based on the separability of exploration and exploitation in training MAPPO. Compared with the existing routing optimization policies, our simulation results demonstrate the excellent performances of the proposed algorithms.