Existing learning-based congestion control methods suffer from myopic decision-making due to their reliance on single-timestep states and fail to model long-term dependencies due to architectural constraints (e.g., recurrent networks’ vanishing gradients). To address these issues, we propose a Decision Transformer-based network congestion control framework named DTCC. DTCC is the first to unify long-context modeling and real-time decision-making within a 4-layer autoregressive Transformer, replacing traditional Markov decision paradigms with sequence-to-action mapping. With enhancement learning strategy such as stochasticity-aware training, DTCC achieves efficient and generalizable performance from heterogeneous dataset. Extensive experiments demonstrate DTCC’s supremacy: it achieves 16.67–29.55% higher winning rate compared to state-of-the-art baselines (e.g., Sage) across diverse network scenarios and 8.33%–29.17% higher winning rate under unseen highly variable network. Leveraging a lightweight Transformer, DTCC enables real-time deployment with approximately 2.8 ms inference per step on general CPU devices. To the best of our knowledge, this is the first work to employ Decision Transformer for training an intelligent congestion control mechanism. Our work, therefore, showcases the potential of combining reinforcement learning with advanced Transformer architectures in real-time network control.

The multipath extension of the Quick UDP Internet Connection (QUIC) protocol, also called MPQUIC, is currently attracting increasing attention from both industry and academia. The multipath scheduler of MPQUIC determines how to distribute the packets onto different paths. However, our experimental results show that they fail to adapt to various receive buffer sizes and Quality of Service (QoS) requirements while applying current multipath schedulers into MPQUIC due to the diversity of devices and applications. These problems are especially severe under heterogeneous and dynamic network environments. To tackle these problems, we propose MARS, a Multi-Agent deep Reinforcement learning (MADRL) based Multipath QUIC Scheduler, which is able to promptly adapt to dynamic network environments. It exploits the MADRL method to learn a neural network for each path and generate scheduling policy. Besides, it introduces a novel multi-objective reward function that takes out-of-order (OFO) queue size and different QoS metrics into consideration to realize adaptive scheduling optimization. We implement MARS in an MPQUIC prototype and compare it with the state-of-the-art multipath schedulers in both emulated and real-world networks. Experimental results show that MARS outperforms the other schedulers with better adaptive capability regarding the receive buffer sizes and QoS.

Multipath TCP (MPTCP) enables multi-home devices to establish multiple paths for simultaneous data transmission. However, due to diverse Quality of Service (QoS) requirements in real network, existing multipath congestion control algorithms (CCAs) fail to fast adapt to dynamic traffic, which leads to performance degradation, especially in heterogeneous network environments. To tackle these problems, in this paper, we first observe the performance limitations of current multipath CCAs by conducting extensive experiments. Then we propose ACCeSS, an adaptive QoS-aware multipath congestion control framework, which is able to promptly adapt to network changes and QoS requirements with a novel control policy optimization phase. In order to adjust and stimulate improvement of the preferred performance metric, ACCeSS exploits Random Forest Regressing (RFR) method to perform QoS-specific utility function optimization. ACCeSS is implemented and compared with other multipath CCAs in Linux kernel. Performances of ACCeSS are evaluated in both emulated and real-world networks, which reveal that ACCeSS outperforms classic multipath CCAs and the state-of-the-art learning based multipath CCA with better adaptive capability of QoS.