Video streaming is one of the most popular Internet applications that makes up a large amount of Internet traffic. A fundamental mechanism in video streaming is adaptive bitrate (ABR) selection which decides the proper compression level for each chunk of a video to optimize the users’ quality of experience (QoE). The existing ABR algorithms require significant tuning and do not generalize to diverse network conditions and personalized QoE objectives. In this article, we propose a novel framework for meta-learning based ABR design and discuss challenges of deploying learning based ABR mechanism in real-world video streaming systems. We utilize the proposed framework to design MetaABR, a novel adaptive bitrate selection algorithm based on meta-reinforcement learning to maximize users’ QoE. By jointly training multiple learning tasks with a shared meta-critic, it can provide transferrable meta-knowledge to supervise bitrate selection across tasks, and can be applied to efficiently learn a new task in unseen environment with only a few trials. We implement MetaABR on an emulation platform which connects to the Linux network protocol stack through virtual network interfaces. Extensive experiments based on real-world traces and wireless testbed show that MetaABR achieves the best comprehensive QoE compared with the state-of-the-art ABR algorithms in a variety of network environments.

Federated learning (FL) has been proposed as a novel paradigm to enable distributed learning on the edge with privacy protection. However, existing federated learning approaches mainly focus on training deep classification and clustering models, and no enough attention has been paid to solve the federated reinforcement learning task on the edge, a challenging task where multiple learning agents observe local state and take local actions to train a global learning model without revealing their local dataset. In this paper, we propose a generalised federated reinforcement learning framework called FedMC that integrates reinforcement learning models trained by multiple edge devices into a general model based on a meta-learning approach. In the proposed framework, each participant adopts a meta-value network (MVN) and task-actor encoder network (TAEN) locally to perform meta-learning training based on local task samples, and periodically uploads the weights of local MVN and TAEN to the server, which aggregate them to a global model with rapid adaptability and cross-task applicability. Extensive experiments based on a number of reinforcement learning tasks show that FedMC outperforms various federated learning baseline algorithms, and it is competitive with the methods that centrally train reinforcement learning task with global dataset.

Congestion control is a fundamental mechanism for TCP protocol, which has been extensively studied in the past three decades. However, our experimental evaluations show that the state-of-art congestion control algorithms such as Cubic and BBR are far from optimal: they have unresolved issues such as insufficient usage of available bandwidth, inadaptable to dynamic bandwidth variants, and compromising on one or more performance dimensions. To address these challenges, we propose a novel congestion control mechanism called NeuRoc that coordinately uses online changepoint detection and deep reinforcement learning (DRL) technique to generate the optimal congestion control policy, which allows TCP operating at Kleinrock ’s optimal operation point to achieve fully bandwidth usage and low latency. To address the practical issues of deploying the deep learning based congestion control mechanism, we propose a cold-started training and deployment framework to reduce the cost of bootstrap. We implement NeuRoc on an emulation platform which connects to the Linux network protocol stack through virtual network interfaces. Extensive experiments show that NeuRoc achieves the best throughput-latency tradeoff compared with the state-of-the-arts in a variety of scenarios.