Low latency live streaming (LLLS) leverages chunked transfer encoding (CTE) to substantially reduce end-to-end latency. However, this paradigm introduces a cascade of challenges for adaptive bitrate (ABR) algorithms: (1) the sending idle periods between chunks in CTE render bandwidth measurement difficult and prone to error; (2) bandwidth prediction in LLLS is an irregular time series forecasting with uncertain future segment size, leading to a circular prediction dependency; (3) stochastic uncertainty within LLLS, such as fluctuating idle time, leads to imprecise buffer evolution and ABR degradation. In this paper, we tackle the issues and present AAR, a novel LLLS framework that comprises 3 key modules: (1) accurate bandwidth measurement that leverages a server-side Flag to identify burst transmission and isolate chunks. We further propose to fuse our two learning and heuristic-based algorithms via confidence estimation; (2) bandwidth prediction via conditional normalizing flow to simultaneously learn joint variable distributions. We further propose a bitrate-aware transformer to capture the intrinsic circular relationships as backbone flow condition; (3) an LLLS tailored ABR with a novel and robust objective to maximize the minimum Quality of Experience (QoE) under uncertainty. We propose two theorems to derive the min solution via download time bounds, and we maximize the QoE via Model Predictive Controller (MPC) with LLLS tailored state evolution. Extensive experiments on real-world network traces demonstrate that AAR significantly outperforms baselines with absolute error reduction by 11%-83% for measurement and up to 17% for prediction. We also improve QoE by up to 102% across all tested network conditions.

Network device and system health management is the foundation of modern network operations and maintenance. Traditional health management methods, relying on expert identification or simple rule-based algorithms, struggle to cope with the heterogeneous networks (HNs) environment. Moreover, current state-of-the-art distributed fault diagnosis methods, which utilize specific machine learning techniques, lack multi-scale adaptivity for heterogeneous device information, resulting in unsatisfactory diagnostic accuracy for HNs. In this paper, we develop an LLM-assisted end-to-end intelligent network health management framework. The framework first proposes a multi-scale data scaling method based on unsupervised learning to address the multi-scale data problem in HNs. Secondly, we combine the semantic rule tree with the attention mechanism to propose a Multi-Scale Semanticized Anomaly Detection Model (MSADM) that generates network semantic information while detecting anomalies. Finally, we embed a chain-of-thought-based large-scale language model downstream to adaptively analyze the fault diagnosis results and create an analysis report containing detailed fault information and optimization strategies. We compare our scheme with other fault diagnosis models and demonstrate that it performs well on several metrics of network fault diagnosis.

Over the past decade, adaptive video streaming technology has witnessed significant advancements, particularly driven by the rapid evolution of deep learning techniques. However, the black-box nature of deep learning algorithms presents challenges for developers in understanding decision-making processes and optimizing for specific application scenarios. Although existing research has enhanced algorithm interpretability through decision tree conversion, interpretability does not directly equate to developers' subjective comprehensibility. To address this challenge, we introduce ComTree, the first bitrate adaptation algorithm generation framework that considers comprehensibility. The framework initially generates the complete set of decision trees that meet performance requirements, then leverages large language models to evaluate these trees for developer comprehensibility, ultimately selecting solutions that best facilitate human understanding and enhancement. Experimental results demonstrate that ComTree significantly improves comprehensibility while maintaining competitive performance, showing potential for further advancement. The source code and appendix are available at https://github.com/thu-media/ComTree.

Existing quality of experience (QoE)-driven adaptive bitrate (ABR) algorithms either fail to consider personalized QoE or rely on over-simplified QoE models, all resulting in unsatisfactory streaming experiences. Recognizing the wide existence of user feedback schemes in existing streaming applications, we introduce Q+, a framework leveraging progressively gathered personal user opinion scores from multiple interaction sessions for enhanced user-system alignment. Q+ first innovates QoE modeling by incorporating both pairwise ordinal and cardinal preferences constructed from scores. The capturing of both preferences ensures reliable and robust preference representation. Moreover, we design a monotonic neural network as the QoE model to capture the inherent monotonicity property in ABR services, improving model expressivity and generalization ability even with limited human feedback. To align the policy with the progressively updated QoE, we then develop a value-based reinforcement learning (RL) algorithm for bitrate control that integrates reward relabeling and calibrated prioritized experience replay. Extensive experiments reveal that Q+ consistently surpasses state-of-the-art rule-based, control-based, and RL-based baselines within only three sessions, improving QoE by 5.69% to 29.39% across diverse network conditions.

Control algorithms in production environments typically require domain experts to tune their parameters and logic for specific scenarios. However, existing research predominantly focuses on algorithmic performance under ideal or default configurations, overlooking the critical aspect of Tuning Potential. To bridge this gap, we introduce \texttt{Crucible}, an agent that employs an LLM-driven, multi-level expert simulation to turn algorithms and defines a formalized metric to quantitatively evaluate their Tuning Potential. We demonstrate \texttt{Crucible}'s effectiveness across a wide spectrum of case studies, from classic control tasks to complex computer systems, and validate its findings in a real-world deployment. Our experimental results reveal that \texttt{Crucible} systematically quantifies the tunable space across different algorithms. Furthermore, \texttt{Crucible} provides a new dimension for algorithm analysis and design, which ultimately leads to performance improvements. Our code is available at https://github.com/thu-media/Crucible.

Learning-based adaptive bitrate (ABR) algorithms have revolutionized video streaming solutions. With the growing demand for data privacy and the rapid development of mobile devices, federated learning (FL) has emerged as a popular training method for neural ABR algorithms in both academia and industry. However, we have discovered that FL-based ABR models are vulnerable to model-poisoning attacks as local updates remain unseen during global aggregation. In response, we propose MAFL (Malicious ABR model based on Federated Learning) to prove that backdooring the learning-based ABR model via FL is practical. Instead of attacking the global policy, MAFL only targets a single ``target client''. Moreover, the unique challenges brought by deep reinforcement learning (DRL) make the attack even more challenging. To address these challenges, MAFL is designed with a two-stage attacking mechanism. Using two representative attack cases with real-world traces, we show that MAFL significantly degrades the model performance on the target client (i.e., increasing rebuffering penalty by 2x and 5x) with a minimal negative impact on benign clients.

Adaptive video streaming has been studied for over 10 years and has demonstrated remarkable performance. However, adaptive video streaming is not an independent algorithm but relies on other components of the video system. Consequently, as other components undergo optimization, the gap between the traditional simulator and the real-world system continues to grow which makes the adaptive video streaming algorithm must adapt to these variations.In order to address the challenges facing industrial adaptive video streaming, we introduce a novel offline reinforcement learning framework called Backwave. This framework leverages history logs to reduce the sim-real gap. We propose new metrics based on counterfactual reasoning to evaluate its performance and we integrate expert knowledge to generate valuable data to mitigate the issue of data override. Furthermore, we employ curriculum learning to minimize additional errors.We deployed Backwave on a mainstream commercial short video platform, Kuaishou. In a series of A/B tests conducted nearly one month with over 400M daily watch times, Backwave consistently outperforms prior algorithms. Specifically, Backwave reduces stall time by 0.45% to 8.52% while maintaining comparable video quality and Backwave demonstrates improvements in average play duration by 0.12% to 0.16%, and overall play duration by 0.12% to 0.26%.

Congestion control has been a fundamental research focus in web transmission for over 30 years. However, with diverse network scenarios like cellular networks and WiFi, traditional models might no longer accurately describe current network conditions -- we empirically observe that the minimum round-trip time (RTTmin) still varies under different network conditions, challenging the assumption of its constancy in traditional models. In this paper, we model it as a normal distribution based on our measurements and propose a novel congestion control algorithm LingBo. LingBo consists of two phases: an offline trained decision model to achieve goals under different RTTmin distributions, and an online perception scheme to detect the current RTTmin distribution. We evaluate LingBo in various network environments and find it consistently performs well in terms of power metric and throughput compared to recent state-of-the-art baselines. Our code is available at https://github.com/thumedia/LingBo.

Bitrate adaptation (also known as ABR) is a crucial technique to improve the quality of experience (QoE) for video streaming applications. However, existing ABR algorithms suffer from severe traffic wastage, which refers to the traffic cost of downloading the video segments that users do not finally consume, for example, due to early departure or video skipping. In this paper, we carefully formulate the dynamics of buffered data volume (BDV), a strongly correlated indicator of traffic wastage, which, to the best of our knowledge, is the first time to rigorously clarify the effect of downloading plans on potential wastage. To reduce wastage while keeping a high QoE, we present a bandwidth-efficient bitrate adaptation algorithm (named BE-ABR), achieving consistently low BDV without distinct QoE losses. Specifically, we design a precise, time-aware transmission delay prediction model over the Transformer architecture, and develop a fine-grained buffer control scheme. Through extensive experiments conducted on emulated and real network environments including WiFi, 4G, and 5G, we demonstrate that BE-ABR performs well in both QoE and bandwidth savings, enabling a 60.87% wastage reduction and a comparable, or even better, QoE, compared to the state-of-the-art methods.

Streamers are the core competency of the crowd-sourced live streaming (CLS) platform. However, little work has explored how different factors relate to their popularity evolution patterns. In this paper, we will investigate a critical problem, i.e., how to discover the promising streamers in their early stage? To tackle this problem, we first conduct large-scale measurement on a real-world CLS dataset. We find that streamers can indeed be clustered into two evolution types (i.e., rising type and normal type), and these two types of streamers will show differences in some inherent properties. Traditional time-sequential models cannot handle this problem, because they are unable to capture the complicated interactivity and extensive heterogeneity in CLS scenarios. To address their shortcomings, we further propose Niffler, a novel heterogeneous attention temporal graph framework (HATG) for predicting the evolution types of CLS streamers. Specifically, through the graph neural network (GNN) and gated-recurrent-unit (GRU) structure, Niffler can capture both the interactive features and the evolutionary dynamics. Moreover, by integrating the attention mechanism in the model design, Niffler can intelligently preserve the heterogeneity when learning different levels of node representations. We systematically compare Niffler against multiple baselines from different categories, and the experimental results show that our proposed model can achieve the best prediction performance.

The low-light environment is an integral surrounding in real-world video analytic applications. Conventional wisdom claims that in order to adapt to the extensive computation requirement of the analytics model and achieve high inference accuracy, the overall pipeline should leverage a client-to-cloud framework that designs a cloud-based inference with on-demand video streaming. However, we show that due to the amplified noise, directly streaming the video in low-light scenarios can introduce significant bandwidth inefficiency.In this paper, we propose Owl, an intelligent framework to optimize the bandwidth utilization and inference accuracy for the low-light video analytic pipeline. The core idea of Owl is two-fold: on the one hand, we will deploy a light-weighted pre-processing module before transmission, through which we will get the denoised video and significantly reduce the transmitted data; on the other hand, we recover the information from the denoised video via an enhancement module in the server-side. Specifically, through well-designed training mechanism and content representation technique, Owl can dynamically select the best configuration for time-varying videos. Experiments with a variety of datasets and tasks show that Owl achieves significant bandwidth benefits, while consistently optimizing the inference accuracy.

Quality of Experience (QoE)-driven adaptive bitrate (ABR) algorithms are typically optimized using QoE models that are based on the mean opinion score (MOS), while such principles may not account for user heterogeneity on rating scales, resulting in unexpected behaviors. In this paper, we propose Jade, which leverages reinforcement learning with human feedback(RLHF) technologies to better align the users' opinion scores. Jade's rank-based QoE model considers relative values of user ratings to interpret the subjective perception of video sessions. We implement linear-based and Deep Neural Network (DNN)-based architectures for satisfying both accuracy and generalization ability. We further propose entropy-aware reinforced mechanisms for training policies with the integration of the proposed QoE models. Experimental results demonstrate that Jade performs favorably on conventional metrics, such as quality and stall ratio, and improves QoE by 8.09%-38.13% in different network conditions, emphasizing the importance of user heterogeneity in QoE modeling and the potential of combining linear-based and DNN-based models for performance improvement.

The delay to obtain analysis results is an important metric in video analytics. Previous work has focused on reducing frame transmission and inference delay to optimize total delay. However, network fluctuations can cause frames to arrive at the backend simultaneously, leading to backend queuing delays. To address this issue, we propose Concerto, a joint front-and backend video analytics pipeline that optimizes both network transmission and backend queuing delays. The backend controls the frame queue, accelerating or skipping inference as needed to mitigate backend queuing delay. The frontend considers both delays when configuring frames to send, resulting in better total delay. Experiments show that Concerto significantly reduces backend queuing delay with minimal loss of accuracy.

Even though conventional wisdom claims that in order to improve viewer engagement, the cloud-edge providers should serve the viewers with the nearest edge nodes, however, we show that doing this for crowdsourced live streaming (CLS) services can introduce significant costs inefficiency. In this paper, we first carry out large-scale measurement analysis by using the real-world service data from Huawei Cloud, a representative cloud-edge provider in China. We observe that the massive number of channels has proposed great burdens to the operating expenditure of the cloud-edge providers, and most importantly, unbalanced viewer distribution makes the edge nodes suffer significant costs inefficiency. To tackle the above concerns, we proposeAggCast, a novel CLS scheduling framework to optimize the edge node utilization for the cloud-edge provider. The core idea ofAggCastis to aggregate some viewers that are initially scattered on different regions, and assign them to fewer pre-selected nodes, thereby reducing bandwidth costs. In particular, by integrating the useful insights obtained from our large-scale measurement,AggCastcan not only ensure that quality of experience (QoS) does not suffer degradation, but also satisfy the systematic requirements of CLS services.AggCasthas been A/B tested and fully deployed. The online and trace-driven experiments show that, compared to the most prevalent method,AggCastsaves over 16.3%back-to-source(BTS) bandwidth costs while significantly improving QoS (startup latency, stall frequency and stall time are reduced over 12.3%, 4.57% and 3.91%, respectively).

Congestion control (CC) algorithms based on deep reinforcement learning (DRL) have shown their great potential to adapt themselves to a variety of network conditions. However, as the real-world network conditions are diverse and the optimization goals for CC are made up of some contradicted metrics, it is hard to balance these contradicted absolute valves, which makes it difficult to faithfully reflect the algorithm performance if only considering transient status as the reward. In this work, we propose a novel DRL-based CC approach ZiXia, which considers the adjusted ranking reward, the long-term relative performance reward adjusted by the transient reward. In detail, we design a virtual algorithm arena including the DRL-agents and other classic algorithms as competitors in the same environment. After these algorithms end, we rank their delay and throughput respectively and combine the two relative rankings as the ranking reward using the special preference. The ranking reward gives us a more flexible and interpretable long-term evaluation method compared with the absolute value of the transient status, which gives us a more intuitive perspective to support multi-objection. To get more fine-grained action rewards, we adjust the ranking reward using a linear combination of transient status. Through various experiments, in simulated environments, ZiXia achieves the highest throughput and reduces 87% delay compared with BBR, and in global real-world environments, ZiXia improves 13% throughput and reduces 3% delay than BBR.

Short video uploading service has become increasingly important, as at least 30 million videos are uploaded per day. However, we find that existing congestion control (CC) algorithms, either heuristics or learning-based, are not applicable for video uploading -- i.e., lacking in the design of the fundamental mechanism and being short of leveraging network modeling. We present DuGu, a novel learning-based CC algorithm designed by considering the unique proprieties of video uploading via the probing phase and internet networking via the control phase. During the probing phase, DuGu leverages the transmission gap of uploading short videos to actively detect the network metrics to better understand network dynamics. DuGu uses a neural network~(NN) to avoid congestion during the control phase. Here, instead of using handcrafted reward functions, the NN is learned by imitating the expert policy given by the optimal solver, improving both performance and learning efficiency. To build this system, we construct an omniscient-like network emulator, implement an optimal solver and collect a large corpus of real-world network traces to learn expert strategies. Trace-driven and real-world A/B tests reveal that DuGu supports multi-objective and rivals or outperforms existing CC algorithms across all considered scenarios.

Conventional wisdom claims that in order to improve viewer engagement, the cloud-edge providers should serve the viewers with the nearest edge nodes, however, we show that doing this for crowdsourced live streaming (CLS) services can introduce significant costs inefficiency. We observe that the massive number of channels has greatly burdened the operating expenditure of the cloud-edge providers, and most importantly, unbalanced viewer distribution makes the edge nodes suffer significant costs inefficiency. To tackle the above concerns, we propose AggCast, a novel CLS scheduling framework to optimize the edge node utilization for the cloud-edge provider. The core idea of AggCast is to aggregate some viewers who are initially scattered on different regions, and assign them to fewer pre-selected nodes, thereby reducing bandwidth costs. In particular, by leveraging the insights obtained from our large-scale measurement, AggCast can not only ensure quality of experience (QoS), but also satisfy the systematic requirements of CLS services. AggCast has been A/B tested and fully deployed in a top cloud-edge provider in China for over eight months. The online and trace-driven experiments show that, compared to the common practice, AggCast can save over 15% back-to-source (BTS) bandwidth costs while having no negative impacts on QoS.

Federated learning (FL) is a privacy-preserving distributed learning paradigm that enables collaborative training of machine learning models among multiple participants. However, despite recent progress, existing federated learning systems can still not handle heterogeneous models. For instance, candidate clients with heterogeneous models are inaccessible to the established federated system. And within the federated system, local models are forbidden to be updated to become heterogeneous models, even though the updated models work better.

Internet adaptive video streaming is a typical form of video delivery that leverages adaptive bitrate (ABR) algorithms to provide video services with high quality of experience (QoE) for various users in diverse and unique network conditions. Such heterogeneous network environments, which can be viewed as exogenous input processes, often lead to the unstable performance of ABR algorithms. Unfortunately, learning-based ABR algorithm which generated by state-of-the-art reinforcement learning (RL) technologies achievesgood average performancebut fails to perform well in all kinds of network conditions. In this work, considering the video playback process as the Input-driven Markov Decision Process (IMDP), we propose$\text{A}^{2}$BR (Adaptation of ABR), a novel meta-RL ABR approach.$\text{A}^{2}$BR is mainly composed of an online stage and an offline stage. It leverages meta-RL to learn an initial meta-policy with various network conditions at the offline stage and makes decisions in personalized network conditions at the online stage. At the same time, we continually optimize the meta-policy to the tailor-made ABR policy for varying the current network environment within few shots. Moreover, in order to improve the learning efficiency, we fully utilize domain knowledge for implementing a virtual player to replay the previously experienced network. Using trace-driven experiments on various scenarios including different vehicles, users, network types, and heterogeneous user-preferences, we show that$\text{A}^{2}$BR outperforming recent ABR approaches with rapidly adapting to the personalized QoE metrics and specific network conditions. Testbed experimental results also illustrate the superiority of$\text{A}^{2}$BR in adapting to the unseen environments.

Video transmission services adopt adaptive algorithms to ensure users’ demands. Existing techniques are often optimized and evaluated by a function that linearly combines several weighted metrics. Nevertheless, we observe that the given function often fails to describe the requirement accurately, resulting in the violation of generating the required methods. We proposeZwei, a self-play reinforcement learning framework that updates the policy by straightforwardly utilizing the actual requirement. Technically, Zwei effectively rolls out the trajectories from the same initial state, and instantly estimate the win rate w.r.t the competition outcome, where the outcome represents which trajectory is closer to the assigned requirement. We evaluate Zwei with different requirements on various video transmission tasks, including adaptive bitrate streaming, crowd-sourced live streaming scheduling, and real-time communication. Results indicate that Zwei optimizes itself according to the assigned requirement faithfully, outperforming the state-of-the-art methods under all considered scenarios. Moreover, we further proposeZwei$^+$, which enables Zwei to learn the policies in the vanilla no-regret reinforcement learning scenario. We validate Zwei$^+$in the adaptive bitrate streaming task and show the superiority of the proposed method over existing state-of-the-art approaches.

Most applications of interactive multimedia require the data to arrive within the specific acceptable end-to-end latency (i.e., meeting deadline). To avoid efforts being wasted, the content must reach the destination before the deadline. In our work, we propose DAP (Deadline And Priority-aware congestion control) to achieve high throughput within acceptable end-to-end latency, especially to send high-priority packets while meeting deadline requirements. DAP is mainly composed of two modules: i) the scheduler decides which packet should be sent at first w.r.t the reward function with fully considering the packets' priority, deadline, and current network conditions. ii) the deadline-sensitive congestion control module transmits packets with high efficiency while guaranteeing the end-to-end latency. Specifically, we propose an improved packet-pair scheme to adjust the best congestion window corresponding to the Bandwidth-Delay Product and to update the instant sending rate by current queue length. Experimental results demonstrate the significant performance of our scheme and DAP ranks first in both the training phase and final phase of the ACM MM 2021 Grand Challenge: Meet Deadline Requirements.

In the typical transcoding pipeline for adaptive video streaming, raw videos are pre-chunked and pre-encoded according to a set of resolution-bitrate or resolution-quality pairs on the server-side, where the pair is often named as bitrate ladder. Different from existing heuristics, we argue that a good bitrate ladder should be optimized by considering video content features, network capacity, and storage costs on the cloud. We propose DeepLadder, a per-chunk optimization scheme which adopts state-of-the-art deep reinforcement learning (DRL) method to optimize the bitrate ladder w.r.t the above concerns. Technically, DeepLadder selects the proper setting for each video resolution autoregressively. We use over 8,000 video chunks, measure over 1,000,000 perceptual video qualities, collect real-world network traces for more than 50 hours, and invent faithful virtual environments to help train DeepLadder efficiently. Across a series of comprehensive experiments on both Constant Bitrate (CBR) and Variable Bitrate (VBR)-encoded videos, we demonstrate significant improvements in average video quality bandwidth utilization, and storage overhead in comparison to prior work as well as the ability to be deployed in the real-world transcoding framework.

Owing to the unprecedented growth of video demands, video caching has been a basic network functionality in today's network architectures to offload backbone traffics, as well as provide users with lower access delay. Although abundant cache replacement algorithms have been proposed recently, they all suffer from a critical limitation: due to their immature rules, inaccurate feature engineering or unresponsive model update, they cannot strike a balance between the long-term history and short-term sudden events. To tackle this problem, we propose LA-E2, a long-short-term fusion cache replacement approach, which is based on a learning-aided exploration-exploitation process. Specifically, by effectively combining the deep neural network (DNN) based prediction with the online exploitation exploration through a top-k method, LA-E2 can both make use of the historical information and adapt to the constantly changing popularity responsively. Through the extensive experiments in two real-world datasets, LA-E2 is demonstrated to achieve state-of-the-art performance and generalize well. Especially when the cache size is small, LA-E2 outperforms the baselines by 17.5% 68.7% higher in total hit rate.

The leading adaptive bitrate (ABR) algorithm leverages model predictive control (MPC) method to determine next chunks' video bitrate, while it heavily relies on the accuracy of throughput prediction, which thereby fails to perform well in all considered network scenarios. In this paper, we propose DeepMPC, which enhances MPC via two deep learning-based modules, i.e., DL-based Throughput Predictor (DTP), which can precisely predict future bandwidth, and Discounted Factor Optimizer (DFO), which estimates the prediction error. Using trace-driven experiments, we illustrate that DeepMPC outperforms existing ABR schemes in all considered network conditions, with the improvements on average QoE of 5.91% - 56.1%. Moreover, we implement DeepMPC in real-world network environments and extensive experimental results demonstrate the superiority of DeepMPC against existing state-of-the-art approaches.

Off-the-shelf buffer-based approaches leverage a simple yet effective buffer-bound to control the adaptive bitrate (ABR) streaming system. Nevertheless, such approaches in standard parameters fail to always provide high quality of experience (QoE) video streaming services under all considered network conditions. Meanwhile, state-of-the-art learning-based ABR approach Pensieve outperforms existing schemes but is impractical to deploy. Therefore, how to harmoniously fuse the buffer-based and learning-based approach has become a key challenge for further enhancing ABR methods. In this paper, we propose Stick, an ABR algorithm that fuses the deep learning method and traditional buffer-based method. Stick utilizes the deep reinforcement learning (DRL) method to train the neural network, which outputs the buffer-bound to control the buffer-based approach for maximizing the QoE metric with different parameters. Trace-driven emulation illustrates that Stick betters Pensieve by 3.5% - 9.41% with an overhead reduction of 88%. Moreover, aiming to further reduce the computational costs while preserving the performances, we propose Trigger, a light-weighted neural network that determines whether the buffer-bound should be adjusted. Experimental results show that Stick+Trigger rivals or outperforms existing schemes in average QoE by 1.7%-28%, and significantly reduces the Stick's computational overhead by 24%-61%. Meanwhile, we show that Trigger also helps other ABR schemes mitigate the overhead. Extensive results on real-world evaluation demonstrate the superiority of Stick over existing state-of-the-art approaches.

Livecast streaming has received great success in recent years. Although many prior efforts have suggested that dynamic viewer scheduling according to the quality of service (QoS) can improve user engagement, they may suffer inefficiency due to their ignorance of viewer heterogeneity in how the QoS impact quality of experience (QoE).

Video transmission services adopt adaptive algorithms to ensure users' demands. Existing techniques are often optimized and evaluated by a function that linearly combines several weighted metrics. Nevertheless, we observe that the given function fails to describe the requirement accurately. Thus, such proposed methods might eventually violate the original needs. To eliminate this concern, we propose Zwei, a self-play reinforcement learning algorithm for video transmission tasks. Zwei aims to update the policy by straightforwardly utilizing the actual requirement. Technically, Zwei samples a number of trajectories from the same starting point, and instantly estimates the win rate w.r.t the competition outcome. Here the competition result represents which trajectory is closer to the assigned requirement. Subsequently, Zwei optimizes the strategy by maximizing the win rate. To build Zwei, we develop simulation environments, design adequate neural network models, and invent training methods for dealing with different requirements on various video transmission scenarios. Trace-driven analysis over two representative tasks demonstrates that Zwei optimizes itself according to the assigned requirement faithfully, outperforming the state-of-the-art methods under all considered scenarios.

Existing Adaptive Bitrate (ABR) algorithms pick future video chunks' bitrates via fixed rules or offline trained models to ensure good quality of experience (QoE) for Internet video. Nevertheless, data analysis demonstrates that a good ABR algorithm is required to continually and fast update for adapting itself to time-varying network conditions. Therefore, we propose Comyco, a video quality-aware learning-based ABR approach that enormously improves recent schemes by i) picking the chunk with higher perceptual video qualities rather than video bitrates; ii) training the policy via imitating expert trajectories given by the expert strategy; iii) employing the lifelong learning method to continually train the model w.r.t the fresh trace collected by the users. To achieve this, we develop a complete quality-aware lifelong imitation learning-based ABR system, construct quality-based neural network architecture, collect a quality-driven video dataset, and estimate QoE metrics with video quality features. Using trace-driven and real-world experiments, we demonstrate Comyco reaches 1700-fold improvements in the number of samples required and 16-fold speedup in the training time compared with the prior work. Meanwhile, Comyco outperforms existing methods, with the improvements on average QoE of 7.5%-16.79%. Moreover, experimental results on continual training also illustrate that lifelong learning helps Comyco further improve the average QoE of 1.07%-9.81% in comparison to the offline trained model.

Scheduling viewers effectively among different Content Delivery Network (CDN) providers is challenging owing to the extreme diversity in the crowdsourced live streaming (CLS) scenarios. Abundant algorithms have been proposed in recent years, which, however, suffer from a critical limitation: Due to their inaccurate feature engineering or naive rules, they cannot optimally schedule viewers. To address this concern, we put forward LTS (Learn to Schedule), a novel scheduling algorithm that can adapt to the dynamics from both viewer traffics and CDN performance. In detail, we first propose LTS-RL, an approach that schedules CLS viewers based on deep reinforcement learning (DRL). Since LTS-RL is trained in an end-to-end way, it can automatically learn scheduling algorithms without any pre-programmed models or assumptions about the environment dynamics. At the same time, to practically deploy LTS-RL, we then use the decision tree and imitation learning to convert LTS-RL into a more light-weighted and interpretable model, which is denoted as Fast-LTS. After the extensive evaluation of the real data from a leading CLS platform in China, we demonstrate that our proposed model (both LTS-RL and Fast-LTS) can improve the average quality of experience (QoE) over state-of-the-art approaches by 8.71--15.63%. At the same time, we also demonstrate that Fast-LTS can faithfully convert the complicated LTS-RL with slight performance degradation (< 2%), while significantly reducing the decision time (×7--10).

Federated learning enables on-device training over distributed networks consisting of a massive amount of modern smart devices, such as smartphones and IoT devices. However, the leading optimization algorithm in such settings, i.e., federated averaging, suffers from heavy communication cost and inevitable performance drop, especially when the local data is distributed in a Non-IID way. In this paper, we propose a feature fusion method to address this problem. By aggregating the features from both the local and global models, we achieve a higher accuracy at less communication cost. Furthermore, the feature fusion modules offer better initialization for newly incoming clients and thus speed up the process of convergence. Experiments in popular federated learning scenarios show that our federated learning algorithm with feature fusion mechanism outperforms baselines in both accuracy and generalization ability while reducing the number of communication rounds by more than 60%.

Existing reinforcement learning (RL)-based adaptive bitrate (ABR) approaches outperform the previous fixed control rules based methods by improving the Quality of Experience (QoE) score, as the QoE metric can hardly provide clear guidance for optimization, finally resulting in the unexpected strategies. In this paper, we propose Tiyuntsong, a self-play reinforcement learning approach with generative adversarial network (GAN)-based method for ABR video streaming. Tiyuntsong learns strategies automatically by training two agents who are competing against each other. Note that the competition results are determined by a set of rules rather than a numerical QoE score that allows clearer optimization objectives. Meanwhile, we propose GAN Enhancement Module to extract hidden features from the past status for preserving the information without the limitations of sequence lengths. Using testbed experiments, we show that the utilization of GAN significantly improves the Tiyuntsong's performance. By comparing the performance of ABRs, we observe that Tiyuntsong also betters existing ABR algorithms in the underlying metrics.

Video transcoding is widely adopted in live streaming services to bridge the format and resolution gap between content producers and consumers (i.e., broadcasters and viewers). Meanwhile, the cloud has been recognized as one of the most reliable and cost-effective ways for video transcoding. However, due to the dynamic and uncertainty of the transcoding workloads in live streaming, it is very challenging for cloud service providers to provision computing resources and schedule transcoding tasks while guaranteeing the Service Level Agreement (SLA). To this end, we propose a joint resource provisioning and task scheduling approach for transcoding live streams in the cloud. We adopt Deep Reinforcement Learning (DRL) to train a neural network model for resource provisioning under dynamic workloads. Moreover, we design a QoS-aware task scheduling algorithm that maps transcoding tasks to Virtual Machines (VMs) by considering the real-time QoS requirement. We evaluate our approach with trace-driven experiments and the results demonstrate that our approach outperforms heuristic baselines by up to 89% improvements on average QoS with 4% extra resource overhead at most.

Learning-based Adaptive Bit Rate~(ABR) method, aiming to learn outstanding strategies without any presumptions, has become one of the research hotspots for adaptive streaming. However, it is still suffering from several issues, i.e., low sample efficiency and lack of awareness of the video quality information. In this paper, we propose Comyco, a video quality-aware ABR approach that enormously improves the learning-based methods by tackling the above issues. Comyco trains the policy via imitating expert trajectories given by the instant solver, which can not only avoid redundant exploration but also make better use of the collected samples. Meanwhile, Comyco attempts to pick the chunk with higher perceptual video qualities rather than video bitrates. To achieve this, we construct Comyco's neural network architecture, video datasets and QoE metrics with video quality features. Using trace-driven and real world experiments, we demonstrate significant improvements of Comyco's sample efficiency in comparison to prior work, with 1700x improvements in terms of the number of samples required and 16x improvements on training time required. Moreover, results illustrate that Comyco outperforms previously proposed methods, with the improvements on average QoE of 7.5% - 16.79%. Especially, Comyco also surpasses state-of-the-art approach Pensieve by 7.37% on average video quality under the same rebuffering time.

Viewer scheduling among different CDN providers in crowdsourced live streaming (CLS) service is especially challenging due to the large-scale dynamic viewers as well as the time-variant performance of the content delivery network. A practical scheduling method should tackle the following challenges: 1) accurate modeling of viewer patterns and CDN performance; 2) intelligent workload offloading to save costs while guaranteeing the quality of service (QoS); 3) and ease of integration with practical CDN infrastructure in CLS platforms.

The leading learning-based rate control method, i.e., QARC, achieves state-of-the-art performances but fails to interpret the fundamental principles, and thus lacks the abilities to further improve itself efficiently. In this paper, we propose EQARC (Explainable QARC) via reconstructing QARC's modules, aiming to demystify how QARC works. In details, we first utilize a novel hybrid attention-based CNN+GRU model to re-characterize the original quality prediction network and reasonably replace the QARC's 1D-CNN layers with 2D-CNN layers. Using trace-driven experiment, we demonstrate the superiority of EQARC over existing state-of-the-art approaches. Next, we collect several useful information from each interpretable modules and learn the insight of EQARC. Following this step, we further propose AQARC (Advanced QARC), which is the light-weighted version of QARC. Experimental results show that AQARC achieves the same performances as the QARC with an overhead reduction of 90%. In short, through learning from deep learning, we generalize a rate control method which can both reach high performance and reduce computation cost.

With the growing demand for crowdsourced live streaming (CLS), how to schedule the large-scale dynamic viewers effectively among different Content Delivery Network (CDN) providers has become one of the most significant challenges for CLS platforms. Although abundant algorithms have been proposed in recent years, they suffer from a critical limitation: due to their inaccurate feature engineering or naive rules, they cannot optimally schedule viewers. To address this concern, we propose LTS (Learn to schedule), a deep reinforcement learning (DRL) based scheduling approach that can dynamically adapt to the variation of both viewer traffics and CDN performance. After the extensive evaluation the real data from a leading CLS platform in China, we demonstrate that LTS improves the average quality of experience (QoE) over state-of-the-art approach by 8.71%-15.63%.

, is one of the major roadblocks that prevent deep neural networks from achieving human-level artificial intelligence. Several research efforts (e.g., lifelong or continual learning algorithms) have proposed to tackle this problem. However, they either suffer from an accumulating drop in performance as the task sequence grows longer, or require storing an excessive number of model parameters for historical memory, or cannot obtain competitive performance on the new tasks. In this letter, we focus on the incremental multitask image classification scenario. Inspired by the learning process of students, who usually decompose complex tasks into easier goals, we propose an adversarial feature alignment method to avoid catastrophic forgetting. In our design, both the low-level visual features and high-level semantic features serve as soft targets and guide the training process in multiple stages, which provide sufficient supervised information of the old tasks and help to reduce forgetting. Due to the knowledge distillation and regularization phenomena, the proposed method gains even better performance than fine-tuning on the new tasks, which makes it stand out from other methods. Extensive experiments in several typical lifelong learning scenarios demonstrate that our method outperforms the state-of-the-art methods in both accuracy on new tasks and performance preservation on old tasks.

Real-time video streaming is now one of the main applications in all network environments. Due to the fluctuation of throughput under various network conditions, how to choose a proper bitrate adaptively has become an upcoming and interesting issue. To tackle this problem, most proposed rate control methods work for providing high video bitrates instead of video qualities. Nevertheless, we notice that there exists a trade-off between sending bitrate and video quality, which motivates us to focus on how to reach a balance between them.

Rate control is widely adopted during video streaming to provide both high video qualities and low latency under various network conditions. However, despite that many work have been proposed, they fail to tackle one major problem: previous methods determine a future transmission rate as a single for value which will be used in an entire time-slot, while real-world network conditions, unlike lab setup, often suffer from rapid and stochastic changes, resulting in the failures of predictions.