Deep reinforcement learning (DRL), which highly depends on the data representation, has shown its potential in many practical decision-making problems. However, the process of acquiring representations in DRL is easily affected by interference from models, and moreover leaves unnecessary parameters, leading to control performance reduction. In this article, we propose a double sparse DRL via multilayer sparse coding and nonconvex regularized pruning. To alleviate interference in DRL, we propose a multilayer sparse-coding-structural network to obtain deep sparse representation for control in reinforcement learning. Furthermore, we employ a nonconvex log regularizer to promote strong sparsity, efficiently removing the unnecessary weights with a regularizer-based pruning scheme. Hence, a double sparse DRL algorithm is developed, which can not only learn deep sparse representation to reduce the interference but also remove redundant weights while keeping the robust performance. The experimental results in five benchmark environments of the deep q network (DQN) architecture demonstrate that the proposed method with deep sparse representations from the multilayer sparse-coding structure can outperform existing sparse-coding-based DRL in control, for example, completing Mountain Car with 140.81 steps, achieving near 10% reward increase from the single-layer sparse-coding DRL algorithm, and obtaining 286.08 scores in Catcher, which are over two times the rewards of the other algorithms. Moreover, the proposed algorithm can reduce over 80% parameters while keeping performance improvements from deep sparse representations.

All large-scale online video systems, for example, Netflix and Youku, make a significant investment on video recommendations that can dramatically affect video information diffusion processes among users. However, there is a lack of efficient methodology to interpret how various recommendation mechanisms affect information diffusion processes resulting in the difficulty to evaluate video recommendation efficiency. In this paper, we propose to quantify and explain video recommendation mechanisms by using epidemic models to mine video view count traces. It is well known that an epidemic model is an efficient approach to model information diffusion processes; while view count traces can be viewed as the results of video information diffusion driven by video recommendations. Thus, we propose a framework based on extended epidemic models to quantify and interpret two recommendation mechanisms, that is, direct and word-of-mouth (WOM) recommendations, by fitting video view count traces collected from Tencent Video, a large-scale online video system in China. Our approach is a novel methodology to evaluate video recommendation mechanisms, and a new perspective to interpret how recommendation mechanisms drive view count evolution.

Video popularity (measured by view count) over time is an essential reference for both online video providers and users. According to state-of-the-art works, video popularity is useful for system optimization, load generation, video caching, and video recommendation. Thus, deeper understanding of video popularity evolution is very helpful for improving video service quality and providers' operating efficiency. The core question to be explored in this paper is what key factors govern online video popularity evolution? Through collaboration with our industry partner, Tencent Video, we obtain historical data of video view counts over a period of time, and observe their patterns. We then propose a stochastic fluid model, named as EvoModel, which captures two processes giving rise to different evolution patterns of a given video: (a) the information spreading process and (b) the user reaction process. The driving forces for process (a) can be either via recommendation from the system directly, or word-of-mouth; the extent of the spread is governed by the intrinsic popularity of the video. The factor affecting the second process can be modeled by a user reaction rate. These processes together determine different video popularity evolution patterns. We validate our model by fitting the historical data obtained from a real-world system. Furthermore, we discuss the feasibility of estimating model parameters and predicting popularity.

Large-scale Internet VoD services often adopt a hybrid overlay approach using both P2P and CDN to stream video content. When the underlay ISP network interconnection is weak, as the situation is found in China, it is important to provision the CDN bandwidth carefully in different ISP networks. Through measurement studies using real world data (from Tencent Video), it is found that users in large ISP networks can leverage more on P2P service, compared to the case for smaller ISP networks, and this is especially true during heavy load periods. We study systematic strategies for allocating CDN bandwidth given fixed total bandwidth purchasing budget, and the influence of P2P factor. We show that a more optimal strategy would bias towards allocating proportionally more CDN bandwidth to smaller ISP networks. This result from our model and analysis is validated by simulation. This problem has not been considered by past works and has practical implications for improving the operation of real-world VoD service providers.

Large Internet video delivery systems serve millions of videos to tens of millions of users on daily basis, via Video-on-Demand (VoD) and live streaming. Video popularity (measured by view count) evolves over time. It represents the workload, as well as business value, of the video to the overall system. The ability to predict video popularity is very helpful for improving service quality and operating efficiency. Previous studies adopted simple (usually static) models for video popularity, or directly adopted patterns from measurement studies. In this paper, we develop a fluid model that tries to capture two hidden processes that give rise to different patterns of a given video's popularity evolution: (a) the information spreading process, and (b) the user reaction process. Specifically, these processes model how the video is recommended to the users, the video's inherent attractiveness, and users' reaction rate; and yield different popularity evolution patterns. We validate our model by fitting the data obtained from a large content provider in China. This model gives us the insight to explain the common and different video popularity evolution patterns and why.

Rapid growth in users of smartphones and other mobile devices is increasing the demand for online video service to these mobile devices. The mobile video on demand (VoD) service is almost exclusively provided by content delivery network (CDN) servers since mobile devices are not powerful enough to provide any peer-to-peer (P2P) service. However, in a mixed VoD system with both non-mobile and mobile peers, an interesting question is to ask whether we can leverage non-mobile peers to provide better support for VoD. In this paper, we try to answer this question by a measurement study on Tencent's existing VoD systems. Our measurement results show various interesting characteristics and potential benefits of leveraging non-mobile users. Such results will guide us towards a practical system design and implementation in the near future.