Designing reinforcement learning (RL) agents is typically a difficult process that requires numerous design iterations. Learning can fail for a multitude of reasons and standard RL methods provide too few tools to provide insight into the exact cause. In this paper, we show how to integrate \textit{value decomposition} into a broad class of actor-critic algorithms and use it to assist in the iterative agent-design process. Value decomposition separates a reward function into distinct components and learns value estimates for each. These value estimates provide insight into an agent's learning and decision-making process and enable new training methods to mitigate common problems. As a demonstration, we introduce SAC-D, a variant of soft actor-critic (SAC) adapted for value decomposition. SAC-D maintains similar performance to SAC, while learning a larger set of value predictions. We also introduce decomposition-based tools that exploit this information, including a new reward \textit{influence} metric, which measures each reward component's effect on agent decision-making. Using these tools, we provide several demonstrations of decomposition's use in identifying and addressing problems in the design of both environments and agents. Value decomposition is broadly applicable and easy to incorporate into existing algorithms and workflows, making it a powerful tool in an RL practitioner's toolbox.

Heterogeneous spatial quality in 360-degree videos is characterized by regions of high and low quality appearing in the same video chunk. Encoding quality-varying omnidirectional videos enables bandwidth waste optimization without compromising the quality inside the client's viewport. Heterogeneous spatial quality has been implemented using either the concept of tiling or Facebook's offset projection. In this paper, we study the preparation of heterogeneous 360-degree videos with three main contributions. We propose two novel approaches that prepare heterogeneous quality versions of a 360-degree video based on two video decomposition techniques: the Gaussian pyramid and the Laplace pyramid. We also introduce a spherical filter for omnidirectional videos which allows to further reduce the video bit-rate by exploring the spherical properties of the 360-degree videos. We compare the performance of the proposed three methods to tiling and the offset projection in representative scenarios of heterogeneous spatial quality in 360-degree videos and highlight the key trade-off between bit-rate and video quality to consider when implementing these approaches.

A video with heterogeneous spatial quality is a video where some regions of the frame have a different quality than other regions (for instance, a better quality could mean more pixels and less encoding distortion). Such a quality-variable encoding is a key enabler of Virtual Reality application, with 360-degree videos. So far, the main technique that has been proposed to prepare spatially heterogeneous quality is based on the concept of tiling. More recently, Facebook has implemented another approach: the offset projection where more emphasis is put on a specific direction of the frame. In this paper, we study quality-variable 360-degree videos with two main contributions. First, we provide the theoretical analysis of the offset projection and show the impact of the parameter settings on the video quality. Second, we propose another approach which consists in preparing the 360-degree video from a Gaussian pyramid of downscaled and blurred versions of the video. We perform an evaluation of tiling, offset and Gaussian-based approaches in representative scenarios of heterogeneous spatial quality in 360-degree videos and highlight the main trade-off to consider when implementing these approaches.

The delivery and display of 360-degree videos on Head-Mounted Displays (HMDs) presents many technical challenges. 360-degree videos are ultra high resolution spherical videos, which contain an omnidirectional view of the scene. However only a portion of this scene is displayed on the HMD. Moreover, HMD need to respond in 10 ms to head movements, which prevents the server to send only the displayed video part based on client feedback. To reduce the bandwidth waste, while still providing an immersive experience, a viewport-adaptive 360-degree video streaming system is proposed. The server prepares multiple video representations, which differ not only by their bit-rate, but also by the qualities of different scene regions. The client chooses a representation for the next segment such that its bit-rate fits the available throughput and a full quality region matches its viewing. We investigate the impact of various spherical-to-plane projections and quality arrangements on the video quality displayed to the user, showing that the cube map layout offers the best quality for the given bit-rate budget. An evaluation with a dataset of users navigating 360-degree videos demonstrates that segments need to be short enough to enable frequent view switches.

With the decreasing price of Head-Mounted Displays (HMDs), 360-degree videos are becoming popular. The streaming of such videos through the Internet with state of the art streaming architectures requires, to provide high immersion feeling, much more bandwidth than the median user's access bandwidth. To decrease the need for bandwidth consumption while providing high immersion to users, scientists and specialists proposed to prepare and encode 360-degree videos into quality-variable video versions and to implement viewport-adaptive streaming. Quality-variable versions are different versions of the same video with non-uniformly spread quality: there exists some so-called Quality Emphasized Regions (QERs). With viewport-adaptive streaming the client, based on head movement prediction, downloads the video version with the high quality region closer to where the user will watch. In this paper we propose a generic theoretical model to find out the optimal set of quality-variable video versions based on traces of head positions of users watching a 360-degree video. We propose extensions to adapt the model to popular quality-variable version implementations such as tiling and offset projection. We then solve a simplified version of the model with two quality levels and restricted shapes for the QER. With this simplified model, we show that an optimal set of four quality-variable video versions prepared by a streaming server, together with a perfect head movement prediction, allow for 45% bandwidth savings to display video with the same average quality as state of the art solutions or allows an increase of 102% of the displayed quality for the same bandwidth budget.

This paper evaluates different means of reducing power consumption of macro base stations (BS) and heterogeneous mobile network deployments (HetNet) considering the time dimension. These approaches are based on the same idea of reducing the load of heavily loaded macro cells and putting them to discontinuous transmission (DTX) mode during the time of inactivity by either (1) macro cell densification or (2) offloading traffic to small cells. Activity factor of a BS is defined as the fraction of time the BS is transmitting over a fixed time period. It is shown that by macro cell layer network densification, the average daily area power consumption can be reduced by up to 73 % with the use of cell DTX. However, reducing the activity factor by macro layer densification is not cost effective, as already demonstrated in previous studies. Alternatively, by adding small cells and enabling their DTX capability, power consumption can be reduced by up to 29 %. Adding small cells is especially effective in terms of energy savings, when users are distributed around hot spots, where additional coverage and capacity is required.

Abstract—This paper describes a novel QoE-aware adaptive video streaming method that enhances the viewing experience on mobile devices and reduces cellular network bandwidth consumed by Dynamic Adaptive Streaming over HTTP (DASH) by consid-ering perceptual video quality and data rate channel conditions in the bitrate adaptation process. By streaming an optimized video for the particular video quality and channel conditions to a mobile device, we can improve the worst video qualities caused by DASH streaming and reduce quality variations using fewer number of bits. Keywords—QoE, Adaptive video streaming, DASH I.

The explosive growth of Over-the-top (OTT) online video strains capacity of operators' networks, which severely threatens video quality perceived by end users. Since video is very bandwidth consuming, its distribution costs are becoming too high to scale with network investments that are required to support the increasing bandwidth demand. Content providers and operators are searching for solutions to reduce this video traffic load, without degrading their customers' perceived Quality of Experience (QoE). This paper proposes a method that can programmatically optimize video content for desired QoE according to perceptual video quality and device display properties, while achieving bandwidth and storage savings for content providers, operators, and end users. The preliminary results obtained with Samsung Galaxy S3 phone show that up to 60% savings can be achieved by optimizing movies without compromising the perceptible video quality, and up to 70% for perceptible, but not annoying video quality difference. Tailoring video optimization to individual user perception can provide seamless QoE delivery across all users, with a low overhead (i.e., 10%) required to achieve this goal. Finally, two applications of video optimization: QoE-aware delivery and storage, are proposed and examined.

The arrival of smart phones and tablets, along with a flat rate mobile Internet pricing model have caused increasing adoption of mobile data services. According to recent studies, video has been the main driver of mobile data consumption, having a higher growth rate than any other mobile application. However, streaming a medium/high quality video files can be an issue in a mobile environment where available capacity needs to be shared among a large number of users. Additionally, the energy consumption in mobile devices increases proportionally with the duration of data transfers, which depend on the download data rates achievable by the device. In this respect, adoption of opportunistic content pre-fetching schemes that exploit times and locations with high data rates to deliver content before a user requests it, has the potential to reduce the energy consumption associated with content delivery and improve the user's quality of experience, by allowing playback of pre-stored content with virtually no perceived interruptions or delays. This paper presents a family of opportunistic content pre-fetching schemes and compares their performance to standard on-demand access to content. By adopting a simulation approach on experimental data, collected with monitoring software installed in mobile terminals, we show that content pre-fetching can reduce energy consumption of the mobile devices by up to 30% when compared to the on demand download of the same file, with a time window of 1 hour needed to complete the content prepositioning.

In a traditional context-aware system, most context information is local to a device. However, we may need access to context information from outside the device. Increasingly mobile electronic devices are equipped with Bluetooth and/or WLAN network interfaces. Both of these technologies enable ad ho