Reaching close-to-optimal bandwidth utilization in dynamic adaptive streaming over HTTP (DASH) systems can, in theory, be achieved with a small discrete set of bit rate representations. This includes typical bit rate ladders used in state-of-the-art DASH systems. In practice, however, we demonstrate that bandwidth utilization, and consequently the quality of experience (QoE), can be improved by offering a continuous set of bit rate representations, i.e., a continuous bit rate slide (COBIRAS). Moreover, we find that the buffer fill behavior of different standard adaptive bit rate (ABR) algorithms is sub-optimal in terms of bandwidth utilization. To overcome this issue, we leverage COBIRAS’ flexibility to request segments with any arbitrary bit rate and propose a novel ABR algorithm MinOff , which helps maximizing bandwidth utilization by minimizing download off-phases during streaming. To avoid extensive storage requirements with COBIRAS and to demonstrate the feasibility of our approach, we design and implement a proof-of-concept DASH system for video streaming that relies on just-in-time encoding ( JITE ), which reduces storage consumption on the DASH server. Finally, we conduct a performance evaluation on our testbed and compare a state-of-the-art DASH system with few bit rate representations and our JITE DASH system, which can offer a COBIRAS, in terms of bandwidth utilization and video QoE for different ABR algorithms.

Audiovisual asynchrony is a significant factor im-pacting the Quality of Experience (QoE), especially for interactive communication like video conferencing. In this paper, we propose a client-side approach to predict the delay between an audio and a video signal, using only the media signals from both streams. Features are extracted from the video and audio stream, respectively, and analyzed using a cross-correlation approach to determine the actual delay. Our approach predicts the delay with an accuracy of over 80% in a time frame of ±1s. We further highlight the potential drawbacks of using a cross-correlation-based analysis and propose different solutions for practical implementations of a delay-based QoE metric.

This paper uses a crowdsourced dataset of online video streaming sessions to investigate opportunities to reduce the power consumption while considering QoE. For this, we base our work on prior studies which model both the end-user's QoE and the end-user device's power consumption with the help of high-level video features such as the bitrate, the frame rate, and the resolution. On top of existing research, which focused on reducing the power consumption at the same QoE optimizing video parameters, we investigate potential power savings by other means such as using a different playback device, a different codec, or a predefined maximum quality level. We find that based on the power consumption of the streaming sessions from the crowdsourcing dataset, devices could save more than 55% of power if all participants adhere to low-power settings.

In the past decade, we have witnessed an enormous growth in the demand for online video services. Recent studies estimate that nowadays, more than 1% of the global greenhouse gas emissions can be attributed to the production and use of devices performing online video tasks. As such, research on the true power consumption of devices and their energy efficiency during video streaming is highly important for a sustainable use of this technology. At the same time, over-the-top providers strive to offer high-quality streaming experiences to satisfy user expectations. Here, energy consumption and QoE partly depend on the same system parameters. Hence, a joint view is needed for their evaluation. In this paper, we perform a first analysis of both end-user power efficiency and Quality of Experience of a video streaming service. We take a crowdsourced dataset comprising 447,000 streaming events from YouTube and estimate both the power consumption and perceived quality. The power consumption is modeled based on previous work which we extended towards predicting the power usage of different devices and codecs. The user-perceived QoE is estimated using a standardized model. Our results indicate that an intelligent choice of streaming parameters can optimize both the QoE and the power efficiency of the end user device. Further, the paper discusses limitations of the approach and identifies directions for future research.

Spatial Information (SI) and Temporal Information (TI) are frequently-used metrics to classify the spatiotemporal complexity of video content. However, they are mostly used on original video sources, and their impact on actual encoding efficiency is not known. In this paper, we propose a method to determine the compressibility of video sources, that is, how good video quality can be under a given bitrate constraint. We show how various aggregations of SI and TI correlate with compressibility scores obtained from a public dataset of H.264/HEVCN P9 content. We observe that the minimum TI value as well as an existing criticality metric from the literature are good indicators for compressibility, as judged by subjective ratings as well as VMAF and P.1204.3 objective scores.

HTTP Adaptive Streaming (HAS) is the de-facto standard for video delivery over the Internet. It enables dynamic adaptation of video quality by splitting a video into small segments and providing multiple quality levels per segment. So far, HAS services typically utilize a fixed segment duration. This reduces the encoding and streaming variability and thus allows a faster encoding of the video content and a reduced prediction complexity for adaptive bit rate algorithms. Due to the content-agnostic placement of I-frames at the beginning of each segment, additional encoding overhead is introduced. In order to mitigate this overhead, variable segment durations, which take encoder placed I-frames into account, have been proposed recently. Hence, a lower number of I-frames is needed, thus achieving a lower video bitrate without quality degradation. While several proposals exploiting variable segment durations exist, no comparative study highlighting the impact of this technique on coding efficiency and adaptive streaming performance has been conducted yet. This paper conducts such a holistic comparison within the adaptive video streaming eco-system. Firstly, it provides a broad investigation of video encoding efficiency for variable segment durations. Secondly, a measurement study evaluates the impact of segment duration variability on the performance of HAS using three adaptation heuristics and the dash.js reference implementation. Our results show that variable segment durations increased the Quality of Experience for 54% of the evaluated streaming sessions, while reducing the overall bitrate by 7% on average.

With the increasing requirement of users to view high-quality videos with a constrained bandwidth, typically realized using HTTP-based adaptive streaming, it becomes more and more important to determine the quality of the encoded videos accurately, to assess and possibly optimize the overall streaming quality. In this paper, we describe a bitstream-based no-reference video quality model developed as part of the latest model-development competition conducted by ITU-T Study Group 12 and the Video Quality Experts Group (VQEG), “P.NATS Phase 2”. It is now part of the new P.1204 series of Recommendations as P.1204.3. It can be applied to bitstreams encoded with H.264/AVC, HEVC and VP9, using various encoding options, including resolution, bitrate, framerate and typical encoder settings such as number of passes, rate control variants and speeds. The proposed model follows an ensemble-modelling-inspired approach with weighted parametric and machine-learning parts to efficiently leverage the performance of both approaches. The paper provides details about the general approach to modelling, the features used and the final feature aggregation. The model creates per-segment and per-second video quality scores on the 5-point Absolute Category Rating scale, and is applicable to segments of 5–10 seconds duration. It covers both PC/TV and mobile/tablet viewing scenarios. We outline the databases on which the model was trained and validated as part of the competition, and perform an additional evaluation using a total of four independently created databases, where resolutions varied from 360p to 2160p, and frame rates from 15–60fps, using realistic coding and bitrate settings. We found that the model performs well on the independent dataset, with a Pearson correlation of 0.942 and an RMSE of 0.42. We also provide an open-source reference implementation of the described P.1204.3 model, as well as the multi-codec bitstream parser required to extract the input data, which is not part of the standard.

As video streaming accounts for the majority of Internet traffic, monitoring its quality is of importance to both Over the Top (OTT) providers as well as Internet Service Providers (ISPs). While OTTs have access to their own analytics data with detailed information, ISPs often have to rely on automated network probes for estimating streaming quality, and likewise, academic researchers have no information on actual customer behavior. In this paper, we present first results from a large-scale crowdsourcing study in which three major video streaming OTTs were compared across five major national ISPs in Germany. We not only look at streaming performance in terms of loading times and stalling, but also customer behavior (e.g., user engagement) and Quality of Experience based on the ITU-T P.1203 QoE model. We used a browser extension to evaluate the streaming quality and to passively collect anonymous OTT usage information based on explicit user consent. Our data comprises over 400,000 video playbacks from more than 2,000 users, collected throughout the entire year of 2019. The results show differences in how customers use the video services, how the content is watched, how the network influences video streaming QoE, and how user engagement varies by service. Hence, the crowdsourcing paradigm is a viable approach for third parties to obtain streaming QoE insights from OTTs.

4K television screens or even with higher resolutions are currently available in the market. Moreover video streaming providers are able to stream videos in 4K resolution and beyond. Therefore, it becomes increasingly important to have a proper understanding of video quality especially in case of 4K videos. To this effect, in this paper, we present a study of subjective and objective quality assessment of 4K ultra-high-definition videos of short duration, similar to DASH segment lengths. As a first step, we conducted four subjective quality evaluation tests for compressed versions of the 4K videos. The videos were encoded using three different video codecs, namely H.264, HEVC, and VP9. The resolutions of the compressed videos ranged from 360p to 2160p with framerates varying from 15fps to 60fps. All the source 4K contents used were of 60fps. We included low quality conditions in terms of bitrate, resolution and framerate to ensure that the tests cover a wide range of conditions, and that e.g. possible models trained on this data are more general and applicable to a wider range of real world applications. The results of the subjective quality evaluation are analyzed to assess the impact of different factors such as bitrate, resolution, framerate, and content. In the second step, different state-of-the-art objective quality models were applied to all videos and their performance was analyzed in comparison with the subjective ratings, e.g. using Netflix's VMAF. The videos, subjective scores, both MOS and confidence interval per sequence and objective scores are made public for use by the community for further research.

The test methods recommended by the International Telecommunication Union (ITU) for assessing 2D video quality are often used for evaluating omnidirectional / 360° videos. In this paper, we compare the performance of three different test methods, Absolute Category Rating (ACR), a modified version of ACR (M-ACR) with double presentation of the test stimulus, and DSIS (Double Stimulus Impairment Scale), based on the statistical reliability, assessment time and simulator sickness. Different settings were used for HEVC encoding of five 360° source videos of 10 s duration. Results indicate that DSIS is statistically more reliable with higher resolving power, followed by M-ACR and ACR. We found that simulator sickness increases with time, but can be reduced by taking breaks in-between the test sessions. The results for simulator sickness are compared across test methods and with similar tests conducted under different contextual conditions. We also recorded and analyzed the exploration behaviour of the users. Apart from the methodological findings, the test results provide insights into video quality for different resolution and encoding settings (“bitrate ladders”). These may be useful for choosing appropriate representations in the context of HTTP-based adaptive streaming in case of full-frame streaming.

Video resolution changes in an HTTP adaptive streaming session may negatively affect the viewer's quality of experience. Our goal is, through encoding, to make such resolution changes less noticeable for the viewers. This can be achieved by taking video complexity features into account during the encoding process. In this paper, we compare Constant Bitrate (CBR) versus Constrained Constant Rate Factor (CRF) coding approaches and their effects on the noticeability of video resolution changes. To this end, we conducted a dedicated subjective study with 20 subjects in a quasi -lab environment. Our results suggest that choices for fixed-bitrate encoding have to be improved by deeper analysis of video complexity and resolution change patterns.

This paper describes an open dataset and software for ITU-T Ree. P.1203. As the first standardized Quality of Experience model for audiovisual HTTP Adaptive Streaming (HAS), it has been extensively trained and validated on over a thousand audiovisual sequences containing HAS-typical effects (such as stalling, coding artifacts, quality switches). Our dataset comprises four of the 30 official subjective databases at a bitstream feature level. The paper also includes subjective results and the model performance. Our software for the standard was made available to the public, too, and it is used for all the analyses presented. Among other previously unpublished details, we show the significant performance improvements of using bitstream-based models over metadata-based ones for video quality analysis, and the robustness of combining classical models with machine-learning-based approaches for estimating user QoE.

The available Internet bandwidth has a strong impact on the Quality of Experience of video services. In order to manage their network efficiently and prevent customer churn, Internet Service Providers need to constantly monitor the QoE of video services such as YouTube. However, they often only rely on simple measurement scenarios that consider only one video being loaded repeatedly. In this paper we compare this scenario against a new approach in which multiple videos are being loaded in a session, thereby simulating user behavior. Using a testbed, we study the impact of download speeds on Key Performance Indicators (KPIs such as initial loading time and stalling events) and user QoE as measured using the ITU-T P.1203 standard. We show that the monitoring paradigm has a significant impact on the obtained results. We further provide a prediction model for estimating the impact of download speed on KPIs and user QoE.

HTTP adaptive streaming (HAS) has become the de-facto standard for video streaming to ensure continuous multimedia service delivery under irregularly changing network conditions. Many studies already investigated the detrimental impact of various playback characteristics on the Quality of Experience of end users, such as initial loading, stalling or quality variations. However, dedicated studies tackling the impact of resolution adaptation are still missing. This paper presents the results of an immersive audiovisual quality assessment test comprising 84 test sequences from four different video content types, emulated with an HAS adaptation mechanism. We employed a novel approach based on systematic creation of adaptivity conditions which were assigned to source sequences based on their spatio-temporal characteristics. Our experiment investigates the resolution switch effect with respect to the degradations in MOS for certain adaptation patterns. We further demonstrate that the content type and resolution change patterns have a significant impact on the perception of resolution changes. These findings will help develop better QoE models and adaptation mechanisms for HAS systems in the future.

There is an ongoing debate in the research community over the improved visual quality of UHD video in comparison to the still widely-deployed HD standard. It is the inspiration of many scientific studies, yet UHD displays and services are continuously spreading on the consumer market. This paper presents the results of a subjective paired-comparison test with both upscaled HD and UHD video sequences, investigating the primary research question of whether UHD can offer a significant improvement over HD for common viewing conditions. In our study, subjects rated their visual preference on video-only clips without lossy encoding. In addition we studied cognitive biases, by presenting a label of what users were about to see (HD or UHD) before the sequence, purposely manipulating the labels in some conditions to suggest different resolutions than actually shown. Finally, we investigated the impact of different rating scales on the precision of the results. Our studies show that HD clips appear indistinguishable from UHD clips when the rating scale chosen is not fine-grained enough. We also found that the labeling effect has a significant impact on the perceived quality, overriding the actual visual perception. Even with a precise scale, users cannot detect major improvements of UHD compared to HD, which may let us question the added value offered by UHD.

The paper presents the scalable video quality model part of the P.1203 Recommendation series, developed in a competition within ITU-T Study Group 12 previously referred to as P.NATS. It provides integral quality predictions for 1 up to 5 min long media sessions for HTTP Adaptive Streaming (HAS) with up to HD video resolution. The model is available in four modes of operation for different levels of media-related bitstream information, reflecting different types of encryption of the media stream. The video quality model presented in this paper delivers short-term video quality estimates that serve as input to the integration component of the P.1203 model. The scalable approach consists in the usage of the same components for spatial and temporal scaling degradations across all modes. The third component of the model addresses video coding artifacts. To this aim, a single model parameter is introduced that can be derived from different types of bitstream input information. Depending on the complexity of the available input, one of four scaling-levels of the model is applied. The paper presents the different novelties of the model and scientific choices made during its development, the test design, and an analysis of the model performance across the different modes.

This paper describes a quality model for HTTP Adaptive Streaming. It integrates existing audio and video quality scores to a final quality estimation, factoring in quality variations over time, the recency effect, as well as location and length of buffering events at the player side. We built the model based on data gathered from more than 17 subjective quality tests. It was submitted to the ITU-T P.NATS competition; parts of it have since been released in the official recommendation ITU-T P.1203.3 as an “audiovisual quality integration module”. In the context of standardization, the model was validated on 30 subjective databases, showing high performance. Its modular approach allows its components to be re-used in other applications and combined with different temporal pooling techniques.

In this paper, we evaluated and compared the integral quality of different omnidirectional contents for two head mounted displays (HMDs), namely HTC Vive and Oculus Rift. We also investigated motion sickness and head-movements. To this aim, we categorized omnidirectional contents into three categories based on the degree of motion: high, medium and low motion. For assessing simulator sickness, we used the Simulator Sickness Questionnaire for each of the contents in both HMDs. The viewing direction for subjects while watching the contents were recorded in terms of the three coordinates yaw, roll and pitch. Experimental results show that HTC Vive offers better integral quality compared to Oculus Rift. We also compared simulator sickness scores along with the behavioral data for different contents and HMDs and discussed the results in the paper.

The ever-increasing network traffic and user expectations at reduced cost make the delivery of high Quality of Experience (QoE) for multimedia services more vital than ever in the eyes of Internet Service Providers (ISPs). Real-time quality monitoring, with a focus on the user, has become essential as the first step in cost-effective provisioning of high quality services. With the recent changes in the perception of user privacy, the rising level of application-layer encryption and the introduction and deployment of virtualized networks, QoE monitoring solutions need to be adapted to the fast changing Internet landscape. In this contribution, we provide an overview of state-of-the-art quality monitoring models and probing technologies, and highlight the major challenges ISPs have to face when they want to ensure high service quality for their customers.

Assuring user engagement has become a key issue for Internet Service Providers and Over-the-Top Providers. How long are users consuming a service? When are they likely to abandon it due to quality problems? Rather than just estimating perceived audiovisual quality, future quality prediction models will also factor in possible user behavior. This contribution presents a novel test method to assess short-term user behavior in web video services, in a controlled living-room-like environment. We show that typical behavioral responses (such as seeking, reloading, or selecting another video) can be elicited, with the real purpose of the test hidden from the viewers. We can also see that when users are not focused on judging quality, their perception of errors changes significantly. This paper highlights the strong impact of laboratory test situations on users' behavior and discusses the challenges revolving around finding valid test methods.