The rise in the consumption of multimedia content has resulted in the demand to provide an exceptional user experience. However, modelling user-perceived Quality of Experience (QoE) presents a complex challenge. This fuels ongoing efforts to understand and measure QoE. To assess it, researchers rely on both subjective quality assessment (where users view and rate videos) and objective quality assessment (where quality metrics are designed to calculate perceived video quality). Although subjective evaluation is essential for mapping user experience to objective metrics, many studies omit the analysis of the impact of content preferences on user experience. This oversight limits our understanding of how video content influences QoE perception.To address this limitation, we created a multi-content video dataset with impairments based on realistic network conditions. Furthermore, we conducted a subjective study in a controlled environment evaluating the impact of user content category preferences and user video preference on QoE. One of our initial findings is that the actual video denoted as uninteresting by users had the most negative impact on the overall user QoE, but had no impact on user perception of other video degradations.

The continuous rise of multimedia entertainment has led to an increased demand for delivering outstanding user experience of multimedia content. However, modeling user-perceived Quality of Experience (QoE) is a challenging task, resulting in efforts for better understanding and measurement of user-perceived QoE. To evaluate user QoE, subjective quality assessment, where people watch and grade videos, and objective quality assessment in which videos are graded using one or many objective metrics are conducted. While there is a plethora of video databases available for subjective and objective video quality assessment, these videos are artificially infused with various temporal and spatial impairments. Videos being assessed are artificially distorted with startup delay, bitrate changes, and stalls due to rebuffering events. To conduct a more credible quality assessment, a reproduction of original user experiences while watching different types of streams on different types and quality of networks is needed. To aid current efforts in bridging the gap between the mapping of objective video QoE metrics to user experience, we developed DashReStreamer, an open source framework for re-creating adaptively streamed video in real networks. The framework takes inputs in the form of video logs captured by the client in a non-regulated setting, along with an .mpd file or a YouTube URL. The ultimate result is a video sequence that encompasses all the data extracted from the video log. DashReStreamer also calculates popular video quality metrics like PSNR, SSIM, MS-SSIM, and VMAF. Finally, DashReStreamer allows creating impaired video sequences from the popular streaming platform YouTube. As a demonstration of framework usage, we created a database of 332 realistic video clips, based on video logs collected from real mobile and wireless networks. Every video clip is supplemented with bandwidth trace and video logs used in its creation and also with objective metrics calculation reports. In addition to dataset, we performed subjective evaluation of video content, assessing its effect on overall user QoE. We believe that this dataset and framework will allow the research community to better understand the impacts of video QoE dynamics.

Different industries are observing the positive impact of 360 video on the user experience. However, the performance of VR systems continues to fall short of customer expectations. Therefore, more research into various design elements for VR streaming systems is required. This study introduces a SW tool that offers straight-forward encoding platforms to simplify the encoding of DASH VR videos. In addition, we developed a dataset composed of 9 VR videos encoded with seven tiling configurations, four segment durations, and up to four different bitrates. A corresponding tile size dataset is also provided, which can be utilised to power network simulations or trace-driven emulations. We analysed the traffic load of various films and encoding setups using the dataset that was presented. Our research indicates that, while smaller tile sizes reduce traffic load, video decoding may require more computational power.

Multimedia streaming over the Internet (live and on demand) is the cornerstone of modern Internet carrying more than 60% of all traffic. With such high demand, delivering outstanding user experience is a crucial and challenging task. To evaluate user Quality of Experience (QoE) many researchers deploy subjective quality assessments where participants watch and rate videos artificially infused with various temporal and spatial impairments. To aid current efforts in bridging the gap between the mapping of objective video QoE metrics to user experience, we developed DashReStreamer, an open-source framework for re-creating adaptively streamed video in real networks. DashReStreamer utilises a log created by a HTTP adaptive streaming (HAS) algorithm run in an uncontrolled environment (i.e., wired or wireless networks), encoding visual changes and stall events in one video file. These videos are applicable for subjective QoE evaluation mimicking realistic network conditions.

Fifth Generation (5G) networks provide high throughput and low delay, contributing to enhanced Quality of Experience (QoE) expectations. The exponential growth of multimedia traffic pose dichotomic challenges to simultaneously satisfy network operators, service providers, and end-user expectations. Building QoE-aware networks that provide run-time mechanisms to satisfy end-users' expectations while the end-to-end network Quality of Service (QoS) varies is challenging, and motivates many ongoing research efforts. The contribution of this work is twofold. Firstly, we present a reproducible data-driven framework with a series of pre-installed Dynamic Adaptive Streaming over HTTP (DASH) tools to analyse state-of-art Adaptive Bitrate Streaming (ABS) algorithms by varying key QoS parameters in static and mobility scenarios. Secondly, we introduce an interactive Jupyter notebook and Binder service providing a live analytical environment, which processes the output dataset of the framework and compares the relationship of five QoE models, three QoS parameters (RTT, throughput, packets), and seven different video KPIs.

In this paper, we present a 5G trace dataset collected from a major Irish mobile operator. The dataset is generated from two mobility patterns (static and car), and across two application patterns (video streaming and file download). The dataset is composed of client-side cellular key performance indicators (KPIs) comprised of channel-related metrics, context-related metrics, cell-related metrics and throughput information. These metrics are generated from a well-known non-rooted Android network monitoring application, G-NetTrack Pro. To the best of our knowledge, this is the first publicly available dataset that contains throughput, channel and context information for 5G networks. To supplement our real-time 5G production network dataset, we also provide a 5G large scale multi-cell ns-3 simulation framework. The availability of the 5G/mmwave module for the ns-3 mmwave network simulator provides an opportunity to improve our understanding of the dynamic reasoning for adaptive clients in 5G multi-cell wireless scenarios. The purpose of our framework is to provide additional information (such as competing metrics for users connected to the same cell), thus providing otherwise unavailable information about the base station (eNodeB or eNB) environment and scheduling principle, to end user. Our framework permits other researchers to investigate this interaction through the generation of their own synthetic datasets.

In this short paper, we present goDASH, an infrastructure for headless streaming of HTTP adaptive streaming (HAS) video content, implemented in the language golang, an open-source programming language supported by Google. goDASH's main functionality is the ability to stream HAS content without decoding actual video (headless player). This results in low memory requirements and the ability to run multiple players in a large-scale-based evaluation setup. goDASH comes complete with numerous state-of-the-art HAS algorithms, and is fully written in the Google golang language, which simplifies the implementation of new adaptation algorithms and functions. goDASH supports two transportation protocols Transmission Control Protocol (TCP) and Quick UDP Internet Connections (QUIC). The QUIC protocol is a relatively new protocol with the promise of performance improvement over the widely used TCP. We believe that goDASH is the first emulation-based HAS player that supports QUIC. The main limitation in using QUIC protocol is the need for a security certificate setup on both ends (client and server) as QUIC demands an encrypted connection. This limitation is eased by providing our own testbed framework, known as goDASHbed. This framework uses a virtual environment to serve video content locally (which allows setting security certificates) through the Mininet virtual emulation tool. As part of Mininet, goDASH can be used in conjunction with other traffic generators.

The growth of online video-on-demand consumption continues unabated. Existing heuristic-based adaptive bit-rate (ABR) selection algorithms are typically designed to optimise video quality within a very narrow context. This may lead to video streaming providers implementing different ABR algorithms/players, based on a network connection, device capabilities, video content, etc., in order to serve the multitude of their users' streaming requirements. In this paper, we present SMASH: a Supervised Machine learning approach to Adaptive Streaming over HTTP, which takes a tentative step towards the goal of a one-size-fits-all approach to ABR. We utilise the streaming output from the adaptation logic of nine ABR algorithms across a variety of streaming scenarios (generating nearly one million records) and design a machine learning model, using systematically selected features, to predict the optimal choice of the bitrate of the next video segment to download. Our evaluation results show that SMASH guarantees a high QoE with consistent performance across a variety of streaming contexts.

In this short demo paper, we introduce godash 2.0 godash is a headless HTTP adaptive streaming (HAS) video streaming platform written in the Google programming language GO. godash has been extensively rewritten for this release so as to provide ease of use, and a host of new features. godash includes options for eight different state of the art adaptive algorithms, five HAS profiles, four video codecs, the ability to stream audio and video segments, two transport protocols (TCP and QUIC), realtime output from five Quality of Experience (QoE) models, as well as a collaborative framework for the evaluation of cooperative HAS streaming. In this demo, we will introduce each of the options available in the godash configuration file, and illustrate examples of how to use the collaborative players using both godash on a native machine as well as godash within godashbed. godashbed is an integrated large-scale testbed framework for the evaluation of HAS streaming, which uses a virtual environment to serve video content locally (which allows setting security certificates) through the Mininet virtual emulation tool. In this manner, godash provides a framework for rapid deployment and testing of new HAS algorithms, QoE models and transport protocols.

As we advance towards smart cities, autonomous vehicles and the avalanche of IoT devices proposed for the future, we need to give careful consideration to how easily compromised devices can impact network state. Current proposals for devices typically use cellular networks as the backhaul or final hop. These devices will leverage existing trust-based client-side channel metrics, such as Channel Quality Indicator (CQI), when the base-station determines scheduling decisions. In this paper, we investigate the scheduling impact of a malicious device when it changes its channel metrics, so as to improve its download rate or even to negate the download rate of others. We utilise real-time 4K ultra-high definition video delivery as an example of high throughput demand application and compare the delivery rates of multiple devices in an open-source 5G simulated NS-3 network. Our results illustrate that when a malicious client deceives the scheduler, the other clients in the network have a noticeable decrease in both viewable quality and underlying delivery rate (25% decrease in the average video quality across the non malicious clients).

As the next generation of cellular networks (5G) nears deployment, autonomous smart nodes, such as Internet of Things (IoT) and vehicular devices, are being developed to take advantage of increased throughput and improved resiliency offered by the new link layer protocols of 5G. Evaluating the impact of real-time 5G air-interface scheduling algorithms on the stringent real-time delivery needs of these devices and their new applications is no small feat. In this short paper, we expand upon the mmwave (5G) module for NS-3 and present DI5GUISE, a configurable and highly dynamic real-time simulated testbed framework, through which the impact of these smart nodes can be investigated. We utilise real-time video delivery as an example of applications with high throughput requirements. The experimental results illustrate that even on low cost commodity hardware, such as Raspberry Pis, over 80Mbps per client of real-time streaming of 4K content can be achieved.

Recent years have witnessed an explosion of multimedia traffic carried over the Internet. Video-on-demand and live streaming services are the most dominant services. To ensure growth, many streaming providers have invested considerable time and effort to keep pace with ever-increasing users' demand for better quality and stall abolition. HTTP adaptive streaming (HAS) algorithms are at the core of every major streaming provider service. Recent years have seen sustained development in HAS algorithms. Currently, to evaluate their proposed solutions, researchers need to create a framework and numerous state-of-the-art algorithms. Often, these frameworks lack flexibility and scalability, covering only a limited set of scenarios. To fill this gap, in this paper we propose DASHbed, a highly customizable real-time framework for testing HAS algorithms in a wireless environment. Due to its low memory requirement, DASHbed offers a means of running large-scale experiments with a hundred competing players. Finally, we supplement the proposed framework with a dataset consisting of results for five HAS algorithms tested in various evaluated scenarios. The dataset showcases the abilities of DASHbed and presents the adaptation metrics per segment in the generated content (such as switches, buffer-level, P. 1203.1 values, delivery rate, stall duration, etc.), which can be used as a baseline when researchers compare the output of their proposed algorithm against the state-of-the-art algorithms.

In this paper we present a Multi-Profile Ultra High Definition (UHD) DASH dataset composed of both AVC (H.264) and HEVC (H.265) video content, generated from three well known open-source 4K video clips. The representation rates and resolutions of our dataset range from 40Mbps in 4K down to 235kbps in 320x240, and are comparable to rates utilised by on demand services such as Netflix, Youtube and Amazon Prime. We provide our dataset for both realtime testbed evaluation and trace-based simulation. The real-time testbed content provides a means of evaluating DASH adaptation techniques on physical hardware, while our trace-based content offers simulation over frameworks such as ns-2 and ns-3. We also provide the original pre-DASH MP4 files and our associated DASH generation scripts, so as to provide researchers with a mechanism to create their own DASH profile content locally. Which improves the reproducibility of results and remove re-buffering issues caused by delay/jitter/losses in the Internet.

In this paper, we present a 4G trace dataset composed of client-side cellular key performance indicators (KPIs) collected from two major Irish mobile operators, across different mobility patterns (static, pedestrian, car, bus and train). The 4G trace dataset contains 135 traces, with an average duration of fifteen minutes per trace, with viewable throughput ranging from 0 to 173 Mbit/s at a granularity of one sample per second. Our traces are generated from a well-known non-rooted Android network monitoring application, G-NetTrack Pro. This tool enables capturing various channel related KPIs, context-related metrics, downlink and uplink throughput, and also cell-related information. To the best of our knowledge, this is the first publicly available dataset that contains throughput, channel and context information for 4G networks.

The dramatic growth of video traffic represents a practical challenge for cellular network operators in providing a consistent streaming Quality of Experience (QoE) to their users. Satisfying this objective has so-far proved elusive, due to the inherent characteristics of wireless networks and varying channel conditions as well as variability in the video bitrate that can degrade streaming performance. In this article, we propose stall-aware pacing as a novel MPEG DASH video traffic management solution that reduces playback stalls and seeks to maintain a consistent QoE for cellular users, even those with diverse channel conditions. These goals are achieved by leveraging both network and client state information to optimize the pacing of individual video flows. We evaluate the performance of two versions of stall-aware pacing techniques extensively, including stall-aware pacing (SAP) and adaptive stall-aware pacing (ASAP), using real video content and clients, operating over a simulated LTE network. We implement state-of-the-art client adaptation and traffic management strategies for direct comparisons with SAP and ASAP. Our results, using a heavily loaded base station, show that SAP reduces the number of stalls and the average stall duration per session by up to 95%. Additionally, SAP ensures that clients with good channel conditions do not dominate available wireless resources, evidenced by a reduction of up to 40% in the standard deviation of the QoE metric across clients. We also show that ASAP achieves additional performance gains by adaptively pacing video streams based on the application buffer state.

The dramatic growth of cellular video traffic represents a practical challenge for cellular network operators in providing a consistent streaming Quality of Experience (QoE) to their users. Satisfying this objective has so-far proved elusive, due to the inherent system complexities that degrade streaming performance, such as variability in both video bitrate and network conditions. In this paper, we present SAP as a DASH video traffic management solution that reduces playback stalls and seeks to maintain a consistent QoE for cellular users, even those with diverse channel conditions. SAP achieves this by leveraging both network and client state information to optimize the pacing of individual video flows. We extensively evaluate SAP performance using real video content and clients, operating over a simulated LTE network. We implement state-of-the-art client adaptation and traffic management strategies for direct comparison. Our results, using a heavily loaded base station, show that SAP reduces the number of stalls and the average stall duration per session by up to 95%. Additionally, SAP ensures that clients with good channel conditions do not dominate available wireless resources, evidenced by a reduction of up to 40% in the standard deviation of the QoE metric.

In this demonstration we present a platform that encompasses all of the components required to realistically evaluate the performance of Dynamic Adaptive Streaming over HTTP (DASH) over a real-time NS-3 simulated network. Our platform consists of a network-attached storage server with DASH video clips and a simulated LTE network which utilises the NS-3 LTE module provided by the LENA project. We stream to clients running an open-source player with a choice of adaptation algorithms. By providing a user interface that offers user parametrisation to modify both client and LTE settings, we can view the evaluated results of real-time interactions between the network and the clients. Of special interest is that our platform streams actual video clips to real video clients in real-time over a simulated LTE network, allowing reproducible experiments and easy modification of LTE and client parameters. The demonstration showcases how changes in LTE network settings (fading model, scheduler, client distance from eNB, etc.), as well as video-related decisions at the clients (streaming algorithm, quality selection, clip selection, etc.), can impact on the delivery and achievable quality.

Dynamic adaptive video over HTTP (DASH) is fast becoming the protocol of choice for content providers for their online video streaming delivery. Concurrently, dependence on cellular Long Term Evolution (LTE) networks is growing to serve user demands for bandwidth-hungry applications, especially video. Each LTE base station's (eNodeB) scheduler assigns wireless resources to individual clients. Several alternative schedulers have been proposed, especially to meet the user's desired quality of experience (QoE) with video. In this paper, we investigate the impact of the scheduler on DASH performance, motivated by the fact that video performance and the underlying traffic models are different from other HTTP/TCP applications. We use our laboratory testbed employing real video content and streaming clients, over a simulated ns-3 LTE network. We quantify the impact of the scheduler and show that it has a significant impact on key video streaming performance metrics such as stalls and QoE, for different client adaptation algorithms. Additionally, we show the impact of user mobility within a cell, which has the side-effect of improving performance by mitigating long-term fading effects. Our detailed assessment of four LTE schedulers in ns-3 shows that the proportional fair scheduler achieves the best overall user experience, although somewhat disadvantaging static cell-edge users.

In this paper we present datasets for both trace-based simulation and real-time testbed evaluation of Dynamic Adaptive Streaming over HTTP (DASH). Our trace-based simulation dataset provides a means of evaluation in frameworks such as NS-2 and NS-3, while our testbed evaluation dataset offers a means of analysing the delivery of content over a physical network and associated adaptation mechanisms at the client. Our datasets are available in both H.264 and H.265 with encoding rates comparative to the representations and resolutions of content distribution providers such as Netflix, Hulu and YouTube.

HTTP streaming currently dominates Internet traffic. It is increasingly common that video players employ adaptive bitrate (ABR) streaming strategies to maximise the user experience by selecting the highest video representation while targeting stall-free playback. Our interest lies in the common situation where a set of video flows are competing for access to a shared bottleneck link, such as in a cellular radio access network. We observe that ISPs (e.g. cellular operators) are considering innetwork techniques for resource allocation and sharing among different users. Buoyed by the ability of software defined networks (SDN) to offer flow-specific control and traffic shaping, we focus on traffic shaping techniques, and experimentally analyse the effect on ABR video flows when sharing a bottleneck link. We conduct experiments using the GPAC video player operating over a Mininet virtual network. We conclude that traffic shaping can allow a balance of fairness, efficiency and quality. Traffic shaping ABR videos reduce the number of stalls and quality switches, while also reducing the peaks for the aggregate network traffic.

Bandwidth constriction and datagram loss are prominent issues that affect the perceived quality of streaming video over lossy networks, such as wireless. The use of layered video coding seems attractive as a means to alleviate these issues, but its adoption has been held back in large part by the inherent priority assigned to the critical lower layers and the consequences for quality that result from their loss. The proposed use of forward error correction (FEC) as a solution only further burdens the bandwidth availability and can negate the perceived benefits of increased stream quality.