The evolving landscape of the delivery of multimedia content requires a deep understanding of how the design of the bitrate ladder (for HTTP adaptive streaming) influences cost and quality. This paper explores the use of Just Noticeable Differences (JND) to select bitrate-resolution pairs for constructing a bitrate ladder with respect to the proportion of satisfied user ratio (SUR). To expand the investigation to various codecs, first, a method is explained that transfers the JND points obtained through subjective testing from one codec (e.g., , AVC) to other codecs (e.g., , HEVC, VVC). This approach helps avoid the additional costs associated with conducting subjective tests to obtain JND points for a wide range of different codecs. To achieve this objective, we investigate the codec-agnostic nature of various video quality metrics, followed by the transfer of JND between two codecs, taking into account the most suitable codec-agnostic video quality metric. Secondly, we delve into the analysis of the bitrate cost of a given bitrate ladder from a JND perspective, i.e., , as a function of the SUR. Among others, our experimental results demonstrate that increasing SUR leads to an exponential increase in bitrate. For example, to raise the SUR from 75% to 90%, it is necessary to double the video bitrate.

Just Noticeable Difference (JND) establishes the threshold between two images or videos wherein differences in quality remain imperceptible to an individual. This threshold, collectively known as the Satisfied User Ratio (SUR), holds significant importance in image and video compression applications, ensuring that differences in quality are imperceptible to the majority (p%) of users, known as p%SUR. While substantial efforts have been dedicated to predicting the p%SUR for various encoding parameters (e.g., QP) and quality metrics (e.g., VMAF), referred to as proxies, systematic consideration of the prediction uncertainties associated with these proxies has hitherto remained unexplored. In this paper, we analyze the uncertainty of p%SUR through Confidence Interval (CI) estimation and assess the consistency of various Video Quality Metrics (VQMs) as proxies for SUR. The analysis reveals challenges in directly using p%SUR as ground truth for training models and highlights the need for uncertainty estimation for SUR with different proxies.

Virtual Reality (VR) promises to benefit training and education by offering ubiquitous risk-free access to handson experiential learning. This paper presents an evaluation of a VR-based occupational safety training application for forestry work, assessing the VR system’s impact on training effectiveness, including skill transfer to real world settings. 73 participants experienced two learning scenarios covering different task types and settings. They were either assigned to the VR training or relied on traditional forestry training materials only.Our results confirm the positive effects of VR-based training on participants’ experience as well as successful skill transfer to real-world challenges, as evidenced by participant feedback on a broad range of experience measures. In addition, our study extends existing work by examining how varying VR training quality affects both subjective experience and objective performance outcomes in terms of learning transfer. Our findings indicate that better technical quality of the VR training indeed can translate to significantly improved performance during skill application in subsequent test situations. Surprisingly, we could not detect a similarly pronounced effect when analyzing participants’ subjective experience ratings. We reflect on these findings and discuss methodological implications for experience assessment in the context of VR-based training and education.

In adaptive video streaming, optimizing the selection of representations for the encoding bitrate ladder has a significant impact on the quality and economics of media delivery. An efficient way to select representations for the bitrate ladder of a given clip is to consider the Satisfied User Ratio (SUR) of the perceived quality of consecutive representations. This ensures that only representations with one Just Noticeable Difference (JND) are encoded and streamed by avoiding encoding similar-quality representations. VMAF (Video Multi-method Assessment Fusion) presently stands as the most commonly utilized quality metric for constructing bitrate ladders. Hence, the precise determination of JND-optimal encoding step-sizes for the VMAF proxy holds paramount importance; nevertheless, this task is intricate and can present considerable challenges.In this paper, we evaluate the effectiveness of different Video Quality Metrics (VQMs) in predicting SUR for the VMAF proxy to better capture content-specific characteristics. Our experimental results provide evidence that incorporating VQMs can improve the precision of the SUR prediction for the VMAF proxy. Compared to a state-of-the-art approach that utilizes video complexity metrics, our proposed approach, which incorporates two quality metrics—specifically, VMAF and SSIM calculated at an optimized quantization parameter (QP)—achieves a substantially reduced Mean Absolute Error (MAE) of 1.67. In contrast, the state-of-the-art approach yields an MAE of 2.01. Hence, we recommend using the above quality metrics to improve the accuracy of the SUR prediction for the VMAF proxy.

We introduce DeepCrypt, a deep-learning based approach to analyze YouTube adaptive video streaming Quality of Experience (QoE) from the Internet Service Provider (ISP) perspective, relying exclusively on the analysis of encrypted network traffic. Using raw features derived on-line from the encrypted stream of bytes, DeepCrypt infers six different video QoE indicators capturing the user-perceived performance of the service, including the initial playback delay, the number and frequency of rebuffering events, the video playback quality and encoding bitrate, and the number of quality changes. DeepCrypt offers deep visibility into the behavior of the end-user, enabling the fingerprinting and detection of different user actions on the video player, such as video pauses and playback scrubbing (forward, backward, out-of-buffer), offering a complete visibility on the video streaming process from in-network traffic measurements. Evaluations over a large and heterogeneous dataset composed of mobile and fixed-line measurements, using the YouTube HTML5 player, the native YouTube mobile app, as well as a generic HTML5 video player built on top of open source libraries, and considering measurements collected at different ISPs, confirm the out-performance of DeepCrypt over previously used shallow-learning models, and its generalization to different video players and network setups.

We currently witness the rapidly growing importance of intelligent video streaming quality optimization and reduction of video delivery costs. Per-Title encoding, in contrast to a fixed bitrate ladder, shows significant promise to deliver higher quality video streams by addressing the trade-off between compression efficiency and video characteristics such as resolution and frame rate. Selecting encodings with noticeable quality differences in between prevents the construction of an inefficient bitrate ladder that suffers from too similar quality representations. In this respect, the VMAF metric represents a promising foundation for bitrate laddering, as it currently yields the highest video quality prediction performance. However, the minimum noticeable quality difference, referred as to just-noticeable-difference (JND), has not been properly validated for VMAF yet, with existing sources proposing highly diverse ΔVMAF step sizes ranging from two [1] to six [2]. This paper investigates JND for VMAF to facilitate the efficient construction of content-specific bitrate ladders. Using a publicly available JND multimedia dataset, we invalidate existing step size rules of thumb by revealing the large errors they introduce when applied to real-world video content. We investigate different factors that influence the actual ΔVMAF step sizes required to obtain correct JNDs and develop a model using content features that more accurately determines the optimal quality step sizes required for properly laddering individual clips.

Training in virtual reality (VR) is widely considered as promising approach to address the rapidly rising demand for adequately trained healthcare professionals. In this paper, we present an experimental study assessing the experience and learning effects of a VR-based training for healthcare professionals (scenario: preparation of an endotracheal intubation). 41 participants from the field of healthcare and nursing were randomly assigned to an intervention group training in VR and a control group training with traditional printed materials. Subsequently, both groups had to apply their newly acquired skills in a real-world practice environment to objectively assess training effectiveness at high validity. Our results demonstrate the added value of the VR training in terms of trainees' significantly reduced time to solve the task in the real-world environment. Moreover, statistical analysis of experience measures (hedonic and pragmatic user experience, sense of presence, VR-induced symptoms, quality of experience, overall satisfaction) confirms participants' appreciation of the quality of the VR-based training. Surprisingly, however, our results do not indicate a significant link between VR training experience and real-world task performance. We discuss implications of these findings and outline a research agenda to better understand the key factors that influence the effectiveness of VR-based training.

Providing sophisticated web Quality of Experience (QoE) has become paramount for web service providers and network operators alike. Due to advances in web technologies (HTML5, responsive design, etc.), traditional web QoE models focusing mainly on loading times have to be refined and improved. In this work, we relate Google's Core Web Vitals, a set of metrics for improving user experience, to the loading time aspects of web QoE. To this end, we first perform objective measurements in the web using Google's Lighthouse. To close the gap between metrics and experience, we complement these objective measurements with subjective assessment by performing multiple crowdsourcing QoE studies. In these studies, we use CWeQS, a customized framework to emulate the entire web page loading process, and ask users for their experience while controlling the Core Web Vitals. Our results suggest that the Core Web Vitals have less predictive value for web QoE than expected and that page loading times remain the main influence factor in this context.

Due to the growing importance of optimizing the quality and efficiency of video streaming delivery, accurate assessment of user-perceived video quality becomes increasingly important. However, due to the wide range of viewing distances encountered in real-world viewing settings, the perceived video quality can vary significantly in everyday viewing situations. In this paper, we investigate and quantify the influence of viewing distance on perceived video quality. A subjective experiment was conducted with full HD sequences at three different fixed viewing distances, with each video sequence being encoded at three different quality levels. Our study results confirm that the viewing distance has a significant influence on the quality assessment. In particular, they show that an increased viewing distance generally leads to increased perceived video quality, especially at low media encoding quality levels. In this context, we also provide an estimation of potential bitrate savings that knowledge of actual viewing distance would enable in practice. Since current objective video quality metrics do not systematically take into account viewing distance, we also analyze and quantify the influence of viewing distance on the correlation between objective and subjective metrics. Our results confirm the need for distance-aware objective metrics when the accurate prediction of perceived video quality in real-world environments is required.

Although software products are becoming increasingly accessible and assistive tools like screen readers becoming widely available, people with low vision still face insufficient support when it comes to consumption of digital video content. In this paper, we present an accessible desktop video player software, which allows people with low vision to adapt the presentation of digital videos according to their specific needs. For visual enhancement, we implemented a broad range of image manipulation techniques, like adaptation of contrast, color manipulation (e.g. inverting, grey scale transformations) and edge detection algorithms for sharpness optimization. Based on the feedback from low vision users, we discuss how to implement enhancement filter configuration and how to consider several input modalities.

Large-scale events pose severe challenges to live video streaming service providers, who need to cope with high, peaking viewer numbers and the resulting fluctuating resource demands, keeping high levels of Quality of Experience (QoE) to avoid end-user frustration and churn. In this paper, we analyze a unique dataset consisting of more than a million 2018 FIFA World Cup mobile live streaming sessions, collected at a large national public broadcaster. Different from previous work, we analyze QoE and user engagement as well as their interaction, in dependency to specific soccer match events, which have the potential to trigger flash crowds during a match. Flash crowds are a particular challenge to video service providers, since they cause sudden load peaks and consequently, the likelihood of quality problems. We further exploit the data to model viewer engagement over the course of a soccer match, and show that client counts follow very similar patterns of change across all matches. We believe that the analysis as well as the resulting models are valuable sources of insight for service providers, equipping them with tools for customer-centric resource and capacity management.

Volumetric media has the potential to provide the six degrees of freedom (6DoF) required by truly immersive media. However, achieving 6DoF requires ultra-high bandwidth transmissions, which real-world wide area networks cannot provide today. Therefore, recent efforts have started to target efficient delivery of volumetric media, using a combination of compression and adaptive streaming techniques. It remains, however, unclear how the effects of such techniques on the user perceived quality can be accurately evaluated. In this paper, we present the results of an extensive objective and subjective quality of experience (QoE) evaluation of volumetric 6DoF streaming. We use PCC-DASH, a standards-compliant means for HTTP adaptive streaming of scenes comprising multiple dynamic point cloud objects. By means of a thorough analysis, we investigate the perceived quality impact of the available bandwidth, rate adaptation algorithm, viewport prediction strategy and user's motion within the scene. We determine which of these aspects has more impact on the user's QoE, and to what extent subjective and objective assessments are aligned.

Telepresence robots (TPRs) are versatile, remotely controlled vehicles that enable physical presence and human-to-human interaction over a distance. Thanks to improving hardware and dropping price points, TPRs enjoy the growing interest in various industries and application domains. Still, a satisfying experience remains key for their acceptance and successful adoption, not only in terms of enabling remote communication with others, but also in terms of managing robot mobility by means of remote navigation. This paper focuses on the latter aspect of remote operation which has been hitherto neglected. We present the results of an extensive subjective study designed to systematically assess remote navigation Quality of Experience (QoE) in the context of using a TPR live over the Internet. Participants were ‘beamed’ into a remote office space and asked to perform characteristic TPR remote operation tasks (driving, turning, parking). Visual and control dimensions of their experience were systematically impaired by altering network characteristics (bandwidth, delay and packet loss rate) in a controlled fashion. Our results show that users can differentiate well between visual and navigation/control aspects of their experience. Furthermore, QoE impairment sensitivity varies with the actual task at hand.

Current developments in digitized industrial manufacturing lead to work processes that are highly complex, frequently change and thus increase workload of staff. In this respect, assistive technologies have the potential to aid workers in better coping with increasing workload and requirements and consequently, maintaining good working experience(s). However, existing research on interactive assistive technologies has hitherto largely neglected work experience as a measurable construct. We addressed this shortcoming by conducting a subjective study that assesses different work experience dimensions in the context of interactive augmented reality assembly instructions, in a realistic industrial setting, with a decent sample size (N=48). In this study, we assessed work experience along the dimensions of perceived utility, hedonic user experience, and satisfaction with the assistive technologies used. On behalf of this approach we were able to detect a significant effect between perceived utility and satisfaction. In addition, we summarize lessons learned from this study as well as resulting implications and discuss open challenges for a research agenda on work experience in industry 4.0 contexts.

In this paper, we investigate the impact of real-time video enhancement techniques on video Quality of Experience (QoE) for users with low vision. To this end, we present the results of a lab-based user study, in which participants with low vision utilize a web-based media player to configure individual video quality enhancement filters (e.g. increased contrast, strong edge enhancement and lowered brightness) depending on types of content. Additionally, for each test participant we measured the visual performance to describe how low vision impacts video QoE. Our results show, that individual video quality filters positively impact the experience of the quality dimensions detail, text and content. Furthermore, we discuss insights from qualitative feedback from participants regarding overall utility and User Experience (UX) aspects of real-time video enhancements.

Omnidirectional video (ODV) streaming applications are becoming increasingly popular. They enable a highly immersive experience as the user can freely choose her/his field of view within the 360-degree environment. Current deployments are fairly simple but viewport-agnostic which inevitably results in high storage/bandwidth requirements and low Quality of Experience (QoE). A promising solution is referred to as tile-based streaming which allows to have higher quality within the user's viewport while quality outside the user's viewport could be lower. However, empirical QoE assessment studies in this domain are still rare. Thus, this paper investigates the impact of different tile-based streaming approaches and configurations on the QoE of ODV. We present the results of a lab-based subjective evaluation in which participants evaluated 8K omnidirectional video QoE as influenced by different (i) tile-based streaming approaches (full vs. partial delivery), (ii) content types (static vs. moving camera), and (iii) tile encoding quality levels determined by different quantization parameters. Our experimental setup is characterized by high reproducibility since relevant media delivery aspects (including the user's head movements and dynamic tile quality adaptation) are already rendered into the respective processed video sequences. Additionally, we performed a complementary objective evaluation of the different test sequences focusing on bandwidth efficiency and objective quality metrics. The results are presented in this paper and discussed in detail which confirm that tile-based streaming of ODV improves visual quality while reducing bandwidth requirements.

Over the last years, QUIC (Quick UDP Internet Connections) has become the default protocol for networked communication of Google services, heralded as improved successor of the prevailing Transport Control Protocol (TCP). While the deployment of QUIC is increasing, QUIC is also planned to be the foundation of HTTP/3, the next generation of the HTTP protocols, which drive almost all applications on the Web. Given these developments, this paper aims to raise the awareness of the QoE research community to the increasing presence of QUIC, which likely brings implications for QoE monitoring and management of networked multimedia applications, as well as for the overall QoE research agenda. In particular, a major promise during the introduction of QUIC has been the improvement of the QoE of web-based applications (like browsing and video) by overcoming certain limitations and inefficiencies of TCP. In order to validate this claim, a measurement study was conducted to test whether the promised QoE benefits of QUIC are indeed noticeable for end users of streaming and browsing services. Surprisingly, no evidence for any QoE improvement of QUIC over TCP could be found. This way this paper aims to demonstrate how QoE research can and should successfully address relevant current and future developments on the Internet.

Experience assessment is a key activity for understanding and developing interactive systems and technologies. Virtual reality (VR) is no exception, considering its rising popularity and the increasing availability of affordable VR devices. However, in experience studies where participants typical have to rate multiple test conditions presented to them, having to repeatedly take off the head mounted display (HMD) just for the purpose of issuing ratings is cumbersome and breaks the immersion. As a solution we present VRate, a Unity3D asset to integrate subjective experience questionnaires directly in virtual reality environments. The suitability of VRate has been proven by successfully using the asset in a QoE and a UX study. We describe the technical implementation and suggest how to integrate the VRate in a study setup. VRate will be provided to the scientific community for research purposes, educational and noncommercial usage under a Creative Commons License.

In this paper, we evaluate the influence of different VR rendering styles on the QoE of users of a training application. To this end, we present the results of a lab-based QoE study featuring a virtual manufacturing training setting in which participants were asked to correctly allocate spare parts to mounting locations inside and outside a vehicle using an HTC Vive VR Setup. We evaluate how five different rendering styles (defined by rendering technique and shader used), ranging from point cloud to highly photo-realistic, affect different quality dimensions such as perceived visual quality, task difficulty, spatial orientation ability, and discomfort (including cyber-sickness). Our results show that while rendering style has a significant impact on QoE and other dimensions (like task completion time), a standard shader already can provide sufficient quality to support the provided training task and can even outperform rendering styles using more sophisticated techniques like ambient occlusion. Furthermore, we also discuss significant influence of user-related factors like previous experience with VR or gender as well as other related critical issues to be considered in VR training application and assessment study design.

The experience of web services depends on multiple factors such as the usability of the service itself as well as influences on Quality of Experience (QoE) like waiting times. Existing studies so far have investigated both experimental factors (usability and QoE) in isolation, thus it is not possible to investigate dependencies between the two. In this study, we manipulated the usability and QoE of a news web page in the same experiment. Participants had to solve 9 tasks (three usability level and three QoE level, all combinations). Data (N=44) from two different sessions with different QoE manipulations were recorded: Simulation of initial page display delay via 1) a complete overlay and 2) a semi-transparent overlay. Our preliminary results show that the usability manipulation resulted in significant effects along multiple dimensions (QoE, aesthetic, and usability). Furthermore, also the implementation of the QoE manipulation had an effect on, e.g., performance parameters. Surprisingly, our results reveal that participants seem to be unable to rate the constructs usability and QoE independently from each other. Even after adding pronounced explanations of the constructs to the introductory phase of the second session of the experiment, the results still show a strong correlation between the two constructs. We thus recommend for similar future experiments to check beforehand whether Quality of Experience and Usability can be reliably judged and distinguished by naive participants.

Monitoring the Quality of Experience (QoE) undergone by cellular network customers has become paramount for cellular ISPs, who need to ensure high quality levels to limit customer churn due to quality dissatisfaction. This paper tackles the problem of QoE monitoring, assessment and prediction in cellular networks, relying on end-user device (i.e., smart-phone) QoS passive traffic measurements and QoE crowdsourced feedback. We conceive different QoE assessment models based on supervised machine learning techniques, which are capable to predict the QoE experienced by the end user of popular smartphone apps (e.g., YouTube and Facebook), using as input the passive in-device measurements. Using a rich QoE dataset derived from field trials in operational cellular networks, we benchmark the performance of multiple machine learning based predictors, and construct a decision-tree based model which is capable to predict the per-user overall experience and service acceptability with a success rate of 91% and 98% respectively To the best of our knowledge, this is the first paper using end-user, in-device passive measurements and machine learning models to predict the QoE of smartphone users in operational cellular networks.

Currently, we witness dramatically increasing interest in immersive media technologies like Virtual Reality (VR), particularly in omnidirectional video (OV) streaming. Omnidirectional (also called 360-degree) videos are panoramic spherical videos in which the user can look around during playback and which therefore can be understood as hybrids between traditional movie streaming and interactive VR worlds. Unfortunately, streaming this kind of content is extremely bandwidth intensive (compared to traditional 2D video) and therefore, Quality of Experience (QoE) tends to deteriorate significantly in absence of continuous optimal bandwidth conditions. In this paper, we present a first approach towards subjective QoE assessment for omnidirectional video (OV) streaming. We present the results of a lab study on the QoE impact of stalling in the context of OV streaming using head-mounted displays (HMDs). Our findings show that subjective testing for immersive media like OV is not trivial, with even simple cases like stalling leading to unexpected results. After a discussion of characteristic pitfalls and lessons learned, we provide a a set of recommendations for upcoming OV assessment studies.

Today's packet-switched networks are subject to bandwidth fluctuations that cause degradation of the user experience of multimedia services. In order to cope with this problem, HTTP adaptive streaming (HAS) has been proposed in recent years as a video delivery solution for the future Internet and being adopted by an increasing number of streaming services, such as Netflix and Youtube. HAS enables service providers to improve users' quality of experience (QoE) and network resource utilization by adapting the quality of the video stream to the current network conditions. However, the resulting time-varying video quality caused by adaptation introduces a new type of impairment and thus novel QoE research challenges. Despite various recent attempts to investigate these challenges, many fundamental questions regarding HAS perceptual performance are still open. In this paper, the QoE impact of different technical adaptation parameters, including chunk length, switching amplitude, switching frequency, and temporal recency, are investigated. In addition, the influence of content on perceptual quality of these parameters is analyzed. To this end, a large number of adaptation scenarios have been subjectively evaluated in four laboratory experiments and one crowdsourcing study. A statistical analysis of the combined data set reveals results that partly contradict widely held assumptions and provide novel insights in perceptual quality of adapted video sequences, e.g., interaction effects between quality switching direction (up/down) and switching strategy (smooth/abrupt). The large variety of experimental configurations across different studies ensures the consistency and external validity of the presented results that can be utilized for enhancing the perceptual performance of adaptive streaming services.

A quarter of the world population will be using smartphones to access the Internet in the near future. In this context, understanding the quality of experience (QoE) of popular apps in such devices becomes paramount to cellular network operators, who need to offer high-quality levels to reduce the risks of customers churning for quality dissatisfaction. In this paper, we address the problem of QoE provisioning in smartphones from a double perspective, combining the results obtained from subjective laboratory tests with end-device passive measurements and QoE crowd-sourced feedback obtained in operational cellular networks. The study addresses the impact of both access bandwidth and latency on the QoE of five different services and mobile apps: YouTube, Facebook, Web browsing through Chrome, Google Maps, and WhatsApp. We evaluate the influence of both constant and dynamically changing network access conditions, tackling in particular the case of fluctuating downlink bandwidth, which is typical in cellular networks. As a main contribution, we show that the results obtained in the laboratory are highly applicable in the live scenario, as mappings track the QoE provided by users in real networks. We additionally provide hints and bandwidth thresholds for good QoE levels on such apps, as well as discussion on end-device passive measurements and analysis. The results presented in this paper provide a sound basis to better understand the QoE requirements of popular mobile apps, as well as for monitoring the underlying provisioning network. To the best of our knowledge, this is the first paper providing such a comprehensive analysis of QoE in mobile devices, combining network measurements with users QoE feedback in laboratory tests, and operational networks.

A quarter of the world population will be using smartphones to access the Internet in the near future. In this context, understanding the Quality of Experience (QoE) of popular apps in such devices becomes paramount to cellular network operators, who need to offer high quality levels to reduce the risks of customers churning for quality dissatisfaction. In this paper we address the problem of QoE provisioning in smartphones from a double perspective, combining the results obtained from subjective lab tests with end-device passive measurements and QoE crowd-sourced feedback obtained in operational cellular networks. The study addresses the impact of the downlink bandwidth on the QoE of three popular smartphone apps: YouTube, Facebook and Google Maps. As a main contribution, we show that the results obtained in the lab are highly applicable in the live scenario, as mappings track the QoE provided by users in real networks. We additionally provide hints and bandwidth thresholds for good QoE levels on such apps, as well as discussion on end-device passive measurements and analysis. The results presented in this paper provide a sound basis to better understand the QoE requirements of popular mobile apps, as well as for monitoring the underlying provisioning network. To the best of our knowledge, this is the first paper providing such a comprehensive analysis of QoE in mobile devices, combining network measurements with users QoE feedback in lab tests and operational networks.

Mobile devices such as smartphones are taking over traditional devices for Internet access in today's scenario, and the near future forecast is overwhelming: by 2016, a quarter of the world population will be using smartphones to access the Internet. In this context, understanding the Quality of Experience (QoE) of popular services in mobile devices becomes paramount for cellular network operators, who need to offer high quality levels to reduce the risks of customers churning for quality dissatisfaction. In this paper we study the problem of QoE provisioning in mobile devices, presenting the results obtained from subjective lab tests performed for popular end-user services accessed through smartphones. Our analysis addresses the impact of access downlink bandwidth on the QoE of four different popular services and mobile apps: Facebook, Web browsing through Chrome, Google Maps, and WhatsApp. The study also considers the characterization of WhatsApp QoE in a real setting, mapping the lab results to large-scale measurements conducted in a major cellular network. The results presented in this paper provide a sound basis for better understanding the QoE requirements of popular services and mobile apps, as well as for dimensioning the underlying provisioning network. To the best of our knowledge, this is the first paper combining QoE lab-test results for mobile devices with large-scale measurements in an operational cellular network.

YouTube is the most popular service in the Internet and is increasingly consumed on mobile devices. With emerging adaptive video streaming technology, the question arises whether it should be also employed in the mobile context, which shows different characteristics in terms of display sizes and reliability of Internet connection. This paper compares YouTube QoE on mobile devices for both classical and adaptive video streaming based on a subjective lab experiment, in which different network conditions were emulated. Our results show that adaptive video streaming provides almost excellent results for the poorest network conditions. Thereby, it clearly outperforms classical video streaming, and thus, should be considered to achieve higher QoE in future mobile streaming applications.

The performance of YouTube in cellular networks is crucial to network operators, who try to find a trade-off between cost-efficient handling of the huge traffic amounts and high perceived end-user Quality of Experience (QoE). In this paper we present YoMoApp (YouTube Performance Monitoring Application), an Android application which passively monitors key performance indicators (KPIs) of YouTube adaptive video streaming on end-user smartphones. The monitored KPIs (i.e., player state/events, re-buffering, and video quality levels) can be used to analyze the QoE of mobile YouTube video sessions. YoMoApp is a valuable tool to assess the performance of cellular networks with respect to YouTube traffic, as well as to develop optimizations and QoE models for mobile HTTP adaptive streaming. We try YoMoApp through real subjective QoE lab tests showing that the tool is accurate to capture the experience of end-users watching YouTube on smartphones.

Since its introduction a few years ago, the concept of `Crowdsourcing' has been heralded as highly attractive alternative approach towards evaluating the Quality of Experience (QoE) of networked multimedia services. The main reason is that, in comparison to traditional laboratory-based subjective quality testing, crowd-based QoE assessment over the Internet promises to be not only much more cost-effective (no lab facilities required, less cost per subject) but also much faster in terms of shorter campaign setup and turnaround times. However, the reliability of remote test subjects and consequently, the trustworthiness of study results is still an issue that prevents the widespread adoption of crowd-based QoE testing. Various ideas for improving user rating reliability and test efficiency have been proposed, with the majority of them relying on a posteriori analysis of results. However, such methods introduce a major lag that significantly affects efficiency of campaign execution. In this paper we address these shortcomings by introducing in momento methods for crowdsourced video QoE assessment which yield improvements of results reliability by factor two and campaign execution efficiency by factor ten. The proposed in momento methods are applicable to existing crowd-based QoE testing approaches and suitable for a variety of service scenarios.

YouTube, the killer application of today's Internet, is changing the way ISPs and network operators manage quality monitoring and provisioning on their IP networks. YouTube is currently the most consumed Internet application, accounting for more than 30% of the overall Internet's traffic worldwide. Coupling such an overwhelming traffic volume with the ever intensifying competition among ISPs is pushing operators to integrate Quality of Experience (QoE) paradigms into their traffic management systems. The need for automatic QoE assessment solutions becomes even more critical in mobile broadband networks, where over-provisioning solutions can not be foreseen and bad user experience translates into churning clients. This paper presents a complete study on the problem of YouTube Quality of Experience monitoring and assessment in mobile networks. The paper considers not only the QoE analysis, modeling and assessment based on real users' experience, but also the passive monitoring of the quality provided by the ISP to its end-customers in a large mobile broadband network.

Over the last couple of years, the scope of Quality of Experience (QoE) research has been constantly extended, most recently to the field of Web QoE in the context of HTTP-based applications. In this paper, we address the question whether it is sufficient to reduce typical Web QoE assessment scenarios to the temporal aspects of waiting for task completion, which would allow to attribute the resulting logarithmic laws to well-known psychological insights on human time perception. We demonstrate that while this attribution is valid for simple waiting tasks which are typical for simple data services like e.g. file downloads, the case of interactive web browsing is much more complex. We show that this is not only because technical issues prevent bandwidth and download time from being directly correlated with each other in a simple manner, but also because user perceived web page load times strongly deviate from technical page load times. Consequently, existing approaches towards assessment and modeling of web browsing QoE have to be critically reviewed and redesigned.

In this paper we demonstrate how speech signal features can be used to detect and explain differences in human to human conversation tests. To this end, we compare the results of two conversational VoIP quality experiments designed to quantify the impact of network delay on perceived speech quality. Both studies followed the same procedures and used the same scenarios, but were conducted in two different labs. Our comparison shows that the two studies, despite having been executed correctly using the same test design, still can produce surprisingly different results regarding the users quality perception on a MOS scale. In this respect, speech signal features extracted from conversation recordings help identifying divergent participant behavior as plausible cause for such differences. Our in-depth analysis reveals how novel parameters developed by us like Intended and Unintended Interruption Rate (IIR, UIR) and the corrected Speaker Alternation Rate SARcorrcan be used to successfully determine the extent to which the results of different conversational speech quality studies are directly comparable and thus eligible for pooling, or not.

Due to the accelerating uptake of mobile broadband services, Quality of Experience (QoE) assessment and optimization for wireless networks has become a topic of vital importance. In this field, subjective quality evaluation methods based on MOS (Mean Opinion Score) are the most widely used approaches for understanding and measuring perceived performance. However, while these methods have been proven to reliably quantify users' degrees of satisfaction, they do not sufficiently address the following important question: "Which quality levels are acceptable to end users and which ones not?". In this paper we investigate the application of acceptability to the domain of QoE assessment for data services. To this end, we perform a critical review of the term's utilization and operationalization in related fields and discuss the results of a series of mobile broadband QoE studies we performed in lab and field settings. Our results demonstrate that acceptability is a relevant and useful concept that also features a consistent mapping between subjects' binary acceptance and ordinal satisfaction ratings, even across application scenarios as different as web browsing and file downloads. However, it is also a highly context-dependent concept since users' rating behaviors and thresholds diverge between lab and field settings. Acceptability thus needs to be evaluated in situ in the field in order to guarantee generalizable, ecologically valid results.

This paper addresses the challenge of assessing and modeling Quality of Experience (QoE) for online video services that are based on TCP-streaming. We present a dedicated QoE model for You Tube that takes into account the key influence factors (such as stalling events caused by network bottlenecks) that shape quality perception of this service. As second contribution, we propose a generic subjective QoE assessment methodology for multimedia applications (like online video) that is based on crowd sourcing - a highly cost-efficient, fast and flexible way of conducting user experiments. We demonstrate how our approach successfully leverages the inherent strengths of crowd sourcing while addressing critical aspects such as the reliability of the experimental data obtained. Our results suggest that, crowd sourcing is a highly effective QoE assessment method not only for online video, but also for a wide range of other current and future Internet applications.

The Weber-Fechner Law (WFL) is an important principle in psychophysics which describes the relationship be- tween the magnitude of a physical stimulus and its perceived intensity. With the sensory system of the human body, in many cases this dependency turns out to be of logarithmic nature. Re- cent quantitative QoE research shows that in several different scenarios a similar logarithmic relationship can be observed be- tween the size of a certain QoS parameter of the communication system and the resulting QoE on the user side as observed during appropriate user trials. In this paper, we discuss this surprising link in more detail. After a brief survey on the background of the WFL, we review its basic implications with respect to related work on QoE assessment for VoIP, most notably the recently published IQX hypothesis, before we present results of our own trials on QoE assessment for mobile broadband scenarios which confirm this dependency also for data services. Finally, we point out some conclusions and directions for further research.

This article gives a brief overview of our Open Source software framework for generating MPEG-2 Transport Streams carrying digital TV programs and related metadata. Due to the framework's modular architecture and its support for a large variety of broadcast standards, it enables the realization of a broad spectrum of prototype implementations and test setups in the context of Multimedia research and development. We describe three different example configurations for the generation of DVB-T and DVB-H streams that also support MHP applications for local interactivity.

There are new challenges to us, as researchers, on how to design and evaluate new mobile applications because they give users access to powerful computing devices through small interfaces, which typically have limited input facilities. One way of overcoming these shortcomings is to utilize the possibilities of multimodality. We report in this paper how we designed, developed, and evaluated a multimodal mobile application through a combination of laboratory and field studies. This is the first time, as far as we know, that a multimodal application has been developed in such a way. We did this so that we would understand more about where and when users envisioned using different modes of interaction and what problems they may encounter when using an application in context.

New sensor technologies, powerful mobile devices, and wireless communication standards strongly proliferate ubiquitous and pervasive computing. This is particularly true for healthcare applications. Modern non-invasive unobtrusive sensors enable sophisticated monitoring of the human body. As a result of this monitoring a growing amount of elderly people suffering from chronic diseases will benefit from an enhanced quality of life and improved treatment. Healthcare applications process a vast amount of continuously generated data, this has become a serious challenge. Data stream management addresses this problem in general. However, healthcare applications have additional requirements, e.g., flexibility and reliability. As a consequence, we propose the combination of a sophisticated information infrastructure, called hyperdatabase, and data stream management. Hyperdatabases already allow for reliable process management in a peer-to-peer fashion. In this paper, we elaborate the close relation between distributed process management and data stream management. Further, we show that data stream processing can significantly benefit from an infrastructure for reliable process management. We present the architecture of our prototype infrastructure that supports processing of continuous data streams with a high degree of flexibility and reliability.