International audience

Omnidirectional media formats, particularly 360° videos with spatial audio, provide new immersive experiences and introduce a novel dimension to content consumption.We explore the relationship between subjective data quality and metric performance evaluation in the context of Omnidirectional videos with spatial audio. While methodologies for 360° video quality assessment have been standardized and well-documented, previous efforts primarily focus on video with limited audio conditions, e.g., mono/stereo rendering. Moreover, the experimental test setup and subjective test methodologies impact data quality and the ability to use these data for objective quality metrics performance evaluation. Such a problem is key in the industry and the standardization activities, as codecs and quality models must be compared. Hence, the requirements on the ground truth data quality have to be clarified to allow proper conclusions.In this paper, we compare two setups and three test methodologies to study how experiment discriminability changes with conditions and participant number. Then, we show how discriminability impacts the resolving power of quality metrics. We show that higher-performing metrics require higher-quality data to reveal their full potential. In doing so, we put into relation the experimental cost, data quality, and resolving power.

User Generated Content (UGC) video streaming is a major application on the Internet. Even small bitrate savings can have large network impacts at this scale. In order to achieve improvements without sacrificing experience, the quality of UGC videos needs to be better understood. In recent years video quality evaluation models designed for the evaluation of UGC videos have received a lot of attention. However, considering that these models are learning-based models, they heavily depend on the training data that has been used. In this paper, a new dataset is introduced that allows studying the differences in characteristics between existing UGC video datasets. It reveals the range of quality that was covered by existing UGC video datasets, and the implication of these quality ranges on training and validation performance of UGC video quality prediction models. Furthermore, this work demonstrates that dataset alignment enables existing UGC models to achieve higher performance. This alignment dataset can be found openly available on Zenodo (https://zenodo.org/doi/10.5281/zenodo.12155934).

To improve users' experience and decrease their likelihood of quitting watching videos, this paper addresses the question of how to encode the videos used in adaptive bitrate (ABR) video streaming. When addressing ABR video streaming, a lot of effort has been put into developing ABR control schemes. However, ways to appropriately encode videos also need to be defined. Unlike previous approaches that focus on coding quality, this paper considers theuser quitting ratio. Theuser quitting ratiois the percentage of users still watching videos at a given time and enables us to address the consequences of quality and stimulus duration on the decision of a user to quit. Considering the value of theuser quitting ratio, this paper describes a method that uses content analysis, as well as a network's historical throughput data, to define how video should be encoded to decrease the likelihood of users quitting watching. Unlike previous approaches, the method is independent of the ABR control scheme used by the video player, and the selected ladders perform equivalently across different players with different behaviors. Results of experiments based on real-world network traces demonstrate the usefulness of the proposed method.

This paper addresses the improvement of the quality expe-rienced by users while using adaptive bitrate (ABR) video streaming by considering long-term aspects in the chunk se-lection process. Compared with traditional approaches that select chunks on the basis of predicted immediate throughput value, leveraging long-term throughput values can enable a long-term chunk selection strategy to be defined that enables perceived quality to be increased in challenging network con-ditions. In results for real-world throughput measurements, quality was improved by at least 0.5 units on a five-grade mean opinion score (MOS) in up to 60% of tested challenging network conditions thanks to the increase in the time frame in the chunk selection mechanism. Furthermore, using a long-term chunk selection mechanism was shown to enable higher quality to be maintained than short-term approaches when throughput prediction becomes noisy. The proposed approach was shown to benefit a quality-centric ABR control scheme but is also applicable to other ABR algorithms.

As adaptive bitrate streaming services are widely used, video streaming providers need to know what factors impact viewing abandonments. As quality and content-related factors are known to influence viewing abandonments, the reasons for viewing abandonments need to be analyzed while considering both factors. For this purpose, previous studies developed machine learning models, but most of them lack the interpretability of the relationships between explanatory variables and a target variable. In addition, causal relationships among explanatory variables need to be considered to disambiguate the effect of each variable. In this paper, we propose using Asymmetric Shapley value (ASV) to study the interpretability of a developed machine learning model and to take into account the effect of causalities among explanatory variables. We used a dataset collected in laboratory experiments on adaptive bitrate streaming and built binary classification models that classify viewing abandonment reasons into quality-induced or content-induced. We selected random forests as the best model and analyzed the relationships between explanatory variables (application quality, users' behavior, content attribute) and the target variable (viewing abandonment reason) with ASV. The results provided meaningful insights into the relationships and showed assuming causality is helpful to modify the estimation of the relationships

In this paper, a method is presented to estimate bitrate ladders on the basis of both content complexity and network traces. Unlike previous methods, the proposed algorithm is independent of the adaptive bitrate (ABR) control scheme used by the client. The proposed approach was evaluated in a simulation on the basis of real-life throughput measurements over a large period of time and across multiple ABR control schemes. Unlike in previous studies, perceptual quality metrics that account for the effect of both coding and stalling on video quality are used in the evaluations. Results show that the proposed method performs consistently over time and across ABR control schemes. Thus, this method will enable video streaming service providers to encode videos by using their knowledge of the network performance. This will ultimately enable network usage to be optimized and the quality of experience to be improved.

Video streaming is an important application on the Internet. To ensure user satisfaction, video streaming service providers monitor their services in terms of quality and engagement. This enables them to ensure high quality services and grow their platform and incomes. However, studying user engagement is difficult as the decision of a user to quit or continue watching videos is jointly affected by many factors such as interest towards content, time available, and service quality. Therefore, the reason for quitting is difficult to identify. To address this, this study is based on usage data of a real-world TV service called Puffer and aims to study the relationship between service quality and quitting actions. Data from December 2020 were collected and correspond to 230,880 distinct viewing sessions. On the basis of these data, performing analysis at different scales (hour, day, month) enables the identification of different reasons for users to quit videos. By using this analysis, quality-related quitting events are identified and put into relation with quality-related parameters as well as state-of-the-art video quality and user quitting prediction models. Results show that quitting prediction models can be used to identify such events. Finally, by the means of logistic regression, this work describes the first steps towards mapping quitting prediction on the basis of models trained using data from laboratory experiments to real-world scenarios and shows a classification accuracy of 75.9%.

To improve user engagement such as viewing time, this paper addresses the understanding and prediction of theuser quitting ratiofor users watching videos using adaptive bit rate video streaming. Theuser quitting ratiois defined as the percentage of users still watching videos at a given time. To perform this study, five subjective experiments involving up to 264 participants were conducted in a laboratory setting. Results indicated the effects of coding quality, initial buffering, and midway stalling onuser quitting ratio. Then, a framework was defined to predict theuser quitting ratioas a function of time. This framework achieves good prediction accuracy and can be used in multiple scenarios including when quality adaptation and stalling occur. Finally, it is suitable for monitoring applications where bitstream are encrypted and low processing cost is required.

In this paper, a model is investigated for optimizing the encoding of adaptive bitrate video streaming. To this end, the relationship between quality, content duration, and acceptability measured by using the completion ratio is studied. This work is based on intensive subjective testing performed in a laboratory environment and shows the importance of stimulus duration in acceptance studies. A model to predict the completion ratio of videos is provided and shows good accuracy. By using this model, quality requirements can be derived on the basis of the target abandonment rate and content duration. This work will help video streaming providers to define suitable coding conditions when preparing content to be broadcast on their platform that will maintain user engagement.

In this study, the effect of coding degradation and stalling events on the percentage of users who quit watching videos midway is studied (referred to as the user quitting rate). The results are based on three laboratory-based subjective experiments involving up to 104 participants. The results show that for the coding condition only, a Mean Opinion Score (MOS) lower than 3 on a 5-point ACR quality scale will result in users quitting the video. The quitting rate was found to increase further when the MOS falls below 2.5. When a stalling event occurs, the quitting rate was found to be dependent on the stalling position, the stalling duration, the percentage of user who already quit, and the MOS. Statistical analysis allowed the identification of interaction terms between the stalling position and percentage of users who had already quit. Finally, the results were used to establish a model of the user quitting rate due to stalling.

Adaptive bitrate streaming services for mobile terminals have drastically spread in recent years, and it is becoming more important for service providers to increase users' satisfaction by understanding users' viewing behaviors (e.g., how long users watch videos, and why users quit viewing videos) and taking measures such as appropriately designing the quality levels of the videos to be placed on their distribution servers. To investigate the impacts of quality factors on users' viewing behaviors in adaptive bitrate streaming, we conducted an experiment in which participants could freely search and watch videos on smartphones under various network conditions. Through 800 10-minute tests, 1,449 valid views were collected, and the collected dataset was analyzed to characterize the impacts of the initial loading delay, average bitrate, and stalling events on the cumulative quit rate (CQR) of users. Furthermore, the impacts of the average bitrate and stalling events were evaluated quantitatively, using the 2-sample Anderson-Darling test, as well as the combined impact of these two quality factors. The characteristics of the impacts of the above quality factors indicate the possibility of applying survival analysis for our dataset, and suggest that the average bitrate, the number of stalling events, and the average stalling duration should be considered as the external covariates when building a model to estimate the users' viewing time.

Advanced traveler information systems (ATISs) provide travelers with public transportation information to improve the quality of individual life and alleviate congestion as well as air pollution. However, existing works have not fully incorporated bike sharing systems within ATIS, providing no interaction with other modalities nor taking bike stocks into account. In addition, the uncertainty of traffic conditions and multimodal routing makes it challenging to accurately estimate the travel time. In this paper, we leverage large-scale historical data collected in London and construct a multimodal transportation network, including bus, tube, public bikes, and walking. We solve the modalities aggregation problem by practically modeling the travel time, arrival time, bike stock, and transfer time between different transport modalities. Furthermore, we propose TERP, a time-event-dependent route planner that optimizes both trip duration and reliability. We conduct experiments on extensive real-world data with over 23 million arrival records and 15 million stock records on more than 10 000 stations from transport for London platform (TfL). The results validates 14.91% reduction of actual total trip duration and 56.28% improvement in terms of route reliability in rush hours comparing with TfL.

In this work, the evaluation of user engagement's characteristics in adaptive bitrate video streaming is addressed. To this aim, the viewing time and its relation with video quality is studied in two carefully designed subjective tests. Video quality and viewing time were addressed in distinct experiments. In case of viewing time, users were allowed to stop watching the videos when they desired. It was found that for low-quality videos, the number of users dropping the video increases logarithmically as a function of time. In addition, when a stalling event occurs, users start dropping video playback after a waiting period of 5 seconds. Then, when the stalling ends, the dropping rate returns to its baseline rate (which depends on video quality). The number of users stopping watching video after a stalling event was found to be a function of stalling position, stalling duration, and the quality affected by coding. A baseline model considering only stalling features is defined. Finally, a model for predicting the video completion rate is proposed that achieves a Pearson correlation of 0.96 and a root-mean-square error (RMSE) of 0.064.

In this paper, a crowdsourcing experiment was conducted involving different panels of participants. The aim of this study is to evaluate how the preference of one image over another one is related with the knowledge of the participant in photography. In previous work the two discriminant evaluation concepts “presence of a main subject” and “exposure” were found to distinguish group participants with different degrees of knowledge in photography. Each of these groups provided different means of aesthetic appeal ratings when asked to rate on an absolute category scale. The present paper extends previous work by studying preference ratings on a set of image pairs as a function of technical knowledge and more specifically adding a focus on the variance of rating and agreement between participants. The conducted study was composed of two different steps where the participants had to first report their preference of one image over another (paired comparison), and an evaluation of the technical background of the participant using a specific set of images. Based on preference-rating patterns groups of participants were identified. These groups were formed by clustering the participants who saw and shared the same preference rating on images in one group, and the participants with low agreement with other participants in another group. A per-group analysis showed that a high agreement between participants could be observed when participants have technical knowledge. This indicates that higher consistency between participants can be reached when expert users are being recruited, and therefore participants should be carefully selected in image aesthetic appeal evaluation to ensure stable results.

This paper describes the results of a subjective test to assess current technology used for 3DTV broadcasting. As a first aspect, the performance of the currently deployed coding schemes was compared to state of the art algorithms. Our results show that down sampling and packing 3D stereoscopic videos according to the so called Side-By-Side format gives the highest perceived quality for a given bit rate. The second aspect of the study was to investigate how common 2D error concealment algorithms perform in case of 3D, and how their 3D-related performance compares with the 2D case. The results provide information on whether binocular suppression or binocular rivalries play the most important role for 3D video quality under transmission error. The results indicate that binocular rivalries and related visual discomfort are the dominant factors. Another aspect of the paper is a comparison of the test results with results from different labs to evaluate the repeatability of a subjective experiment in the 3D case, and to compare the employed test methodologies. Here, the study shows the variation between observers when they are rating visual discomfort and illustrates the difficulty to evaluate this new dimension.