Video streaming applications, both on-demand and real-time, have seen tremendous growth and acceptance in the past two decades. To meet the increasing demand and user expectations of such any time, any device, any place availability of such services, there is a need for higher compression efficiency codecs and intelligent encoding strategies.This paper presents a comparative codec compression efficiency evaluation of the four most popular and widely used codec compression standards (H.264, HEVC, VP9 and AV1) for cloud gaming applications considering ultra-low latency encoding settings on a 4K gaming video dataset. Our results considering multiple resolution-bitrate pair encoding show that under strict ultra-low latency encoding settings, software codec implementations of VP9 and AV1 codecs perform better than H.264 and HEVC in terms of quality and rate savings albeit at an increased encoding time compared to H.264, especially at higher resolutions.

With the rapid increase of video streaming content, high-quality video quality metrics, mainly signal-based video quality metrics, are emerging, notably VMAF, SSIMPLUS, and AVQM. Besides signal-based video quality metrics, within the standardization body, ITU-T Study Group 12, two well-known bitstream-based video quality metrics are developed named P.1203 and P.1204.3. Due to the low complexity and low level of access to the bitstream data, these models gained attention from network providers and service providers. In this paper, we proposed a new bitstream-based model named Deep-BVQM, which outperforms the standard models on the tested datasets. While the model comes with slightly higher computational complexity, it offers a frame-level quality prediction which is essential diagnostic information for some video streaming services such as cloud gaming. Deep-BVQM is developed in two layers; first, the frame quality was predicted using a lightweight CNN model. Next, the latent features of the CNN were used to train an LSTM network to predict the video quality in a short-term duration.

Abstract Gaming video streaming services are growing rapidly due to new services such as passive video streaming of gaming content, e.g. Twitch.tv, as well as cloud gaming, e.g. Nvidia GeForce NOW and Google Stadia. In contrast to traditional video content, gaming content has special characteristics such as extremely high and special motion patterns, synthetic content and repetitive content, which poses new opportunities for the design of machine learning-based models to outperform the state-of-the-art video and image quality approaches for this special computer generated content. In this paper, we train a Convolutional Neural Network (CNN) based on an objective quality model, VMAF, as ground truth and fine-tuned it based on subjective image quality ratings. In addition, we propose a new temporal pooling method to predict gaming video quality based on frame-level predictions. Finally, the paper also describes how an appropriate CNN architecture can be chosen and how well the model performs on different contents. Our result shows that among four popular network architectures that we investigated, DenseNet performs best for image quality assessment based on the training dataset. By training the last 57 convolutional layers of DenseNet based on VMAF values, we obtained a high performance model to predict VMAF of distorted frames of video games with a Spearman’s Rank correlation (SRCC) of 0.945 and Root Mean Score Error (RMSE) of 7.07 on the image level, while achieving a higher performance on the video level leading to a SRCC of 0.967 and RMSE of 5.47 for the KUGVD dataset. Furthermore, we fine-tuned the model based on subjective quality ratings of images from gaming content which resulted in a SRCC of 0.93 and RMSE of 0.46 using one-hold-out cross validation. Finally, on the video level, using the proposed pooling method, the model achieves a very good performance indicated by a SRCC of 0.968 and RMSE of 0.30 for the used gaming video dataset.

Mobile gaming has the largest market shares of all gaming domains, accounting for an estimated $ 77.2 billion in 2020. In recent times, one can witness an increase in highly interactive mobile online games. However, the gaming Quality of Experience (QoE) can be strongly influenced by network degradations, concretely by delay and packet loss. Thus, network providers need to ensure fast and reliable connections between the gaming servers and the users' clients. To maintain a satisfying user experience, QoE prediction models are fundamental. Aiming at the development of such a model, a detailed parameter space consisting of various delay and packet loss conditions will be investigated in this paper. Here, especially the importance of jitter is of interest. Next, it will be examined whether a recently published opinion model for cloud gaming, the ITU-T Rec. G.1072, can also be used for online mobile gaming. Finally, a new proposal for a model targeting online mobile gaming services will be presented and evaluated concerning its performance.

Humans are fascinated by video games for many years, which intrinsically immerse players in their virtual environments. Apart from the challenges offered by steadily new game concepts, aesthetically pleasing environments and characters, stories, and sound effects are highly important for the player experience. Recently, the new concept of cloud gaming, which offers users to play games executed on a cloud server remotely, is becoming increasingly popular. While cloud gaming offers many advantages, additional audio and video streaming poses many technical challenges. To ensure a satisfying Quality of Experience (QoE) of their customers, all stakeholders are interested in finding out which aspects of a gaming experience are of high importance and how resources can be optimally allocated. However, gaming QoE is a multidimensional construct including hedonic and pragmatic aspects and could be strongly influenced by interaction quality, video quality, and audio quality. While the impact of network and video encoding parameters of cloud gaming services was investigated in much detail in recent years, not many studies about the effect of audio quality on gaming experiences are available. Thus, in this paper, the impact of audio quality on gaming experience under different bitrate and packet loss conditions using two popular games is investigated. Therefore, a subjective experiment adhering to the ITU-T Rec. P.809 was conducted. The results show a significant impact of packet loss on audio quality and the overall gaming QoE. However, no significant effect of the bitrate, which was reduced to a minimum of 32 kbps, was revealed. Additionally, the influence of audio quality on gaming QoE was stronger for a game, which contained mainly diegetic effect sounds compared to a game containing few diegetic sounds and affect sound as background music.

Cloud Gaming (CG) is an immersive multimedia service that promises many benefits. In CG, the games are rendered in a cloud server, and the resulted scenes are streamed as a video sequence to the client. Using CG users are not forced to update their gaming hardware frequently, and available games can be played on any operating system or suitable device. However, cloud gaming requires a reliable and low-latency network, which makes it a very challenging service. Transmission latency strongly affects the playability of a cloud game and consequently reduces the users' Quality of Experience (QoE). In this paper, we propose a latency compensation technique using game adaptation that mitigates the influence of delay on QoE. This technique uses five game characteristics for the adaptation. These characteristics, in addition to an Aim-assistance technique, were implemented in four games for evaluation. A subjective study using 194 participants was conducted using a crowdsourcing approach. The results showed that the majority of the proposed adaptation techniques lead to significant improvements in the cloud gaming QoE.

The gaming industry is one of the largest digital markets for decades and is steady developing as evident by new emerging gaming services such as gaming video streaming, online gaming, and cloud gaming. While the market is rapidly growing, the quality of these services depends strongly on network characteristics as well as resource management. With the advancement of encoding technologies such as hardware accelerated engines, fast encoding is possible for delay sensitive applications such as cloud gaming. Therefore, already existing video quality models do not offer a good performance for cloud gaming applications. Thus, in this paper, we provide a gaming video quality dataset that considers hardware accelerated engines for video compression using the H.264 standard. In addition, we investigate the performance of signal-based and parametric video quality models on the new gaming video dataset. Finally, we build two novel parametric-based models, a planning and a monitoring model, for gaming quality estimation. Both models are based on perceptual video quality dimensions and can be used to optimize the resource allocation of gaming video streaming services.

Traditionally, the Quality of Experience (QoE) is assessed in a controlled laboratory environment where participants give their opinion about the perceived quality of a stimulus on a standardized rating scale. Recently, the usage of crowdsourcing micro-task platforms for assessing the media quality is increasing. The crowdsourcing platforms provide access to a pool of geographically distributed, and demographically diverse group of workers who participate in the experiment in their own working environment and using their own hardware. The main challenge in crowdsourcing QoE tests is to control the effect of interfering influencing factors such as a user's environment and device on the subjective ratings. While in the past, the crowdsourcing approach was frequently used for speech and video quality assessment, research on a quality assessment for gaming services is rare. In this paper, we present a method to measure gaming QoE under typically considered system influence factors including delay, packet loss, and framerates as well as different game designs. The factors are artificially manipulated due to controlled changes in the implementation of games. The results of a total of five studies using a developed evaluation method based on a combination of the ITU-T Rec. P.809 on subjective evaluation methods for gaming quality and the ITU-T Rec. P.808 on subjective evaluation of speech quality with a crowdsourcing approach will be discussed. To evaluate the reliability and validity of results collected using this method, we finally compare subjective ratings regarding the effect of network delay on gaming QoE gathered from interactive crowdsourcing tests with those from equivalent laboratory experiments.

The ultimate goal of designing game applications is to evoke a state of mental immersion in human users. Recently, advancing and very promising cloud gaming services catch high interest of the research community and industry. Cloud gaming services reduce computational costs of a client by outsourcing the game logic and rendering to a remote server. Consequently, the degree to which a game runs smoothly and enables uninterrupted interaction depends on impairments of the network connection between client and server. Furthermore, the visibility of such impairments may be constrained by properties of the visual display at the client-side. The present paper investigates the impact of common network impairments (bit rate, delay, packet loss) and screen size (small, medium, large) on mental immersion. In addition to traditional subjective assessment using the Immersive Experience Questionnaire (IEQ), also less intrusive, continuous physiological methods are employed (electrocardiography, ECG; electro-dermal activity, EDA). Participants engaged in playing an action platform computer game under different combinations of network impairments and screen sizes. Results revealed a small main effect of screen size on gaming Quality of Experience and real-world dissociation ratings between the small and medium screen size. Effects of network impairments on all IEQ scales were significant and also manifested as increased heart rate variability for packet loss. Besides, positive correlations were found between IEQ scales and heart rate variability. These findings suggest that network impairments influencing gameplay interaction might be of higher importance for immersive experience than varying screen size. Heart rate variability shows promise as a useful ECG measure in future studies on gaming immersion.

Recently, streaming of gameplay scenes has gained much attention, as evident with the rise of platforms such as Twitch.tv and Facebook Gaming. These streaming services have to deal with many challenges due to the low quality of source materials caused by client devices, network limitations such as bandwidth and packet loss, as well as low delay requirements. Spatial video artifact such as blockiness and blurriness as a result of as video compression or up-scaling algorithms can significantly impact the Quality of Experience of end-users of passive gaming video streaming applications. In this paper, we investigate solutions to enhance the video quality of compressed gaming content. Recently, several super-resolution enhancement techniques using Generative Adversarial Network (e.g., SRGAN) have been proposed, which are shown to work with high accuracy on non-gaming content. Towards this end, we improved the SRGAN by adding a modified loss function as well as changing the generator network such as layer levels and skip connections to improve the flow of information in the network, which is shown to improve the perceived quality significantly. In addition, we present a performance evaluation of improved SRGAN for the enhancement of frame quality caused by compression and rescaling artifacts for gaming content encoded in multiple resolution-bitrate pairs.

Cloud Gaming is an emerging service that is considered by many as the future of the gaming industry. This service requires a highly reliable network with low latency and high bandwidth. If these requirements are not satisfied, cloud gaming services cannot create a good Quality of Experience (QoE) for its users. However, gaming QoE can vary significantly among different game scenarios and users. For an optimal resource allocation and quality estimation, it is highly important for cloud providers, game developers, and network planners to consider the influence of the game content and gamers. This paper presents the result of a subjective study that investigated the impact of different player strategies and user inputs on their perceived delay. The results indicated that the user input characteristics vary among the games but stays the same between different users and different strategies. In addition to the users' inputs, the input quality and the overall gaming experience of the users were also investigated, and results did not show any main effect of user strategy on the delay sensitivity of the games.

One of the fields where Virtual Reality (VR) is finding a potentially growing market is in the combination of exercising and gaming - also called exergaming. When it comes to competition in gaming, is important to investigate how different levels of delay influence overall quality of experience (QoE) in VR multiplayer exergames. The experimental setup consisted of a VR application coupled with a rowing ergometer, allowing races between the user and an artificially created opponent that is following the player with a similar speed and keeping the race tight. To investigate the influence of the delay, on both user's and opponent's side three levels of network delay were introduced (30ms, 100ms, and 500ms) and mixed throughout different conditions. After each session, participants rated perceived flow, sense of presence, and the degree to which they have noticed the delay in their or the opponent's system. Interestingly, results show different perception of delay and QoE depending on user's own delay. Participants perceived the opponent's player as being delayed even if only the player itself had network delay along with significantly lower rating of QoE only when their delay was high.

Both online and cloud gaming services require a very low network delay to create a good Quality of Experience (QoE) for their users. The required network latency cannot be guaranteed due to the current best effort-nature of the network, and as a result, network latency often degrades the gamers performance and QoE. In this paper, the adaptability of gamers to different variations on delay is investigated both subjectively and objectively using three self-developed games. The results show that gamers can adapt to constant delay while they are playing and change their behavior if the actions in a game are predictable. Such adaptation leads to a significant increase in gamers performance and QoE. The paper also provides evidence that regardless of performance frequent delay switching annoys gamers. The result of this study can be used to create a network resource allocation technique which controls a congested network by giving more priority and resource to the unadaptable games than the adaptable games.

With emerging delay sensitive gaming services such as cloud gaming and online gaming, the importance of understanding and reducing the effect of delay on the gamer's Quality of Experience (QoE) becomes highly important for the success of these services. In this paper, the findings of two subjective experiments investigating the relationship between delay and QoE are reported. In the first study, it was shown that in addition to the direct effect of the delay on QoE, there is a significant indirect effect between delay and QoE through the relationship with performance. In the second part of the paper, we illustrate that adapting characteristics of a game can strongly mitigate the negative effect of delay on gaming QoE due to increased player performance. This adaptation in addition to compensation the effect of the delay, in contrast to the other difficulty adjustment systems, does not require to track the gamer's interaction, behaviors, and profile.

Gaming as a popular system has recently expanded the associated services, by stepping into live streaming services. Live gaming video streaming is not only limited to cloud gaming services, such as Geforce Now, but also include passive streaming, where the players' gameplay is streamed both live and ondemand over services such as Twitch.tv and YouTubeGaming. So far, in terms of gaming video quality assessment, typical video quality assessment methods have been used. However, their performance remains quite unsatisfactory. In this paper, we present a new No Reference (NR) gaming video quality metric called NR-GVQM with performance comparable to state-of-the-art Full Reference (FR) metrics. NR-GVQM is designed by training a Support Vector Regression (SVR) with the Gaussian kernel using nine frame-level indexes such as naturalness and blockiness as input features and Video Multimethod Assessment Fusion (VMAF) scores as the ground truth. Our results based on a publicly available dataset of gaming videos are shown to have a correlation score of 0.98 with VMAF and 0.89 with MOS scores. We further present two approaches to reduce computational complexity.

Video games and sport are an essential part in the life of millions of people. With recent advances of immersive virtual reality devices such as the HTC Vive, Oculus Rift, or PlayStation VR, the use of virtual environments (VE) for exergames is becoming more and more popular. An exergame combines a physical activity with video game elements by tracking body movements or reactions of user, attempting to engage users in a more enjoyable system. In this paper, we present the results of a subjective experiment carried out with the aim to compare different kinds of virtual environments with each other. A rowing ergometer, connected either to a virtual reality system using a head-mounted display (HMD) or to a CAVE environment, was used as an exergame device. While for rowing experts, fitness and performance improvements are of major interest, we wanted to focus on the motivation and engagement of non-professionals. By means of a series of questionnaires and a follow-up interview, the Quality of Experience of participants using the system was assessed. Measurements include concepts such as flow, presence, video quality and well-being. Results show significant advantages of the HMD as well as of the CAVE compared to a system without a VE for the overall quality, system feedback, and flow. While the CAVE and HMD system mainly differed in their autotelic experience, the HMD was favored by the majority of participants due to a superior feeling of presence.

Subjective tests to assess the Quality of Experience (QoE) of gaming services are necessary to enable service providers to ensure the satisfaction of their customers. Since gaming is an interactive activity, interactive tests are typically conducted to measure the full spectrum of the player experience. However, carrying out such tests is expensive and time-consuming. Furthermore, the results can be influenced by the behavior and abilities of participants. For this reason, it is of high interest whether such interactive tests can be partially replaced with passive viewing-and-listening tests. In this paper, we present a comparison of an interactive gaming test with passive tests using two different durations. To investigate the differences between the test paradigms, we assessed the overall quality, the video quality and the reactiveness of the game as well as other player experience aspect for different frame rates and bit rates. Results show that once certain requirements are fulfilled, passive tests offer indeed a valuable quality assessment method. However, if the duration of the presented video material is too short, the passive test overestimated the gaming and video quality. Furthermore, we show that the player performance has no impact on the video quality ratings.

Virtual Reality (VR) multiplayer games are currently developed more frequently, while in parallel VR is becoming more accessible to the broader public. Also exergames, which refer to computer games combined with a physical activity, are getting widely available, driven by developments in sensor technologies. In this paper, we present results of a multiplayer exergame experiment where two players race against each other four times on two rowing ergometers. Experimental conditions were type of environment (with VR vs. without VR) and the possibility of conversation with the opponent (conversation enabled vs. conversation disabled). To measure subjective experience, participants rated perceived flow, sense of presence and social presence on standardized questionnaires. Results show that the type of environment has a significant influence on the perceived presence, as participants rated the presence higher in VR. The possibility of conversation resulted in a perception of a significantly higher social presence. This illustrates that the usage of virtual environments as well as the possibility of conversations between users are highly valuable to increase the Quality of Experience during exergames. In our future research, we plan to further investigate means to improve the user experience by applying gamification, different representations of opponents, and to assess the impact of network impairments during multiplayer scenarios.

Measuring the quality experienced during online gaming activities is an important prerequisite for managing gaming services, but up to now no standardized methods were available for this purpose. This paper presents the results from two study items promoted by Study Group 12 of the International Telecommunication Union. Work on these items resulted in one draft Recommendation on subjective methods for evaluating gaming Quality of Experience (QoE), as well as one approved Recommendation on influence factors on gaming QoE which might be considered in service planning and management. Future work which is necessary to predict gaming QoE is addressed in the end of the paper.

Recent advancements of network architecture such as 5G networks, promise cloud services with strict network constrains a bright future. Cloud gaming as an interactive service has strict end-to-end delay constraints. Therefore, many studies investigated the impact of network parameters such as delay or packet loss on gaming QoE. However, they mostly compared games or genres with each other and neglected the fact even two levels of the same game may have different sensitivity toward delay. In order to understand the game characteristics that cause this difference in delay sensitivity, a bottom-up approach by means of modifiable open source games can be of high value. In this paper we present a game designed to tackle this issue. The game allows to artificially change characteristics of the game, such as the pace and size of objects, and also simulate influences like delay, packet loss or a reduced frame rate. This allows the usage of the game also for crowdsourcing studies, where it is not possible to control the different network conditions of the participants, and to investigate the impact of spatial and temporal accuracy in respect to the sensitivity towards impairments.

Recent advances of streaming services and the upcoming new generation of mobile networks, 5G, offering low end-to-end delay as well as high bandwidths, promise a bright future for cloud gaming applications. Cloud gaming, in the contrary to traditional gaming services, suffers not only from system factors on the client, but is also affected significantly by the network, server specification and encoding parameters. In this paper, we present the results of a subjective experiment aiming to investigate the impact of two influencing factors, frame rate and bit rate, on the gaming Quality of Experience. The results reveal that a trade-off between an acceptable video quality and interaction quality exists. In case of very low bit rates, lowering the frame rate can improve the video quality while at some point, jerkiness becomes visible which affects the video quality negatively and the control over the game will be strongly reduced. Furthermore, even though in the gaming community a frame rate of 60 fps is desired, no significant difference for quality ratings, as well as performance ratings, was found between 60 fps and 25 fps. Therefore, it would be highly valuable for service providers to find an ideal strategy for this issue. In addition, we investigate the impact of video encoding on gaming experience dimensions. Finally, a first attempt to model the impact of two influencing factors on overall quality will be presented.

Recent advances in Virtual Reality (VR) technologies have resulted in a wider availability of Head Mounted Displays (HMDs). However, it is still unclear if VR gaming offers a substantial added value to players. For this reason a comparison of gaming experiences on VR HMD to those on mobile and PC, two other popular gaming platforms, is performed by conducting a user study via two games available on all three platforms. We explore the QoE of gaming by investigating momentous dimensions using the Player Experience of Need Satisfaction (PENS) questionnaire. The results show higher Presence and Autonomy obtained by using HMD when compared to the two other platforms. However, these factors alone did not improve the Overall Quality. To take advantage of the new technology, satisfaction of all psychological needs, especially Competency, must be assured.

Cloud gaming promises many advantages from the user and game developer perspective. But besides its benefits, cloud gaming suffers from two bottlenecks: bandwidth and latency. Although, many researchers have identified delay as an important factor on the QoE in gaming, the cause of varying tolerances towards delay in different games is not yet well understood. In this paper, we want to show that delay sensitivity should not be generalized for an entire game or genre. Instead, a specific scenario with its underlying characteristics should be considered, since differences within the same game can be higher than differences to another game. To investigate the delay sensitivity of games, we assessed quality features such as control, difficulty, delay perception, annoyance, and fairness for highly similar scenarios only differing in their pace or perspective. Results show that changing the pace within the same game can lead to stronger differences in respect to the delay sensitivity than using another game type. Furthermore, we explain how quality features might be used in the judgement process of a player and define a set of input action metrics for a future delay sensitivity based classification of games.