The recent improvements of artificial intelligence (AI) technologies present a revolutionary opportunity for a wide variety of services. One of the most impactful AI technologies, gaining hundreds of millions of users in mere months, are conversational agents based on Large Language Models (LLMs). An obvious application example of conversational agents is found in gaming, where players often engage in conversation with non-player-characters (NPCs). The use of contemporary AI in this context is expected to bring unprecedented levels of adaptability and freedom of expression to in-game conversations, but may come with its own set of drawbacks, such as increased response latency, which may impair user experience during such interactions. In this paper we present the results of a user study investigating the impact of two different types of latency — time to first token and time per output token — on the overall Quality of Experience and player experience in a conversational task with an LLM-powered NPC in a text-based fantasy role-playing game. Conducted with a sample of fairly regular AI users (N=20), our study provides valuable insights regarding the possible threshold for noticeable latency for this type of service.

While the topic of investigating user experience with immersive services, such as Social Virtual Reality (VR), is starting to gain traction in the research community, the unique case of multiplayer VR games requires a more specific approach. Attempts to investigate user experiences with this complex, multidimensional service are hindered by the absence of specific standards and guidelines going beyond what we know about non-immersive gaming. In this article, we present the results of a user study (N = 32) exploring participants’ experience of playing two competitive VR games of different genres (shooter, sports game), as we focus on three distinct facets of multiplayer VR gaming—network, interpersonal competitiveness, and social interaction. Furthermore, approaching the issue from the perspective of Quality of Experience researchers looking to conduct future user studies on the topic, we also present our findings as a way to shed light on factors that need further consideration, especially in the context of participant recruitment and methodology design, such as the choice of access network to be used for testing, and the issue of matchmaking study participants based on skill and prior relationship.

With mobile gaming becoming ever more popular, it is necessary to understand various aspects of player behavior in those games. This is especially true for free-to-play games as revenue comes from a small subset of the playerbase. This article presents a large-scale analysis of data collected over 9 months, comprising over 10 million players of a case study free-to-play mobile interactive story telling video game. The data are analyzed from a temporal standpoint, followed by presentation of findings in three important player engagement aspects: 1) retention; 2) conversion (i.e., players becoming payers); 3) lifetime revenue collected. These findings confirm the importance of early conversion in casual mobile games as the retention is low due to wide target audience. Additionally, players who convert very early also tend to spend more in the game in general. Most of the monetization depends on a small fraction of large spenders as one fifth of the entire revenue comes from only 2% of spenders. The article stresses out the key checkpoints in casual gameplay that are potentially most profitable topics for further research and player targeted actions. The insights of long-term analysis are relevant to both academia and industry in the efforts to optimize service quality, players' satisfaction, and profitability.

With advances in network capabilities, the gaming industry is increasingly turning towards offering "gaming on demand" solutions, with cloud gaming services such as Sony PlayStation Now, Google Stadia, and NVIDIA GeForce NOW expanding their market offerings. Similar to adaptive video streaming services, cloud gaming services typically adapt the quality of game streams (e.g., bitrate, resolution, frame rate) in accordance with current network conditions. To select the most appropriate video encoding parameters given certain conditions, it is important to understand their impact on Quality of Experience (QoE). On the other hand, network operators are interested in understanding the relationships between parameters measurable in the network and cloud gaming QoE, to be able to invoke QoE-aware network management mechanisms. To encourage developments in these areas, comprehensive datasets are crucial, including both network and application layer data. This paper presents CGD, a dataset consisting of 600 game streaming sessions corresponding to 10 games of different genres being played and streamed using the following encoding parameters: bitrate (5, 10, 20 Mbps), resolution (720p, 1080p), and frame rate (30, 60 fps). For every combination repeated five times for each game, the dataset includes: 1) gameplay video recordings, 2) network traffic traces, 3) user input logs (mouse and keyboard), and 4) streaming performance logs.

The technology of Unmanned Aerial Vehicles (UAVs), or simply drones, has gained much attention over the past two decades, with a rapid decrease in cost and increase in potential applications such as surveillance, journalism, agriculture, film-making, entertainment, etc. With the growth in the popularity of First-person view (FPV) drone piloting comes the challenge of transmitting control information and streaming video at very low latencies to enable proper real-time control. Due to strict latency requirements, FPV technology stagnated, with analog video transmission currently representing the most common implementation. In this paper, we present our preliminary research directed towards developing a digital alternative to this technology. We report on a user study designed to assess the Quality of Experience (QoE) of an FPV system coupled with a cloud-based drone flight simulator, with the focus being on video quality in ideal network conditions. Obtained results show that video encoding parameters have a significant impact on user perceived QoE. Furthermore, our findings highlight the relevance of simulator sickness in this kind of system.

The capabilities of commercial virtual reality (VR) systems can be utilized to implement a wide spectrum of interaction and locomotion methods. However, immersion in VR is often accompanied by undesirable physical sensations that undermine the VR gaming experience. Presenting the results of a user study (N=20) comparing the physiological and cognitive aftereffects of three different VR games with game mechanics centered around natural interaction, this paper attempts to shed light on the impact of different content factors (e.g., game genre, visual environment, interaction method) on user comfort. While moving away from cybersickness as the primary measure of VR-induced discomfort through investigating a broader set of physical symptoms, such as device-related discomfort, muscle fatigue, and pain, we also include a discussion on the complexities of measuring cybersickness for physically demanding games. Further focusing on cognitive performance, we discuss preliminary findings regarding different effects of common VR game interactions (pick and place, slicing, and shooting) on reaction time.

In this paper we present the results of a user study focused on evaluating the overall usability of a wearable video-communication system aimed at facilitating hands-free communication between a field technician and a remote expert. An asymmetrical cooperative electronics-based bomb defusal game was used to examine the subject's ability to perform a manual technical task with the aid of a head-mounted-display (compared to only verbal communication) in terms of both subjective (perceived usefulness for the given task) and objective (time to complete task, number of errors) metrics. We analyze different dimensions of usability (overall system usefulness, information quality, interface quality, and device comfort) as we discuss the impact of human factors (i.e., gender, wearing Rx eye-glasses) on user experience. While system use has been shown to improve both subjective and objective metrics in some cases, results appear to be highly dependant on the task itself. Furthermore, our findings highlight the importance of exploring different ways in which ergonomic issues present in commercial smart glasses may decrease perceived usability in certain demographics.

In this paper we present the results of two user studies aimed at exploring the effects of network latency on user experience in a First Person Shooter Virtual Reality multiplayer game. In both studies we observed the following metrics: Quality of Experience (QoE), willingness to continue playing in given network conditions, and the final outcome of the game (survival/death). After conducting the first study, we concluded that test methodology and contextual factors (e.g., social context, level of difficulty, weapon choice) may have had a strong influence on perceived QoE, subsequently causing inconclusive results. Therefore, we aimed to mitigate those factors through different methodology choices in the second study, which led to more conclusive, statistically significant results, indicating that user experience for the chosen game begins to suffer in cases of latency greater than approx. 100 ms (round trip time between client and server). In addition to the analysis of acquired results, we discuss the significance of methodology and context when conducting studies exploring gaming QoE in multiplayer environments.

Over the last few years, different client-side QoE monitoring apps have been developed that benchmark the performance of popular video streaming services. Such tools also provide the means for collecting ground truth data when developing models to estimate or classify QoE and various KPls from encrypted network traffic. We present a client-side YouTube QoE monitoring tool named ViQMon, which extracts YouTube performance data from the official app's Stats for Nerds window, and is applicable on various devices and platforms (Android, iOS). We compare ViQMon to approaches relying on YouTube's APls, and show relevant differences in buffering and application behavior in cases when videos are embedded and when videos are played in the official YouTube app. We further use ViQMon together with the collection of network measurements in both a laboratory and commercial mobile network to collect a large dataset of almost 500 YouTube videos streamed under different network conditions. The dataset is used to build machine learning based models for estimating QoE and various application-layer KPls solely from IP-level network traffic features. As such, the approach is applicable in the context of both TLS and QUIC traffic. The paper further compares and analyses the performance of the built models.

This paper presents the results of a subjective study aimed at assessing different locomotion (may also be referred to as navigation) techniques in virtual reality environments. We implemented a simple virtual environment in which participants were asked to assess the following four techniques using the HTC Vive platform: teleportation, controller movement, controller movement with tunneling vision, and human joystick. A total of 29 participants took part in the study and rated perceived level of immersion, physical discomfort, and overall Quality of Experience for all tested locomotion techniques. Moreover, participants indicated whether they needed a break after a specific test due to physical discomfort. The results indicate that teleportation yields overall highest QoE and lowest discomfort.

Cloud gaming has been recognized as a promising shift in the online game industry, with the aim of implementing the “on demand” service concept that has achieved market success in other areas of digital entertainment such as movies and TV shows. The concepts of cloud computing are leveraged to render the game scene as a video stream that is then delivered to players in real-time. The main advantage of this approach is the capability of delivering high-quality graphics games to any type of end user device; however, at the cost of high bandwidth consumption and strict latency requirements. A key challenge faced by cloud game providers lies in configuring the video encoding parameters so as to maximize player Quality of Experience (QoE) while meeting bandwidth availability constraints. In this article, we tackle one aspect of this problem by addressing the following research question: Is it possible to improve service adaptation based on information about the characteristics of the game being streamed? To answer this question, two main challenges need to be addressed: the need for different QoE-driven video encoding (re-)configuration strategies for different categories of games, and how to determine a relevant game categorization to be used for assigning appropriate configuration strategies. We investigate these problems by conducting two subjective laboratory studies with a total of 80 players and three different games. Results indicate that different strategies should likely be applied for different types of games, and show that existing game classifications are not necessarily suitable for differentiating game types in this context. We thus further analyze objective video metrics of collected game play video traces as well as player actions per minute and use this as input data for clustering of games into two clusters. Subjective results verify that different video encoding configuration strategies may be applied to games belonging to different clusters.

In this short paper we propose several objective and subjective metrics and present a comparison between two “commodity” VR systems: HTC Vive and Oculus Rift. Objective assessment focuses on frame rate, impact of ambiance light, and impact of sensors' line of sight obstruction. Subjective study aims at evaluating and comparing the pick-and-place task performance in a virtual world. We collected user ratings of overall quality, perceived ease of use, and perceived intuitiveness, with results indicating that HTC Vive slightly outperforms the Oculus Rift for the pick-and-place task under test.

Cloud gaming has been recognized as a promising shift in the online game industry, with the aim being to deliver high-quality graphics games to any type of end user device. The concepts of cloud computing are leveraged to render the game scene as a video stream which is then delivered to players in real-time. Given high bandwidth and strict latency requirements, a key challenge faced by cloud game providers lies in configuring the video encoding parameters so as to maximize player Quality of Experience (QoE) while meeting bandwidth availability constraints. In this paper we address this challenge by conducting a subjective laboratory study involving 52 players and two different games aimed at identifying QoE-driven video encoding adaptation strategies. Empirical results are used to derive analytical QoE estimation models as functions of bitrate and framerate, while also taking into account game type and player skill. Results have shown that under certain identified bandwidth conditions, reductions of framerate lead to QoE improvements due to improved graphics quality. Given that results indicate that different QoE-driven video adaptation policies should likely be applied for different types of games, we further report on objective video metrics that may be used to classify games for the purpose of choosing an appropriate and QoE-driven video codec configuration strategy.

Cloud gaming represents a highly interactive service whereby game logic is rendered in the cloud and streamed as a video to end devices. While benefits include the ability to stream high-quality graphics games to practically any end user device, drawbacks include high bandwidth requirements and very low latency. Consequently, a challenge faced by cloud gaming service providers is the design of algorithms for adapting video streaming parameters to meet the end user system and network resource constraints. In this paper, we conduct an analysis of the commercial NVIDIA GeForce NOW game streaming platform adaptation mechanisms in light of variable network conditions. We further conduct an empirical user study involving the GeForce NOW platform to assess player Quality of Experience when such adaptation mechanisms are employed. The results provide insight into limitations of the currently deployed mechanisms, as well as aim to provide input for the proposal of designing future video encoding adaptation strategies.

Massive Multiplayer Online Role Playing Games (MMORPGs) have been reported as real-time applications using TCP. This surprising fact makes it interesting to study their behavior when sharing a link with other TCP flows. In this paper the coexistence of an MMORPG flow and an FTP background application is studied. A network scenario which fits an access network is simulated with NS2, using synthetic traffic of a game, generated according to a statistical model. The FTP background application uses different TCP variants. The results show that TCP Vegas is able to maintain a constant rate while competing with the traffic of the game, since it prevents packet loss and high queuing delays by avoiding the increase of the sending window size. In contrast, TCP SACK and TCP New Reno tend to indefinitely increase their window size, thus causing packet loss and adding undesired delays to the game traffic. Finally, small buffers have been proved to be more convenient when dealing with real-time applications, since big ones add higher delays.