Collaborative training applications leveraging Augmented Reality (AR) are increasingly being used across various domains to provide immersive and interactive experiences. In this paper, we report on a user study designed to evaluate and compare user experiences across heterogeneous AR platforms used simultaneously within a collaborative co-located cybersecurity training application called Login PARtners. 36 participants were organised into pairs, with one participant using a Head-Mounted Display (HMD), HoloLens 2, and another a Hand-Held Device (HHD), an Android mobile phone, representing different affordability and availability spectra. Through a set of questionnaires, we investigated users’ perception of the virtual environment’s visual aspects, virtual object interaction and gestures, as well as collaboration, across both devices. While results showed some significant differences between HMD and HHD users, namely with respect to perceived virtual object three-dimensionality and interaction intuitiveness, both participant groups generally found the collaborative experience to be successful.

The recent improvements of artificial intelligence (AI) technologies present a revolutionary opportunity for a wide variety of services. One of the most impactful AI technologies, gaining hundreds of millions of users in mere months, are conversational agents based on Large Language Models (LLMs). An obvious application example of conversational agents is found in gaming, where players often engage in conversation with non-player-characters (NPCs). The use of contemporary AI in this context is expected to bring unprecedented levels of adaptability and freedom of expression to in-game conversations, but may come with its own set of drawbacks, such as increased response latency, which may impair user experience during such interactions. In this paper we present the results of a user study investigating the impact of two different types of latency — time to first token and time per output token — on the overall Quality of Experience and player experience in a conversational task with an LLM-powered NPC in a text-based fantasy role-playing game. Conducted with a sample of fairly regular AI users (N=20), our study provides valuable insights regarding the possible threshold for noticeable latency for this type of service.

While the topic of investigating user experience with immersive services, such as Social Virtual Reality (VR), is starting to gain traction in the research community, the unique case of multiplayer VR games requires a more specific approach. Attempts to investigate user experiences with this complex, multidimensional service are hindered by the absence of specific standards and guidelines going beyond what we know about non-immersive gaming. In this article, we present the results of a user study (N = 32) exploring participants’ experience of playing two competitive VR games of different genres (shooter, sports game), as we focus on three distinct facets of multiplayer VR gaming—network, interpersonal competitiveness, and social interaction. Furthermore, approaching the issue from the perspective of Quality of Experience researchers looking to conduct future user studies on the topic, we also present our findings as a way to shed light on factors that need further consideration, especially in the context of participant recruitment and methodology design, such as the choice of access network to be used for testing, and the issue of matchmaking study participants based on skill and prior relationship.

While factors such as discomfort due to ergonomic design, cybersickness, and accessibility have been studied, industry platforms offering a wide range of VR apps have been found to report only limited 'comfort ratings' (e.g., Meta), age ratings confirming appropriateness of content (e.g., Steam, Microsoft), or seizure warnings (e.g., Microsoft). In this position paper, we argue that such ratings are insufficient in terms of appropriately warning users and enabling them to make informed decisions on which apps to use. We therefore call for a joint industry and academic initiative toward proposing a standardized method of rating the comfort, accessibility, and safety of publicly available VR applications, thus aiming to address the diverse capabilities, requirements, and preferences of VR users.

The capabilities of commercial virtual reality (VR) systems can be utilized to implement a wide spectrum of interaction and locomotion methods. However, immersion in VR is often accompanied by undesirable physical sensations that undermine the VR gaming experience. Presenting the results of a user study (N=20) comparing the physiological and cognitive aftereffects of three different VR games with game mechanics centered around natural interaction, this paper attempts to shed light on the impact of different content factors (e.g., game genre, visual environment, interaction method) on user comfort. While moving away from cybersickness as the primary measure of VR-induced discomfort through investigating a broader set of physical symptoms, such as device-related discomfort, muscle fatigue, and pain, we also include a discussion on the complexities of measuring cybersickness for physically demanding games. Further focusing on cognitive performance, we discuss preliminary findings regarding different effects of common VR game interactions (pick and place, slicing, and shooting) on reaction time.

In this paper we present the results of a user study focused on evaluating the overall usability of a wearable video-communication system aimed at facilitating hands-free communication between a field technician and a remote expert. An asymmetrical cooperative electronics-based bomb defusal game was used to examine the subject's ability to perform a manual technical task with the aid of a head-mounted-display (compared to only verbal communication) in terms of both subjective (perceived usefulness for the given task) and objective (time to complete task, number of errors) metrics. We analyze different dimensions of usability (overall system usefulness, information quality, interface quality, and device comfort) as we discuss the impact of human factors (i.e., gender, wearing Rx eye-glasses) on user experience. While system use has been shown to improve both subjective and objective metrics in some cases, results appear to be highly dependant on the task itself. Furthermore, our findings highlight the importance of exploring different ways in which ergonomic issues present in commercial smart glasses may decrease perceived usability in certain demographics.

In this paper we present the results of two user studies aimed at exploring the effects of network latency on user experience in a First Person Shooter Virtual Reality multiplayer game. In both studies we observed the following metrics: Quality of Experience (QoE), willingness to continue playing in given network conditions, and the final outcome of the game (survival/death). After conducting the first study, we concluded that test methodology and contextual factors (e.g., social context, level of difficulty, weapon choice) may have had a strong influence on perceived QoE, subsequently causing inconclusive results. Therefore, we aimed to mitigate those factors through different methodology choices in the second study, which led to more conclusive, statistically significant results, indicating that user experience for the chosen game begins to suffer in cases of latency greater than approx. 100 ms (round trip time between client and server). In addition to the analysis of acquired results, we discuss the significance of methodology and context when conducting studies exploring gaming QoE in multiplayer environments.

This paper presents the results of a subjective study aimed at assessing different locomotion (may also be referred to as navigation) techniques in virtual reality environments. We implemented a simple virtual environment in which participants were asked to assess the following four techniques using the HTC Vive platform: teleportation, controller movement, controller movement with tunneling vision, and human joystick. A total of 29 participants took part in the study and rated perceived level of immersion, physical discomfort, and overall Quality of Experience for all tested locomotion techniques. Moreover, participants indicated whether they needed a break after a specific test due to physical discomfort. The results indicate that teleportation yields overall highest QoE and lowest discomfort.