Virtual Reality (VR) promises to benefit training and education by offering ubiquitous risk-free access to handson experiential learning. This paper presents an evaluation of a VR-based occupational safety training application for forestry work, assessing the VR system’s impact on training effectiveness, including skill transfer to real world settings. 73 participants experienced two learning scenarios covering different task types and settings. They were either assigned to the VR training or relied on traditional forestry training materials only.Our results confirm the positive effects of VR-based training on participants’ experience as well as successful skill transfer to real-world challenges, as evidenced by participant feedback on a broad range of experience measures. In addition, our study extends existing work by examining how varying VR training quality affects both subjective experience and objective performance outcomes in terms of learning transfer. Our findings indicate that better technical quality of the VR training indeed can translate to significantly improved performance during skill application in subsequent test situations. Surprisingly, we could not detect a similarly pronounced effect when analyzing participants’ subjective experience ratings. We reflect on these findings and discuss methodological implications for experience assessment in the context of VR-based training and education.

eXtended Reality (XR) environments appear to be a promising approach to training in various contexts; however, their potential for safety and medical procedures is still not fully understood. The aim of the presented research project is the user-centered implementation of an XR training for various occupational groups and volunteers of first responder organizations (e.g., physicians, paramedics, health care workers, police officers, firefighters). To achieve this aim, the project has followed three implementation steps: First, multi-stakeholder workshops were conducted to define two use cases for the most promising application areas. Second, based on the previous step, identified use cases were implemented as XR training scenarios. Third, scientific evaluations of the training applications are conducted focusing on subjective and objective performance indicators, learning success, training experience, acceptance, and potential barriers from an end-user perspective. As the last step is still ongoing, this paper puts emphasis on the outcomes of the stakeholder workshops, the respective decisions taken towards selecting two use cases, and the design of the evaluation study.

Training in virtual reality (VR) is widely considered as promising approach to address the rapidly rising demand for adequately trained healthcare professionals. In this paper, we present an experimental study assessing the experience and learning effects of a VR-based training for healthcare professionals (scenario: preparation of an endotracheal intubation). 41 participants from the field of healthcare and nursing were randomly assigned to an intervention group training in VR and a control group training with traditional printed materials. Subsequently, both groups had to apply their newly acquired skills in a real-world practice environment to objectively assess training effectiveness at high validity. Our results demonstrate the added value of the VR training in terms of trainees' significantly reduced time to solve the task in the real-world environment. Moreover, statistical analysis of experience measures (hedonic and pragmatic user experience, sense of presence, VR-induced symptoms, quality of experience, overall satisfaction) confirms participants' appreciation of the quality of the VR-based training. Surprisingly, however, our results do not indicate a significant link between VR training experience and real-world task performance. We discuss implications of these findings and outline a research agenda to better understand the key factors that influence the effectiveness of VR-based training.

Abstract Virtual Reality (VR) training has become increasingly important for police first responders in recent years. Improving the training experience in such complex contexts requires ecological validity of virtual training. To achieve this, VR systems need to be capable of simulating the complex experiences of police officers ‘in the field.’ One way to do this is to add stressors into training simulations to induce stress similar to the stress experienced in real-life situations, particularly in situations where this is difficult (e.g., dangerous or resource-intensive) to achieve with traditional training. To include stressors in VR, this paper thus presents the concept of so-called ‘stress cues’ for operationalizing stressors to augment training in VR simulations for the context of police work. Considering the level of complexity of police work and training, a co-creation process that allows for creative collaboration and mitigation of power imbalances was chosen to access the police officers’ knowledge and experience. We assert that stress cues can improve the training experience from the trainer’s perspective as they provide novel interaction design possibilities for trainers to control the training experience. E.g., by actively intervening in training and dynamically changing the interaction space for trainees which also improves the trainee’s experience. Stress cues can also improve the trainee’s experience by enabling personalizable and customizable training based on real-time stress measurements and supplementing information for improved training feedback.

Haptic qualities of surfaces play an important role for the user's experience with a system. In that context, soft touch surfaces are important as they are often used for surfaces of industrial products. For assessing related experiences, research is often focusing on physical properties of the surfaces rather than the (quality of) experience of the user. Therefore, we set out to analyze how people describe their experience of surface properties in a bottom-up approach. In an Open Profiling study we let people self-describe, cluster and rate surface properties. Based on these ratings we applied a Principal Component Analysis to identify characteristics and related perception dimensions of soft touch surfaces. Furthermore, we complement this with an expert clustering of user self-descriptions. Our analysis shows that descriptions can be categorized in pragmatic and hedonic (and/or affective) dimensions. With this research we contribute to better understanding the quality of material related experiences.

Large-scale events pose severe challenges to live video streaming service providers, who need to cope with high, peaking viewer numbers and the resulting fluctuating resource demands, keeping high levels of Quality of Experience (QoE) to avoid end-user frustration and churn. In this paper, we analyze a unique dataset consisting of more than a million 2018 FIFA World Cup mobile live streaming sessions, collected at a large national public broadcaster. Different from previous work, we analyze QoE and user engagement as well as their interaction, in dependency to specific soccer match events, which have the potential to trigger flash crowds during a match. Flash crowds are a particular challenge to video service providers, since they cause sudden load peaks and consequently, the likelihood of quality problems. We further exploit the data to model viewer engagement over the course of a soccer match, and show that client counts follow very similar patterns of change across all matches. We believe that the analysis as well as the resulting models are valuable sources of insight for service providers, equipping them with tools for customer-centric resource and capacity management.

Current developments in digitized industrial manufacturing lead to work processes that are highly complex, frequently change and thus increase workload of staff. In this respect, assistive technologies have the potential to aid workers in better coping with increasing workload and requirements and consequently, maintaining good working experience(s). However, existing research on interactive assistive technologies has hitherto largely neglected work experience as a measurable construct. We addressed this shortcoming by conducting a subjective study that assesses different work experience dimensions in the context of interactive augmented reality assembly instructions, in a realistic industrial setting, with a decent sample size (N=48). In this study, we assessed work experience along the dimensions of perceived utility, hedonic user experience, and satisfaction with the assistive technologies used. On behalf of this approach we were able to detect a significant effect between perceived utility and satisfaction. In addition, we summarize lessons learned from this study as well as resulting implications and discuss open challenges for a research agenda on work experience in industry 4.0 contexts.

The experience of web services depends on multiple factors such as the usability of the service itself as well as influences on Quality of Experience (QoE) like waiting times. Existing studies so far have investigated both experimental factors (usability and QoE) in isolation, thus it is not possible to investigate dependencies between the two. In this study, we manipulated the usability and QoE of a news web page in the same experiment. Participants had to solve 9 tasks (three usability level and three QoE level, all combinations). Data (N=44) from two different sessions with different QoE manipulations were recorded: Simulation of initial page display delay via 1) a complete overlay and 2) a semi-transparent overlay. Our preliminary results show that the usability manipulation resulted in significant effects along multiple dimensions (QoE, aesthetic, and usability). Furthermore, also the implementation of the QoE manipulation had an effect on, e.g., performance parameters. Surprisingly, our results reveal that participants seem to be unable to rate the constructs usability and QoE independently from each other. Even after adding pronounced explanations of the constructs to the introductory phase of the second session of the experiment, the results still show a strong correlation between the two constructs. We thus recommend for similar future experiments to check beforehand whether Quality of Experience and Usability can be reliably judged and distinguished by naive participants.

This paper takes a view on the Quality-of-Experience (QoE) perspective of Cyber-Physical System (CpS) applications involving users. We provide an overview of characteristics of CPS applications, their relation to QoE and implications for assessing QoE of such systems. Furthermore, we incorporate eudaimonic aspects such as meaningfulness and personal growth that need to be considered in order to grasp the strongly multi-dimensional aspects of QoE in this context.

Today's packet-switched networks are subject to bandwidth fluctuations that cause degradation of the user experience of multimedia services. In order to cope with this problem, HTTP adaptive streaming (HAS) has been proposed in recent years as a video delivery solution for the future Internet and being adopted by an increasing number of streaming services, such as Netflix and Youtube. HAS enables service providers to improve users' quality of experience (QoE) and network resource utilization by adapting the quality of the video stream to the current network conditions. However, the resulting time-varying video quality caused by adaptation introduces a new type of impairment and thus novel QoE research challenges. Despite various recent attempts to investigate these challenges, many fundamental questions regarding HAS perceptual performance are still open. In this paper, the QoE impact of different technical adaptation parameters, including chunk length, switching amplitude, switching frequency, and temporal recency, are investigated. In addition, the influence of content on perceptual quality of these parameters is analyzed. To this end, a large number of adaptation scenarios have been subjectively evaluated in four laboratory experiments and one crowdsourcing study. A statistical analysis of the combined data set reveals results that partly contradict widely held assumptions and provide novel insights in perceptual quality of adapted video sequences, e.g., interaction effects between quality switching direction (up/down) and switching strategy (smooth/abrupt). The large variety of experimental configurations across different studies ensures the consistency and external validity of the presented results that can be utilized for enhancing the perceptual performance of adaptive streaming services.

In this work, we identify influencing factors on modality choices of older adults. In detail, we investigated when and why older adults prefer speech over touch interaction and vice versa when interacting with a mobile multimodal health and wellbeing service. We conducted a study with 19 older adults using a mobile application with a duration of three to six weeks. Due to this long duration of the study we were able to gain highly external valid insights as our results are based on real world experiences. We identify additional influencing factors within the areas of user characteristics, contextual factors and perceived system characteristics. We outline the impact of the factors and highlight the importance of several of these factors to enable accessible user interfaces. Our results provide first steps towards a more holistic model of modality choices taking into account interdependencies of different factors.

Since its introduction a few years ago, the concept of `Crowdsourcing' has been heralded as highly attractive alternative approach towards evaluating the Quality of Experience (QoE) of networked multimedia services. The main reason is that, in comparison to traditional laboratory-based subjective quality testing, crowd-based QoE assessment over the Internet promises to be not only much more cost-effective (no lab facilities required, less cost per subject) but also much faster in terms of shorter campaign setup and turnaround times. However, the reliability of remote test subjects and consequently, the trustworthiness of study results is still an issue that prevents the widespread adoption of crowd-based QoE testing. Various ideas for improving user rating reliability and test efficiency have been proposed, with the majority of them relying on a posteriori analysis of results. However, such methods introduce a major lag that significantly affects efficiency of campaign execution. In this paper we address these shortcomings by introducing in momento methods for crowdsourced video QoE assessment which yield improvements of results reliability by factor two and campaign execution efficiency by factor ten. The proposed in momento methods are applicable to existing crowd-based QoE testing approaches and suitable for a variety of service scenarios.

A new model of speech quality under delay is presented that includes conversational interactivity. It is based on two previously reported narrowband telephony conversation tests involving different delays, with subject-pairs judging overall quality after each conversation. The tests were conducted with different conversation scenarios targeting different levels of interactivity. The instructions given prior to the tests were varied in their emphasis on speed of task completion. Based on the test results, the paper proposes an extension of a widely used conversational speech quality model, the so-called E-model (ITUT Rec. G.107), to cover the joint effect of interactivity and delay. To this aim, two new parameters are introduced, one of which represents the minimum perceivable delay, and the other expresses in how far users will attribute the delay-effect to the conversational quality of the line. Based on the analysis of the recorded test conversations in terms of its surface structure (turns, speaker activities, etc.), prominent differences and delay-dependencies of a number of conversation parameters were found that characterize the impact of delay on the conversational flow and on perceived quality.

Over the last couple of years, the scope of Quality of Experience (QoE) research has been constantly extended, most recently to the field of Web QoE in the context of HTTP-based applications. In this paper, we address the question whether it is sufficient to reduce typical Web QoE assessment scenarios to the temporal aspects of waiting for task completion, which would allow to attribute the resulting logarithmic laws to well-known psychological insights on human time perception. We demonstrate that while this attribution is valid for simple waiting tasks which are typical for simple data services like e.g. file downloads, the case of interactive web browsing is much more complex. We show that this is not only because technical issues prevent bandwidth and download time from being directly correlated with each other in a simple manner, but also because user perceived web page load times strongly deviate from technical page load times. Consequently, existing approaches towards assessment and modeling of web browsing QoE have to be critically reviewed and redesigned.

In this paper we demonstrate how speech signal features can be used to detect and explain differences in human to human conversation tests. To this end, we compare the results of two conversational VoIP quality experiments designed to quantify the impact of network delay on perceived speech quality. Both studies followed the same procedures and used the same scenarios, but were conducted in two different labs. Our comparison shows that the two studies, despite having been executed correctly using the same test design, still can produce surprisingly different results regarding the users quality perception on a MOS scale. In this respect, speech signal features extracted from conversation recordings help identifying divergent participant behavior as plausible cause for such differences. Our in-depth analysis reveals how novel parameters developed by us like Intended and Unintended Interruption Rate (IIR, UIR) and the corrected Speaker Alternation Rate SARcorrcan be used to successfully determine the extent to which the results of different conversational speech quality studies are directly comparable and thus eligible for pooling, or not.

Due to the accelerating uptake of mobile broadband services, Quality of Experience (QoE) assessment and optimization for wireless networks has become a topic of vital importance. In this field, subjective quality evaluation methods based on MOS (Mean Opinion Score) are the most widely used approaches for understanding and measuring perceived performance. However, while these methods have been proven to reliably quantify users' degrees of satisfaction, they do not sufficiently address the following important question: "Which quality levels are acceptable to end users and which ones not?". In this paper we investigate the application of acceptability to the domain of QoE assessment for data services. To this end, we perform a critical review of the term's utilization and operationalization in related fields and discuss the results of a series of mobile broadband QoE studies we performed in lab and field settings. Our results demonstrate that acceptability is a relevant and useful concept that also features a consistent mapping between subjects' binary acceptance and ordinal satisfaction ratings, even across application scenarios as different as web browsing and file downloads. However, it is also a highly context-dependent concept since users' rating behaviors and thresholds diverge between lab and field settings. Acceptability thus needs to be evaluated in situ in the field in order to guarantee generalizable, ecologically valid results.

The Weber-Fechner Law (WFL) is an important principle in psychophysics which describes the relationship be- tween the magnitude of a physical stimulus and its perceived intensity. With the sensory system of the human body, in many cases this dependency turns out to be of logarithmic nature. Re- cent quantitative QoE research shows that in several different scenarios a similar logarithmic relationship can be observed be- tween the size of a certain QoS parameter of the communication system and the resulting QoE on the user side as observed during appropriate user trials. In this paper, we discuss this surprising link in more detail. After a brief survey on the background of the WFL, we review its basic implications with respect to related work on QoE assessment for VoIP, most notably the recently published IQX hypothesis, before we present results of our own trials on QoE assessment for mobile broadband scenarios which confirm this dependency also for data services. Finally, we point out some conclusions and directions for further research.