Computer games have evolved alongside technologies such as personal computers, touchscreens, and virtual reality (VR). The devices have different interaction modes with distinct affordances, including direct physical control with tablets and immersive experiences in VR. Moreover, games are now easily accessible across devices due to cloud-based streaming. Despite this benefit, the challenge of delay persists and continues to affect both performance and quality of experience in games. The negative impact of delay is well-documented, but less is known about how its effect may interplay with a device’s interaction mode. This study applies a cross-platform game with controlled levels of delay, where players use different interaction modes to select moving targets. The findings highlight how the detrimental effect of delay on player performance degradation varies across devices (worst on a PC compared to VR and a tablet, and best on the tablet) even while the interaction mode mediates the effects of delay (quality of experience is fairly stable across all conditions and devices, with VR the most immersive).

Despite the long history of virtual reality (VR) technology and its current availability, it remains unclear how the general public perceives and uses the technology. To shed light on this, we conducted a panel survey with 931 respondents in the spring of 2021. The survey revealed that while most people knew what VR is, only 20% had tried it; even among the age groups below 40 years, the share with VR experience ranged from 34 to 54%. Moreover, the vast majority were infrequent users, only 0.6% of the full sample used VR weekly. Reasons provided for not trying VR were mainly related to either lack of interest or lack of opportunity, while infrequent usage seemed to be linked to poor user experience. User experience was in turn tied to sound and image quality, novelty, usability, content, responsiveness and cybersickness symptoms such as vertigo and nausea. This suggests that those who are already familiar with VR require high technical quality as well as high creative quality if they are to adopt VR as a consumer technology.

This paper presents the finding of a subjective study investigating the impact of delay on a VR gaming experience, applying an experimental design to a virtual game of squash. 32 participants were asked to rate the stability of the VR experience while playing the game, with results showing an impact of delay even at the lowest level of 20 ms added delay. With 120 ms added delay, most participants judged the experience to be fully unstable. Exploring individual differences, we also found that males and experienced gamers were more critical, or more sensitive, to higher levels of delay, compared to females and less experienced gamers.

When software and hardware processes cause delay between an input and an output, consequences can be both harmful and annoying. In games, this type of delay can cause actions to lag behind a player's inputs. Studies on delayed game actions show negative impact on both player performance and quality of experience. Further research could expand upon this knowledge with more robust data modeling and analysis, as well as a wider range of actions. This paper describes publicly available datasets from four user studies on the effects of delay on the fundamental game action of selecting a moving target with a pointing device. They include performance data, such as time to selection, and demographic data, such as age and gaming experience. Example analysis illustrating the use of the datasets is shown, including comparison of pointing devices, demographic analysis with performance, and analytic modeling.

In this paper, we present PMData: a dataset that combines traditional lifelogging data with sports-activity data. Our dataset enables the development of novel data analysis and machine-learning applications where, for instance, additional sports data is used to predict and analyze everyday developments, like a person's weight and sleep patterns; and applications where traditional lifelog data is used in a sports context to predict athletes' performance. PMData combines input from Fitbit Versa 2 smartwatch wristbands, the PMSys sports logging smartphone application, and Google forms. Logging data has been collected from 16 persons for five months. Our initial experiments show that novel analyses are possible, but there is still room for improvement.

Delays are ever-present in interactive tasks, such as controlling a cursor with a mouse. Unfortunately, perceptible delays are likely to affect both quality of experience and task performance, and even imperceptible delays can potentially be harmful to performance. This paper presents a controlled behavioural experiment that explores the impact of delay on interactions with motor inputs and visual outputs. Because system and network delays interact and overlap, we address total interface delay, focusing on the effect rather than the cause. In the experiment, 51 participants played a simple game of chase-and -catch, using the mouse to intercept a bouncing target. The game includes three levels of difficulty, defined by the speed of the target, with controlled interface delay added between the mouse and the corresponding cursor. The delay values ranged from the system's minimum processing time of 40 ms up to a total of 440 ms. We evaluated participants' game performance, as well as perceived game responsiveness. In line with predictions, our analyses show a negative relation between delay and quality of experience, along with deteriorating performance. In contrast, performance does not co-vary with self-reported game skill. Moreover, an individual's experience with other time-dependent activities has no significant effect on neither performance nor experience, with one exception - musical practice appears to benefit performance for this type of interaction.

The speedy interactions of many computer games demand prompt reactions from player and system alike. This demonstration invites participants to experience in-game latency. We present variations of two familiar games that use either mouse or key inputs to control actions seen on the monitor. While playing the games, participants can make changes to the temporal offset between a motor input and a resulting visual action, providing them with a direct experience of motor-visual delays. In turn, they will gain an understanding of how performance may suffer when trying to compensate for such delays.

Interactive panoramic systems are currently on the rise. However, one of the major challenges in such a system is the overhead involved in transferring a full-quality panorama to the client when only a part of the panorama is used to extract a virtual view. Thus, such a system should maximize the user experience while simultaneously minimizing the bandwidth required. In this paper, we apply tiling to deliver different quality levels for different parts of the panorama. Tiling has traditionally been applied to the delivery of very high-resolution content to clients. Here, we apply similar ideas in a real-time interactive panoramic video system. A major challenge lies in the movement of such a virtual view, for which clients' regions of interest change dynamically and independently from each other. We show that our algorithms, which progressively increase in quality toward the point of the view, manage to (i) reduce the bandwidth requirement and (ii) provide a similar quality of experience (QoE) compared to a full panorama system.

Rules-of-thumb for noticeable and detrimental asynchrony between audio and video streams have long since been established from the contributions of several studies. Although these studies share similar findings, none have made any discernible assumptions regarding audio and video quality. Considering the use of active adaptation in present and upcoming streaming systems, audio and video will continue to be delivered in separate streams; consequently, the assumption that the rules-of-thumb hold independent of quality needs to be challenged. To put this assumption to the test, we focus on the detection, not the appraisal, of asynchrony at different levels of distortion. Cognitive psychologists use the term temporal integration to describe the failure to detect asynchrony. The term refers to a perceptual process with an inherent buffer for short asynchronies, where corresponding auditory and visual signals are merged into one experience. Accordingly, this paper discusses relevant causes and concerns with regards to asynchrony, it introduces research on audiovisual perception, and it moves on to explore the impact of audio and video quality on the temporal integration of different audiovisual events. Three content types are explored, speech from a news broadcast, music presented by a drummer, and physical action in the form of a chess game. Within these contexts, we found temporal integration to be very robust to quality discrepancies between the two modalities. In fact, asynchrony detection thresholds varied considerably more between the different content than they did between distortion levels. Nevertheless, our findings indicate that the assumption concerning the independence of asynchrony and audiovisual quality may have to be reconsidered.

With teleconferencing becoming more accessible as a communication platform, researchers are working to understand the consequences of the interaction between human perception and this unfamiliar environment. Given the enclosed space of a teleconference room, along with the physical separation between the user, microphone and speakers, the transmitted audio often becomes mixed with the reverberating auditory components from the room. As a result, the audio can be perceived as smeared in time, and this can affect the user experience and perceived quality. Moreover, other challenges remain to be solved. For instance, during encoding, compression and transmission, the audio and video streams are typically treated separately. Consequently, the signals are rarely perfectly aligned and synchronous. In effect, timing affects both reverberation and audiovisual synchrony, and the two challenges may well be inter-dependent. This study explores the temporal integration of audiovisual continuous speech and speech syllables, along with a non-speech event, across a range of asynchrony levels for different reverberation conditions. Non-reverberant stimuli are compared to stimuli with added reverberation recordings. Findings reveal that reverberation does not affect the temporal integration of continuous speech. However, reverberation influences the temporal integration of the isolated speech syllables and the action-oriented event, with perceived subjective synchrony skewed towards audio lead asynchrony and away from the more common audio lag direction. Furthermore, less time is spent on simultaneity judgements for the longer sequences when the temporal offsets get longer and when reverberation is introduced, suggesting that both asynchrony and reverberation add to the demands of the task.

In this article, we argue that the energy spent in designing autonomous camera control systems is not spent in vain. We present a real-time virtual camera system that can create smooth camera motion. Similar systems are frequently benchmarked with the human operator as the best possible reference; however, we avoid a priori assumptions in our evaluations. Our main question is simply whether we can design algorithms to steer a virtual camera that can compete with the user experience for recordings from an expert operator with several years of experience? In this respect, we present two low-complexity servoing methods that are explored in two user studies. The results from the user studies give a promising answer to the question pursued. Furthermore, all components of the system meet the real-time requirements on commodity hardware. The growing capabilities of both hardware and network in mobile devices give us hope that this system can be deployed to mobile users in the near future. Moreover, the design of the presented system takes into account that services to concurrent users must be supported.

Streaming video over the Internet requires mechanisms that limit the streams' bandwidth consumption within its fair share. TCP streaming guarantees this and provides lossless streaming as a side-effect. Adaptation by packet drop does not occur in the network, and excessive startup latency and stalling must be prevented by adapting the bandwidth consumption of the video itself. However, when the adaptation is performed during an ongoing session, it may influence the perceived quality of the entire video and result in improved or reduced visual quality of experience. We have investigated visual artifacts that are caused by adaptive layer switching -- we call them flicker effects -- and present our results for handheld devices in this paper. We considered three types of flicker, namely noise, blur and motion flicker. The perceptual impact of flicker is explored through subjective assessments. We vary both the intensity of quality changes (amplitude) and the number of quality changes per second (frequency). Users' ability to detect and their acceptance of variations in the amplitudes and frequencies of the quality changes are explored across four content types. Our results indicate that multiple factors influence the acceptance of different quality variations. Amplitude plays the dominant role in delivering satisfactory video quality, while frequency can also be adjusted to relieve the annoyance of flicker artifacts.

Subjective quality perception studies with human observers are essential for multimedia system design. Such studies are known to be expensive and difficult to administer. They require time, a detailed knowledge of experimental designs and a level of control which can often only be achieved in a laboratory setting. Hence, only very few researchers consider running subjective studies at all.

Research has shown that visual information becomes less reliable when images are severely distorted. Furthermore, while voicing is generally identified from acoustical cues, it may also provide perception with visual cues. The current study investigated the impact of video distortion on the audiovisual perception of voicing. Audiovisual stimuli were presented to 30 participants with the original video quality, or with reduced video resolution (75x60 pixels, 45x36 pixels). Results revealed that in addition to increased auditory reliance with video distortion, particularly for voiceless stimuli, perception of voiceless stimuli was more influenced by the visual modality than voiced stimuli. Index Terms: audiovisual perception, speech, voicing, visual distortion