Similarity estimation is essential for many game AI applications, from the procedural generation of distinct assets to automated exploration with game-playing agents. While similarity metrics often substitute human evaluation, their alignment with our judgement is unclear. Consequently, the result of their application can fail human expectations, leading to e.g. unappreciated content or unbelievable agent behaviour. We alleviate this gap through a multi-factorial study of two tile-based games in two representations, where participants (N=456) judged the similarity of level triplets. Based on this data, we construct domain-specific perceptual spaces, encoding similarity-relevant attributes. We compare 12 metrics to these spaces and evaluate their approximation quality through several quantitative lenses. Moreover, we conduct a qualitative labelling study to identify the features underlying the human similarity judgement in this popular genre. Our findings inform the selection of existing metrics and highlight requirements for the design of new similarity metrics benefiting game development and research.

The evaluation of procedural content generation (PCG) systems for generating video game levels is a complex and contested topic. Ideally, the field would have access to robust, generalisable and widely accepted evaluation approaches that can be used to compare novel PCG systems to prior work, but consensus on how to evaluate novel systems is currently limited. We argue that the field can benefit from a structured analysis of how procedural level generation systems can be evaluated, and how these techniques are currently used by researchers. This analysis can then be used to both inform on the current state of affairs, and to provide data to justify changes to this practice. This work aims to provide this by first developing a novel taxonomy of PCG evaluation approaches, and then presenting the results of a survey of recent work in the field through the lens of this taxonomy. The results of this survey highlight several important weaknesses in current practice which we argue could be substantially mitigated by 1) promoting use of evaluation free system descriptions where appropriate, 2) promoting the development of diverse research frameworks, 3) promoting reuse of code and methodology wherever possible.

Expressive Range Analysis (ERA), an approach for visualising the output of Procedural Content Generation (PCG) systems, is widely used within PCG research to evaluate and compare generators, often to make comparative statements about their relative performance in terms of output diversity and search space exploration.Producing a standard ERA visualisation requires the selection of two metrics which can be calculated for all generated artefacts to be visualised.However, to our knowledge there are no methodologies or heuristics for justifying the selection of a specific metric pair over alternatives.Prior work has typically either made a selection based on established but unjustified norms, designer intuition, or has produced multiple visualisations across all possible pairs.This work aims to contribute to this area by identifying valuable characteristics of metric pairings, and by demonstrating that pairings that have these characteristics have an increased probability of producing an informative ERA projection of the underlying generator.We introduce and investigate three quantifiable selection criteria for assessing metric pairs, and demonstrate how these criteria can be operationalized to rank those available.Though this is an early exploration of the concept of quantifying the utility of ERA metric pairs, we argue that the approach explored in this paper can make ERA more useful and usable for both researchers and game designers.

The past decade has seen a rapid increase in the level of research interest in procedural content generation (PCG) for digital games, and there are now numerous research avenues focused on new approaches for driving and applying PCG systems. An area in which progress has been comparatively slow is the development of generalisable approaches for comparing alternative PCG systems, especially in terms of their generative spaces. It is to this area that this paper aims to make a contribution, by exploring the utility of data compression algorithms in compressing the generative spaces of PCG systems. We hope that this approach could be the basis for developing useful qualitative tools for comparing PCG systems to help designers better understand and optimize their generators. In this work we assess the efficacy of a selection of algorithms across sets of levels for 2D tile-based games by investigating how much their respective generative space compressions correlate with level behavioral characteristics. We conclude that the approach looks to be a promising one despite some inconsistency in efficacy in alternative domains, and that of the algorithms tested Multiple Correspondence Analysis appears to perform the most effectively.

Sensorimotor interactions in the physical world and in immersive virtual reality (IVR) offer different feedback. Actions in the physical world almost always offer multi-modal feedback: pouring a jug of water offers tactile (weight-change), aural (the sound of running water) and visual (water moving out the jug) feedback. Feedback from pouring a virtual jug, however, depends on the IVR’s design. This study examines if the richness of feedback from IVR actions causes a detectable cognitive impact on users. To do this, we compared verb-learning outcomes between two conditions in which participants make actions with objects and (1) audiovisual feedback is presented; (2) audiovisual feedback is not presented. We found that participants (n = 74) had cognitively distinct outcomes based on the type of audiovisual feedback experienced, with a high feedback experience harming learning outcomes compared with a low feedback one. This result has implications for IVR system design and theories of cognition and memorisation.

Extended Reality (XR) technology - such as virtual and augmented reality - is now widely used in Human Computer Interaction (HCI), social science and psychology experimentation. However, these experiments are predominantly deployed in-lab with a co-present researcher. Remote experiments, without co-present researchers, have not flourished, despite the success of remote approaches for non-XR investigations. This paper summarises findings from a 30-item survey of 46 XR researchers to understand perceived limitations and benefits of remote XR experimentation. Our thematic analysis identifies concerns common with non-XR remote research, such as participant recruitment, as well as XR-specific issues, including safety and hardware variability. We identify potential positive affordances of XR technology, including leveraging data collection functionalities builtin to HMDs (e.g. hand, gaze tracking) and the portability and reproducibility of an experimental setting. We suggest that XR technology could be conceptualised as an interactive technology and a capable data-collection device suited for remote experimentation.

New opportunities for immersive storytelling experiences have arrived through the technology in mobile phones, including the ability to overlay or register digital content on a user’s real world surroundings, to greater immerse the user in the world of the story. This raises questions around the methods and freedom to interact with the digital elements, that will lead to a more immersive and engaging experience. To investigate these areas the Augmented Virtuality (AV) mobile phone application Home Story was developed for iOS devices. It allows a user to move and interact with objects in a virtual environment displayed on their phone, by physically moving in the real world, completing particular actions to progress a story. A mixed methods study with Home Story either guided participants to the next interaction, or offered them increased agency to choose what object to interact with next. Virtual objects could also be interacted with in one of three ways; imagining the interaction, an embodied interaction using the user’s free hand, or a virtual interaction performed on the phone’s touchscreen. Similar levels of immersion were recorded across both study conditions suggesting both can be effective, though highlighting different issues in each case. The embodied free hand interactions proved particularly memorable, though further work is required to improve their implementation, arising from their novelty and lack of familiarity.

Modern immersive virtual reality (IVR) often uses embodied controllers for interacting with virtual objects. However, it is not clear how we should conceptualise these interactions. They could be considered either gestures, as there is no interaction with a physical object; or as actions, given that there is object manipulation, even if it is virtual. This distinction is important, as literature has shown that in the physical world, action-enabled and gesture-enabled learning produce distinct cognitive outcomes. This study attempts to understand whether sensorimotor-embodied interactions with objects in IVR can cognitively be considered as actions or gestures. It does this by comparing verb-learning outcomes between two conditions: (1) where participants move the controllers without touching virtual objects (gesture condition); and (2) where participants move the controllers and manipulate virtual objects (action condition). We found that (1) users can have cognitively distinct outcomes in IVR based on whether the interactions are actions or gestures, with actions providing stronger memorisation outcomes; and (2) embodied controller actions in IVR behave more similarly to physical world actions in terms of verb memorization benefits.

Immersive virtual environments (IVEs) are increasingly being explored as potential educational tools. However, it is unclear which aspects of IVEs contribute to learning, including hardware modalities and learner responses (e.g. motivation, usability, cognitive load and presence). One IVE hardware modality particularly backed by theory is embodied controls, with their potential for leveraging embodied cognition for enhanced learning outcomes. This paper explores if embodied controls can be leveraged to enhance learning in an IVE by comparing language learning outcomes from an IVE using embodied controls, and a non-embodied control. It explores two words classes - verbs and nouns - to examine if there is a difference in learning outcome for embodied controls with actions (verbs) and object interactions (nouns). This paper also explores co-variables often linked with IVE learning (motivation, presence, cognitive load) to understand why learning gain occurs. It finds that leveraging embodied controls provides better learning outcomes, with no impact on cognitive load. It also finds that the benefit does not correlate with motivation or presence ratings, suggesting that embodiment-induced motivation or immersion is not the cause of the learning enhancements, and therefore this could be evidence for embodied cognition-based learning in IVEs.

The use of immersive virtual environments (IVEs) for educational purposes has increased in recent years, but the mechanisms through which they contribute to learning is still unclear. Popular explanations for the learning benefits brought by IVEs come from motivation, presence and embodied perspectives; either as individual benefits or through mediation effects on each other. This paper describes an experiment designed to interrogate these approaches, and provides evidence that embodied controls and presence encourage learning in immersive virtual environments, but for distinct,non-interacting reasons, which are also not explained by motivational benefits.

Video games have proved very valuable in rehabilitation technologies. They guide therapy and keep patients engaged and motivated. However, in order to realize their full potential, a good understanding is required of the players’ motor control. In particular, little is known regarding player behaviour in tasks demanding bimanual interaction. In this work, an experiment was designed to improve the understanding of such tasks. A driving game was developed in which players were asked to guide a differential wheeled robot (depicted as a rocket) along a trajectory. The rocket could be manipulated by using an Xbox controller’s triggers, each supplying torque to the corresponding side of the robot. Such a task is redundant, i.e. there exists an infinite number of input combinations to yield a given outcome. This allows the player to strategize according to their own preference. 10 participants were recruited to play this game and their input data was logged for subsequent analysis. Two different motor strategies were identified: an "intermittent" input pattern versus a "continuous" one. It is hypothesized that the choice of behaviour depends on motor skill and minimization of effort and error. Further testing is necessary to determine the exact relationship between these aspects.

In this paper, we present PlayMapper, a novel variant of the MAP-Elites algorithm that has been adapted to map the level design space of the Super Mario Bros game. Our approach uses player and level based features to create a map of playable levels. We conduct an experiment to compare the effect of different sets of input features on the range of levels generated using this technique. In this work, we show that existing search-based techniques for PCG can be improved to allow for more control and creative freedom for designers. Current limitations of the system and directions for future work are also discussed.