In this short paper we introduce the first version of a 'spell simulator' for 1 Billion Spells, an in development action game, and explore the simulator's utility for conducting large scale exploration and evaluation of that game's spell creation system.Using this simulator we conduct initial experiments to explore specific developer concerns about the game which would be challenging to explore with conventional playtesting.We argue that our initial exploration demonstrates the potential of this approach in evaluating content generation systems for games that have large player explorable possibility spaces.

The evaluation of procedural content generation (PCG) systems for generating video game levels is a complex and contested topic. Ideally, the field would have access to robust, generalisable and widely accepted evaluation approaches that can be used to compare novel PCG systems to prior work, but consensus on how to evaluate novel systems is currently limited. We argue that the field can benefit from a structured analysis of how procedural level generation systems can be evaluated, and how these techniques are currently used by researchers. This analysis can then be used to both inform on the current state of affairs, and to provide data to justify changes to this practice. This work aims to provide this by first developing a novel taxonomy of PCG evaluation approaches, and then presenting the results of a survey of recent work in the field through the lens of this taxonomy. The results of this survey highlight several important weaknesses in current practice which we argue could be substantially mitigated by 1) promoting use of evaluation free system descriptions where appropriate, 2) promoting the development of diverse research frameworks, 3) promoting reuse of code and methodology wherever possible.

Expressive Range Analysis (ERA), an approach for visualising the output of Procedural Content Generation (PCG) systems, is widely used within PCG research to evaluate and compare generators, often to make comparative statements about their relative performance in terms of output diversity and search space exploration.Producing a standard ERA visualisation requires the selection of two metrics which can be calculated for all generated artefacts to be visualised.However, to our knowledge there are no methodologies or heuristics for justifying the selection of a specific metric pair over alternatives.Prior work has typically either made a selection based on established but unjustified norms, designer intuition, or has produced multiple visualisations across all possible pairs.This work aims to contribute to this area by identifying valuable characteristics of metric pairings, and by demonstrating that pairings that have these characteristics have an increased probability of producing an informative ERA projection of the underlying generator.We introduce and investigate three quantifiable selection criteria for assessing metric pairs, and demonstrate how these criteria can be operationalized to rank those available.Though this is an early exploration of the concept of quantifying the utility of ERA metric pairs, we argue that the approach explored in this paper can make ERA more useful and usable for both researchers and game designers.

The past decade has seen a rapid increase in the level of research interest in procedural content generation (PCG) for digital games, and there are now numerous research avenues focused on new approaches for driving and applying PCG systems. An area in which progress has been comparatively slow is the development of generalisable approaches for comparing alternative PCG systems, especially in terms of their generative spaces. It is to this area that this paper aims to make a contribution, by exploring the utility of data compression algorithms in compressing the generative spaces of PCG systems. We hope that this approach could be the basis for developing useful qualitative tools for comparing PCG systems to help designers better understand and optimize their generators. In this work we assess the efficacy of a selection of algorithms across sets of levels for 2D tile-based games by investigating how much their respective generative space compressions correlate with level behavioral characteristics. We conclude that the approach looks to be a promising one despite some inconsistency in efficacy in alternative domains, and that of the algorithms tested Multiple Correspondence Analysis appears to perform the most effectively.