Game content has long been created using procedural generation. However, many of these systems are currently designed in an ad-hoc manner, and there is a lack of knowledge around the design criteria that lead to generators producing the most successful results. In this study, we conduct a qualitative examination of the comments left by judges for the 2018–2020Generative Design in Minecraftcompetition. Using abductive thematic analysis, we identify the core design criteria that contribute to a generator that creates “good” content – here defined as interesting or engaging. By performing this study, we have identified that the core design criteria that create and interesting settlement are usability of the settlement environment, the thematic coherence within the settlement, and an anchoring in real-world simulacra.

The GDMC AI settlement generation challenge is a procedural content generation (PCG) competition about producing an algorithm that can create a settlement in the game Minecraft. In contrast to the majority of AI competitions, the GDMC entries are evaluated by human experts on several criteria such as adaptability, functionality, evocative narrative, and visual aesthetics – all of which represent challenges to state-of-the-art PCG systems. This paper contains a collection of written experiences with this competition, by participants, judges, organizers and advisors. We asked people to reflect both on the artifacts themselves, and on the competition in general. The aim of this paper is to offer a shareable and edited collection of experiences and qualitative feedback which have the potential to push forward PCG and computational creativity, but would be lost once the individual assessments are compressed to scalar ratings. We reflect upon organizational issues for AI competitions, and discuss the future of the GDMC competition.

Procedural content generation for games is a growing trend in both research and industry, even though there is no consensus on how good content looks, nor how to automatically evaluate it. A number of metrics have been developed in the past, usually focused on the artifact as a whole, and mostly lacking grounding in human experience. In this study, we develop a new set of automated metrics, motivated by ideas from architecture, namely isovists, which have a track record of capturing the human experience of space. These metrics can be computed for a specific game state, from the player’s perspective, and take into account their embodiment in the game world. We show how to apply those metrics to the 3d blockworld of Minecraft. We use a dataset of generated settlements from the GDMC Settlement Generation Challenge in Minecraft and establish several rank-based correlations between the isovist properties and the rating human judges gave those settlements. We also produce a range of heat maps that demonstrate the location-based applicability of the approach, which allows for the development of those metrics as measures for a game experience at a specific time and space.

There are a range of metrics that can be applied to the artifacts produced by procedural content generation, and several of them come with qualitative claims. In this paper, we adapt a range of existing PCG metrics to generated Minecraft settlements, develop a few new metrics inspired by PCG literature, and compare the resulting measurements to existing human evaluations. The aim is to analyze how those metrics capture human evaluation scores in different categories, how the metrics generalize to another game domain, and how metrics deal with more complex artifacts. We provide an exploratory look at a variety of metrics and provide an information gain and several correlation analyses. We found some relationships between human scores and metrics counting specific elements, measuring the diversity of blocks and measuring the presence of crafting materials for the present complex blocks.