Modding is a creative practice through which players negotiate the structure and values of games to better reflect the priorities of their communities. This paper analyzes the popularity and categories of active mods for Stardew Valley, Skyrim, and Cyberpunk 2077 on the website Nexus Mods. Through an initial thematic analysis, game-specific categories were mapped into broader forms of change, including character identities, environments, and gameplay expansions. We additionally use keyword and text-based analysis of mod summaries, descriptions, and tags to identify more specific patterns, particularly around value-oriented themes such as accessibility and identity. Across all three games, several broad priorities recur, including strong support for user interface changes, gameplay and story expansions, and the modding tools that make further modification possible. At the same time, each game exhibits a distinct pattern of creative priorities: Skyrim is dominated by atmosphere and VFX-based interface, animation, and gameplay expansion mods, Cyberpunk 2077 by character identity and appearance mods, and Stardew Valley by NPCs, environments, and quality-of-life upgrades. Additionally, targeted analysis of accessibility and identity-related mods suggest that these themes are strongly valued and supported by player communities. These early results suggest that players use mods to express their creativity and change who games are for and how they are experienced. Through these findings, this paper aims to position modding as a form of creative negotiation through which player communities transform games to better reflect their own values.

When designing procedural content generation systems for games, we imagine that there is a space of potential designs, and each of those designs affords a space of potential play.However, in most generative systems, there is a complex relationship between input parameters and the design or play properties of system outputs.As generators grow in complexity, it becomes harder to predict what experiences players will have in the generated design.In both constructive and solver-based approaches, this leads to uncertainty about how changes to the generator itself will affect the distribution of outputs.In this paper, we contribute a new method for constructing generators by manipulating closed-form expressions for spaces of designs and their associated play properties.The resulting generators have highly predictable running times, precisely controllable output distributions, and allow enforcing arbitrary constraints about their outputs.We offer a sequence of increasingly complex examples showing how to compute design-interaction expressions and use them for generation.Finally, we document scaling strategies for handling design-interaction spaces where millions of gameplay states are reachable in each of trillions of designs.

In procedural content generation, modeling the generation task as a constraint satisfaction problem lets us define local and global constraints on the generated output. However, a generator’s perceived quality often involves statistics rather than just hard constraints. For example, we may desire that generated outputs use design elements with a similar distribution to that of reference designs. However, such statistical properties cannot be expressed directly as a hard constraint on the generation of any one output. In contrast, methods which do not use a general-purpose constraint solver, such as Gumin’s implementation of the WaveFunctionCollapse (WFC) algorithm, can control output statistics but have limited constraint propagation ability and cannot express non-local constraints. In this paper, we introduce You-Only-Randomize-Once (YORO) pre-rolling, a method for crafting a decision variable ordering for a constraint solver that encodes desired statistics in a constraint-based generator. Using a solver-based WFC as an example, we show that this technique effectively controls the statistics of tile-grid outputs generated by several off-the-shelf SAT solvers, while still enforcing global constraints on the outputs.1 Our approach is immediately applicable to WFC-like generation problems and it offers a conceptual starting point for controlling the design element statistics in other constraint-based generators.

We introduce a new approach to deploying constraint-based content generators that better supports online generation. Constraint-based generators ensure that certain properties hold in each design they output. However, when deployed a general-purpose solver is often required, thus guarantees come with unpredictable search times and little control over sequentially-generated outputs. In this paper, we outline how we can encode design constraints into a compact circuit representation that affords generation without search. These generators yield samples that are distributed uniformly over the space of valid designs. We illustrate our approach with binary decision diagrams (BDDs) in comparison to the traditional approach with answer-set programming (ASP) in two scenarios: a grid-based tile placement scenario inspired by WaveFunctionCollapse, and a playable platformer level design scenario. These compiled design-space models make constraint-based methods easier to deploy by improving on both the running time and diversity of previous constraint-based methods.

In large game worlds, players can get lost and feel overwhelmed as they try to figure out which immediate choices will make progress towards their own long-range goals in the game. This is a planning problem, but these games often have large state spaces. In this paper, we show that Binary Decision Diagrams (BDDs) can directly manipulate very large game state spaces, and this power can be leveraged to construct instantly responsive player assistance systems. We use BDDs to build a compressed representation of a game’s state-transition function, and use it to derive an action policy that makes shortest-path recommendations for any feasible state towards any achievable goal in milliseconds. We introduce the intuition behind planning with BDDs using a tiny grid world, and eventually scale to an integrated system for start-to-finish gameplay assistance in Super Metroid.

Gumin’s WaveFunctionCollapse (WFC) algorithm attempts to generate output designs that resemble provided input designs. While the algorithm’s constraint-solving core is able to ensure that no local patterns are adjacent in the outputs that were not adjacent in the input, it does not accurately reproduce statistical properties of the input designs. Examining the algorithm’s behavior at the level of pattern adjacencies, we show that there are large gaps between the statistics of the input and output designs, even when applying Gumin’s search heuristic intended to influence output statistics. By offering a very small revision to this search heuristic, we show that the resemblance of outputs to inputs can be dramatically improved. Another way of improving resemblance is to increase the size of local patterns considered by WFC, but this can easily lead to a kind of overfitting that results in the outputs plagiarizing large portions of the input design. By contrast, our alternate revision increases resemblance without increasing pattern size. The simplicity of our method, requiring a very localized change to existing WFC implementations, allows it to be immediately applied to a wide range of applications.

When discussing a work of literature, text quotes can provide access to specific pieces of content and allow them to be placed in a larger context. However, we have no obvious analog for this notion of quotes in the medium of videogames. In this paper, we introduce the concept of playable quotes. To support the needs of developers, educators, streamers, and other stakeholders, playable quotes should be playable (directly interactive), partitioned (small slices of larger works), permanent (able to outlive the original work), and performative (able to demonstrate specific styles of play). Focusing on the Game Boy hand-held game console, we describe Tenmile, a deployed prototype for creating and sharing self-contained playable quotes of Game Boy games that satisfy our key requirements. Using Tenmile’s technical design as a template, we lay out a strategy for implementing similar notions of playable quotes across other computing platforms and raise issues that should be considered before continuing beyond Game Boy.

Machine playtesting systems often aim to demonstrate how to reach certain moments of play. To provide design feedback in a timely manner, they often internally rely on heuristic-guided search. However, there are many types of videogames for which sufficiently accurate heuristics are not available. We use an imperfect abstraction of an underlying game to define progress scores, and show that combining these scores yields a highly effective cell selection heuristic for use in the Go-Explore algorithm. We demonstrate the impact of this approach in automated gameplay for Super Metroid (involving mandatory item collection, destructible blocks, and backtracking) using a tile-based abstraction of the game that only models a small subset of the game’s mechanics. Surprisingly, our abstraction guidance mechanism is able to explore this complex game several orders of magnitude more efficiently than past work with similar exploration methods in Montezuma’s Revenge.

Academic Procedural Content Generation research has until recently overlooked a significant real-world application of generative methods to existing games: game randomizers. These programs remix existing games by changing things like item locations, enemy stats, or even room connections to create a fresh experience based on a beloved game, and are especially popular among speedrunning and streaming communities. They generate where high-production-quality full-scale games, explicitly geared towards replay value. Randomizers fulfill many of the stated motivations of the academic PCG research community, and important new research directions can be developed by investigating this space.

In this article, we describe WaveFunctionCollapse (WFC), a new family of algorithms for content generation. WFC was recently invented by independent game developer M. Gumin and has since been adopted and adapted by other game developers. Trends in academic research on content generation have only recently suggested the use of ideas from constraint solving and machine learning, so it is surprising to see these manifested in in-the-wild algorithms developed outside of an academic context. We illuminate the common components in this family of algorithms by way of a rational reconstruction. Through experiments with the reconstruction we probe the impact of design choices made in various adaptations of WFC (e.g., the role of backtracking, search heuristics, or pattern classification and rendering strategies). This article highlights a mode of incremental content generation that has been overlooked by past surveys of content generation methods.

Machine perception research primarily focuses on processing static inputs (e.g. images and texts). We are interested in machine perception of interactive media (such as games, apps, and complex web applications) where interactive audience choices have long-term implications for the audience experience. While there is ample research on AI methods for the task of playing games (often just one game at a time), this work is difficult to apply to new and in-development games or to use for non-playing tasks such as similarity-based retrieval or authoring assistance. In response, we contribute a corpus of 755 games and structured metadata, spread across several platforms (Twine, Bitsy, Construct, and Godot), with full source and assets available and appropriately licensed for use and redistribution in research. Because these games were sourced from student projects in an undergraduate game development program, they reference timely themes in their content and represent a variety of levels of design polish rather than only representing past commercial successes. This corpus could accelerate research in understanding interactive media while anchoring that work in freshly-developed games intended as legitimate human experiences (rather than lab-created AI testbeds). We validate the utility of this corpus by setting up the novel task of predicting tags relevant to the player experience from the game source code, showing that representations that better exploit the structure of the media outperform a text-only baseline.

Automated game exploration methods benefit from having human demonstration data, but this kind of data is not available for each incremental build of a videogame. If we want to make use of exploration inside of continuous integration (CI) workflows, we need to leverage stale human data (from a recent version of the game). In this paper, we show how to train a goal-conditioned action policy from stale human data used in the context of RRT-based exploration of a modestly changed game version. We demonstrate the benefit of this transfer with experiments in the MiniGrid environment (which has a three-dimensional agent configuration space).

Existing game description languages (GDLs) aspire to generality, but their focus on the specification of low-level mechanics leaves game generators that target these GDLs in the awkward position of having to invent combinations of mechanics that work well together from scratch. As a result, many existing game generators are good at producing games that contain novel and surprising combinations of mechanics, but bad at generating games that are readily interpretable by players as cultural artifacts. To address this problem, we introduce the concept of a genre-specific game description language (GSGDL): a game description language that deliberately encodes assumptions about a particular genre of games as a cultural form. As a proof of concept, we demonstrate the use of an internal representation of game structure used by the game creation tool GB Studio as a GSGDL for top-down 2D roleplaying games targeting the Game Boy. The use of this GSGDL gives us leverage to rapidly iterate on game generation features targeting a specific game genre and platform; to work with an existing toolchain that offers graphical editing, code generation, and automated play testing; and to more readily generate games that are interpretable by players as examples of a particular cultural form.

We introduce a hybrid neural + symbolic approach to map generation that combines neural discrete representation learning with symbolic constraint solving methods. In application to WarCraft II and Super Metroid map designs, we show how a vocabulary of directly manipulable latent tiles can be inferred from the raw pixels of design training data. Despite working with a very small tile vocabulary, our method is able to express a very large effective set of unique tiles at the level of pixel appearances. This work shows new ways of combining generative methods, resulting in directly controllable generators for domains that are primarily specified only by visual design examples.

Videogame level designs can contain errors called softlocks where a player traversing the level in an unintended manner can become permanently stuck. In this paper, we explore the automated detection of softlocks in the game Super Metroid using Computation Tree Logic (CTL). Super Metroid distinguishes itself as an example domain because of its velocity-based movement and rich item upgrade hierarchy. These factors can cause softlocks in Super Metroid to be challenging to detect visually. We contribute a tile-based gameplay abstraction for Super Metroid, and demonstrate verification of CTL properties for scenarios based on a segment of the original game’s level design. CTL can be used to define and test many other gameplay properties (e.g. which bosses can be skipped or which order items may be collected) and is immediately applicable to other game designs for which a compact abstraction of their state space can be enumerated. By making plausible design changes to a Super Metroid level fragment, we show how highly nonobvious softlocks can be detected and how the counterexamples resulting from verification failure can be turned into visualizations that explain the problem.

We need to teach AI to students in and outside of traditional computer science degree programs, including those designer-engineer hybrid students who will design and implement games or engage in technical games research later. The need to rethink AI curriculum is pressing in a design education context because AI powers many emerging practical techniques such as drama management, procedural content generation, player modeling, and machine playtesting. In this paper, we describe a 5-year experimental effort to teach a Game AI course structured around a broad and expanding set of roles AI can play in game design (e.g., Adversary and Actor, as well as Design Assistant and Storyteller). This course sets up computer science and computer game design students to transform practices in the game industry as well as create new forms of media that were previously unreachable. Our students gained mastery over the relevant techniques and further demonstrated (via novel prototype systems) many new roles for AI along the way.

Failure is integral to playing videogames, we do not have a precise terminology for discussing different categories of failure. This is a problem because certain failures are critical to the design intent of the game, and, as such, are desirable, whereas other failures detract from the play experience and are meant to be avoided. In this paper, we taxonomize several classes of failure in the player’s experience, offering a diagnostic tool that distinguishes in-loop failures from out-of-loop failures. By classifying both games and specific failure instances, the Taxonomy of Failure extends present vocabularies for: game designers making design choices, scholars critiquing the usability of games, educators teaching games, data analysts segmenting players, and game developers creating more adaptive games.

Attempting to maximize coverage of a game via human gameplay is laborious and repetitive, introducing delays in the development process. Despite the importance of quality assurance (QA) testing, QA remains an underinvested area in the technical games research community. In this paper, we show that relatively simple automatic exploration techniques can be used to multiplicatively amplify coverage of a game starting from human tester data. Instead of attempting to displace human QA efforts, we seek to grow the impact that a human tester can make. Experiments with two games for the Super Nintendo Entertainment System highlight the qualitative and quantitative differences between isolated human and machine play compared to our hybrid approach called Reveal-More. We contribute a QA testing workflow that scales with the amount of human and machine time allocated to the effort.

We describe a machine playtesting system that combines two paradigms of artificial intelligence - learning and tree search - and intends to place them in the hands of independent game developers. This integration approach has shown great success in Go-playing systems like AlphaGo and AlphaZero, but until now has not been available to those outside of artificial intelligence labs. Our system expands the Monster Carlo machine playtesting framework for Unity games by integrating its tree search capabilities with the behavior cloning features of Unity's Machine Learning Agents Toolkit. Because experience gained in one playthrough may now usefully transfer to other playthroughs via imitation learning, the new system overcomes a serious limitation of the older one with respect to stochastic games (when memorizing a single optimal solution is ineffective). Additionally, learning allows search-based automated play to be bootstrapped from examples of human play styles or even from the best of its own past experiences. In this paper we demonstrate that our framework discovers higher-scoring and more-representative play with minimal need for machine learning or search expertise.

Videogames are a rich domain for scholarship, and their dynamic content makes them a new and unique challenge for information retrieval (IR). Recent work has made it possible to cite specific moments in a videogame (like pages in a book), but currently finding those moments to cite is laborious because there are no videogame moment search engines. We conducted an in-depth interview study with ten users across a variety of user profiles: developers, educators, speedrunners, scholars, and streamers. From these interviews, we identify the unique needs of each user profile for retrieving moments from videogames. We outline implications for future research in retrieval and design implications for new tools focused on interactive media.

Procedural content generation via machine learning (PCGML) is typically framed as the task of fitting a generative model to full-scale examples of a desired content distribution. This approach presents a fundamental tension: the more design effort expended to produce detailed training examples for shaping a generator, the lower the return on investment from applying PCGML in the first place. In response, we propose the use of discriminative models, which capture the validity of a design rather the distribution of the content, trained on positive and negative example design fragments. Through a modest modification of WaveFunctionCollapse, a commercially-adopted PCG approach that we characterize as using elementary machine learning, we demonstrate a new mode of control for learning-based generators. We demonstrate how an artist might craft a focused set of additional positive and negative design fragments by critique of the generator's previous outputs. This interaction mode bridges PCGML with mixed-initiative design assistance tools by working with a machine to define a space of valid designs rather than just one new design.

Search engines for books can usually tell us which specific pages in a book mention the concepts we seek. A similar ability to search within the contents of games, locating specific moments in their spaces of interactivity, is not yet available. This limits players' ability to find deeply relevant games and game scholars' ability to find moments that advance their arguments. Drawing on computer vision and natural language processing, our work introduces the ability to search within a space of game moments using natural language queries. We describe and evaluate a prototype system which is capable of retrieving moments from two contemporary, narrative-driven games by semantic matching on both the auditory and visual content of scenes.

There exists a gap between visualization design guidelines and their application in visualization tools. While empirical studies can provide design guidance, we lack a formal framework for representing design knowledge, integrating results across studies, and applying this knowledge in automated design tools that promote effective encodings and facilitate visual exploration. We propose modeling visualization design knowledge as a collection of constraints, in conjunction with a method to learn weights for soft constraints from experimental data. Using constraints, we can take theoretical design knowledge and express it in a concrete, extensible, and testable form: the resulting models can recommend visualization designs and can easily be augmented with additional constraints or updated weights. We implement our approach in Draco, a constraint-based system based on Answer Set Programming (ASP). We demonstrate how to construct increasingly sophisticated automated visualization design systems, including systems based on weights learned directly from the results of graphical perception experiments.

We introduce the problem of content-based retrieval for moments in videogames. This new area for artificial intelligence in games exercises automated gameplay and visual understanding while making connections to information retrieval. We propose a number of techniques to discover the interesting moments in a game (crawling), show how to compress moments into an efficiently searchable structure (indexing), and recall those moments most relevant to a user-provided query (retrieving). We combine these ideas in a prototype visual search engine and compare it with commercial visual search engines. Searching within a corpus of moments from Super Nintendo Entertainment System games using query images extracted from YouTube videos, our prototype is able to identify moments that web-oriented search engines rarely see.

Generative Adversarial Networks (GANs) are a machine learning approach capable of generating novel example outputs across a space of provided training examples. Procedural Content Generation (PCG) of levels for video games could benefit from such models, especially for games where there is a pre-existing corpus of levels to emulate. This paper trains a GAN to generate levels for Super Mario Bros using a level from the Video Game Level Corpus. The approach successfully generates a variety of levels similar to one in the original corpus, but is further improved by application of the Covariance Matrix Adaptation Evolution Strategy (CMA-ES). Specifically, various fitness functions are used to discover levels within the latent space of the GAN that maximize desired properties. Simple static properties are optimized, such as a given distribution of tile types. Additionally, the champion A* agent from the 2009 Mario AI competition is used to assess whether a level is playable, and how many jumping actions are required to beat it. These fitness functions allow for the discovery of levels that exist within the space of examples designed by experts, and also guide the search towards levels that fulfill one or more specified objectives.

Maxim Gumin's WaveFunctionCollapse (WFC) algorithm is an example-driven image generation algorithm emerging from the craft practice of procedural content generation. In WFC, new images are generated in the style of given examples by ensuring every local window of the output occurs somewhere in the input. Operationally, WFC implements a non-backtracking, greedy search method. This paper examines WFC as an instance of constraint solving methods. We trace WFC's explosive influence on the technical artist community, explain its operation in terms of ideas from the constraint solving literature, and probe its strengths by means of a surrogate implementation using answer set programming.

A long-term goal of game design research is to achieve end-to-end automation of much of the design process, one aspect of which is creating effective level progressions. A key difficulty is getting the player to practice with interesting combinations of learned skills while maintaining their engagement. Although recent work in task generation and sequencing has reduced this effort, we still lack end-to-end automation of the entire content design process. We approach this goal by incorporating ideas from intelligent tutoring systems and proposing progression strategies that seek to achieve mastery of not only base concepts but arbitrary combinations of these concepts. The input to our system is a model of what the player needs to do to complete each level, expressed as either an imperative procedure for producing solutions or a representation of features common to all solutions. The output is a progression of levels that can be adjusted by changing high-level parameters. We apply our framework to a popular math puzzle game and present results from 2,377 players showing that our automatic level progression is comparable to expert-crafted progression after a few design iterations based on a key engagement metric.

Creating game content requires balancing design considerations at multiple scales: each level requires effort and iteration to produce, and broad-scale constraints such as the order in which game concepts are introduced must be respected. Game designers currently create informal plans for how the game's levels will fit together, but they rarely keep these plans up-to-date when levels change during iteration and testing. This leads to violations of constraints and makes changing the high-level plans expensive. To address these problems, we explore the creation of mixed-initiative game progression authoring tools which explicitly model broad-scale design considerations. These tools let the designer specify constraints on progressions, and keep the plan synchronized when levels are edited. This enables the designer to move between broad and narrow-scale editing and allows for automatic detection of problems caused by edits to levels. We further leverage advances in procedural content generation to help the designer rapidly explore and test game progressions. We present a prototype implementation of such a tool for our actively-developed educational game, Refraction. We also describe how this system could be extended for use in other games and domains, specifically for the domains of math problem sets and interactive programming tutorials.

Some problems in procedural content generation for games involve hard constraints (e.g. that a generated puzzle is necessarily solvable). Common techniques for generator design lack a way to specify crisp (yes/no) constraints on what counts as a valid content artifact and guarantee these constraints are satisfied in the generator's output. In this paper we present two independent implementations of three diverse level design automation tools for the popular online educational game Refraction. All of the systems guarantee key properties of their output. Applying a constraint-focused generator design perspective in depth, we found that even emergent aesthetic style properties were straightforward to directly control. Our results with Refraction provide further concrete evidence for the claim that the expressive power of constraints and the ease with which they can be incorporated into suitably designed generative processes makes them a powerful tool for producing reliably-controllable generators for game content.

"Player modeling" is a loose concept. It can equally apply to everything from a predictive model of player actions resulting from machine learning to a designer's description of a player's expected reactions in response to some piece of game content. This lack of a precise terminology prevents practitioners from quickly finding introductions to applicable modeling methods or determining viable alternatives to their own techniques. We introduce a vocabulary that distinguishes between the major existing player modeling applications and techniques. Four facets together define the kind for a model: the scope of application, the purpose of use, the domain of modeled details, and the source of a model's derivation or motivation. This vocabulary allows the identification of relevant player modeling methods for particular problems and clarifies the roles that a player model can take.

Procedural content generators for games produce artifacts from a latent design space. This space is often only implicitly defined, an emergent result of the procedures used in the generator. In this paper, we outline an approach to content generation that centers on explicit description of the design space, using domain-independent procedures to produce artifacts from the described space. By concisely capturing a design space as an answer set program, we can rapidly define and expressively sculpt new generators for a variety of game content domains. We walk through the reimplementation of a reference evolutionary content generator in a tutorial example, and review existing applications of answer set programming to generative-content design problems in and outside of a game context.

We are interested in reconstructing real world locations as detailed 3D models, but to achieve this goal, we require a large quantity of photographic data. We designed a game to employ the efforts and digital cameras of everyday people to not only collect this data, but to do so in a fun and effective way. The result is PhotoCity, a game played outdoors with a camera, in which players take photos to capture flags and take over virtual models of real buildings. The game falls into the genres of both games with a purpose (GWAPs) and alternate reality games (ARGs). Each type of game comes with its own inherent challenges, but as a hybrid of both, PhotoCity presented us with a unique combination of obstacles. This paper describes the design decisions made to address these obstacles, and seeks to answer the question: Can games be used to achieve massive data-acquisition tasks when played in the real world, away from standard game consoles? We conclude with a report on player experiences and showcase some 3D reconstructions built by players during gameplay.

We present a longitudinal investigation of Tableau Machine, an intelligent entity that interprets and reflects the lives of occupants in the home. We created Tableau Machine (TM) to explore the parts of home life that are unrelated to accomplishing tasks. Task support for "smart homes" has inspired many researchers in the community. We consider design for experience, an orthogonal dimension to task-centric home life. TM produces abstract visualizations on a large LCD every few minutes, driven by a set of four overhead cameras that capture a sense of the social life of a domestic space. The openness and ambiguity of TM allow for a cycle of co-interpretation with householders. We report on three longitudinal deployments of TM for a period of six weeks. Participant families engaged with TM at the outset to understand how their behaviors were influencing the machine, and, while TM remained puzzling, householders interacted richly with TM and its images. We extract some key design implications for an experience-focused smart home.