With the fast progress of generative AI in recent years, more games are integrating generated content, raising questions regarding how players perceive and respond to this content. To investigate, we ran a mixed-method survey on the games Super Mario Bros. and Sokoban, comparing procedurally generated levels and levels designed by humans to explore how perceptions of the creator relate to players’ overall experience of gameplay. Players could not reliably identify the level’s creator, yet their experiences were strongly linked to their beliefs about that creator rather than the actual truth. Levels believed to be human-made were rated as more fun and aesthetically pleasing. In contrast, those believed to be AI-generated were rated as more frustrating and challenging. This negative bias appeared spontaneously without knowing the levels’ creator and often was based on unreliable cues of “human-likeness.” Our results underscore the importance of understanding perception biases when integrating generative systems into games.

Traditionally, crosswords are made as complete puzzles and then shared with players. In the digital age, crosswords can be created dynamically while the player plays, and can even react to how the player is performing. In this paper, we present a system for reactive crossword construction as a form of dynamic difficulty adjustment. Difficulty can be adapted within a single puzzle, by constructing a puzzle while the player plays, and the word/clue pairs can be chosen to be more or less difficult. We also show that hints, in the form of additional words, can be embedded into the puzzle for struggling players. We tested this system with three player personas across five levels of puzzle difficulty, using time as a proxy for difficulty. Our results show that reactive puzzle construction reduced the time needed to solve puzzles compared to static construction. The addition of hints further reduced the time to solve. These results are limited to simulated player personas, but establish a foundation and identify directions for future evaluation with real players. We expect that reactive puzzle construction could be applied to otherwise static puzzle types, by revealing clues as players progress.

In this work we explore an approach for creating personalized microgames, drawing inspiration from fill-in-the-blank stories such as Mad Libs. In fill-in-the-blank stories, the reader is given a list of word categories such as place, number, or adjective, and provides keywords to fill in the list. These keywords are then copied into the blanks of a short story, with often humorous results. We develop a similar process for generating microgames, where the keywords are used to fill in the blanks of a Large Language Model game generation prompt. Rather than extending the capabilities of LLMs for game generation, part of our aim is using LLMs as-is for generating unpolished, small-scale, personalized microgames that can be played for a brief duration. We compare two template prompting strategies, apply these across four genres, and evaluate the generated games based on incorporation of keywords, variety, and qualitative observations.

Procedural content generation via machine learning (PCGML) in games involves using machine learning techniques to create game content such as maps and levels.2D tile-based game levels have consistently served as a standard dataset for PCGML because they are a simplified version of game levels while maintaining the specific constraints typical of games, such as being solvable.In this work, we highlight the unique characteristics of game levels, including their structured discrete data nature, the local and global constraints inherent in the games, and the sensitivity of the game levels to small changes in input.We define the robustness of data as a measure of sensitivity to small changes in input that cause a change in output, and we use this measure to analyze and compare these levels to state-of-the-art machine learning datasets, showcasing the subtle differences in their nature.We also constructed a large dataset from four games inspired by popular classic tile-based games that showcase these characteristics and address the challenge of sparse data in PCGML by providing a significantly larger dataset than those currently available.

Expressive range analysis is a visualization-based technique used to evaluate the performance of generative models, particularly in game level generation.It typically employs two quantifiable metrics to position generated artifacts on a 2D plot, offering insight into how content is distributed within a defined metric space.In this work, we use the expressive range of a generator as the conceptual space of possible creations.Inspired by the quality diversity paradigm, we explore this space to generate levels.To do so, we use a constraintbased generator that systematically traverses and generates levels in this space.To train the constraint-based generator we use different tile patterns to learn from the initial example levels.We analyze how different patterns influence the exploration of the expressive range.Specifically, we compare the exploration process based on time, the number of successful and failed sample generations, and the overall interestingness of the generated levels.Unlike typical quality diversity approaches that rely on random generation and hope to get good coverage of the expressive range, this approach systematically traverses the grid ensuring more coverage.This helps create unique and interesting game levels while also improving our understanding of the generator's strengths and limitations.

This work investigates the role of uncertainty in Slay the Spire using an information-theoretic framework.Focusing on the entropy of game paths (which are based on procedurally-generated maps) we analyze how randomness influences player decision-making and success.By examining a dataset of 20,000 game runs, we quantify the entropy of paths taken by players and relate it with their outcomes and skill levels.The results show that victorious runs are associated with higher normalized entropy, suggesting more risktaking.Additionally, higher-skill players tend to exhibit distinct patterns of risk-taking behavior in later game stages.

When generating game levels, it is desirable for them to be completable.Depending on the designer's goals, it may also be desirable to generate levels without softlocks.A softlock is a situation where the player has not won or lost, but cannot make progress toward the goal, and is thus stuck (unless they reset, or possibly lose the level).In this work we present a constraint-based approach to generating 2D tile-based levels that are completable and do not have areas where the player can get stuck.The approach uses a constraint-based reachability categorization of locations during level generation.This categorization can be used to prevent softlocks by ensuring that all locations reachable going forward from the start of the level are also reachable going backward from the goal, unless they are sinks (areas where the player will inevitably lose the level, e.g.falling off the bottom).Using this approach, it is also possible to intentionally generate levels with softlocks and to repair levels to remove softlocks.

Table-top role-playing games (TTRPGs) are a form of gameplay that often requires a variety of complex tasks to be completed both in preparation and throughout gameplay: from tracking game state to the creation of fictional worlds.This has presented an opportunity for computational assistance in TTRPG sessions, both in the creation of artifacts and throughout the gameplay.We investigate the current research in computational tools for TTRPGs through a scoping review of academic works and present the major trends and opportunities from these works.We screened over one thousand works sourced from three different academic databases: ACM Digital Library, IEEE-Xplore, and Google Scholar.Papers were included based on relevance to TTRPGs, computational interface, and academic venue.In total, we evaluated 46 works in terms of produced artifacts, computational methods, evaluation, and outcomes.These papers include a diverse set of produced artifacts and computational methods, with an emphasis on tangible interfaces and generative AI systems.However, we found an opportunity for future work in terms of long-term studies, mixed-initiative methods, and different aspects of gameplay.

Recent research has begun exploring games as a medium for reflection due to their affordances as interactive systems of challenge. However, little effort has been put into (1) synthesizing insights across studies and disciplines and (2) translating the academic work on reflective play into practical takeaways for game developers. This article takes the first steps toward summarizing existing work on reflective play and translating insights for practical implementation by identifying key game elements present in games that evoke reflection. We divide these elements into five approaches: Disruptions, Slowdowns, Questioning, Revisiting, and Enhancers. Finally, we provide an actionable supplement for practicing game developers to apply these concepts to their games.

When generating levels, algorithmically evaluating the results is essential. In this paper, we looked at predicting a level’s difficulty and enjoyment. Past work has approached this problem for puzzle games like Sudoku by analyzing the characteristics of the initial level, the solved level, and the process that led to that solution. In this work, we examined a set of heuristics for Roguelike levels and their solutions, and their relationship to subjective player ratings of the levels. We gathered ratings of difficulty and enjoyment of levels in a study with 143 players. We ran an ablation study on the set of heuristics to find the best combination of heuristics for predicting difficulty and enjoyment with a linear regression model, and found solution path-based heursitics performed well. However, these models did not outperform a simple baseline for predicting enjoyment. Jaccard similarity on paths—a method we have not seen used in the field of game AI—was a useful predictor of difficulty. Testing proximity to enemies across a solution path is the only heuristic needed to predict how enjoyable a level will be.

Most research in procedural content generation has, understandably, focused on generating levels that are completable—that is, levels where it is possible for a player to complete them. In this work we explore the generation of uncompletable levels and their applications. Building on an existing constraint-based level generator, we add support for constraints that a level’s goal is not reachable from its start. The generator can thus create levels that are similar to completeable levels in many ways (such as local tile patterns), yet are not possible to complete. We then describe several applications of those constraints and the resulting levels, including: qualitatively characterizing what makes levels uncompletable; creating training data for completability classifiers; checking that a generator can only generate completable levels; and generating levels that require the player to use a special move.

Human computation games can crowdsource human intuition and reasoning for complex problems. Collaboration on these problems may help players build on each others’ work, but may also reduce the variety of solutions due to convergence. Previous work has found that applying quality-diversity approaches to crowdsourced human computation can encourage individuals to discover a wider variety of solutions. Thus, we investigate if quality-diversity approaches can similarly improve solution variety while also benefiting from collaboration. We ran a crowdsourced study in which two factors were varied: whether participants could asynchronously collaborate via access to solutions previously provided by other participants, and whether participants had access to quality-diversity search tools. We evaluated on two human computation puzzle tasks, and found the impact varied based on task; the quality-diversity tools appeared to help more in the travelling salesperson task, while collaboration appeared to help more in the knapsack task.

Game authoring can be a difficult, technical process; exploring new ways to describe games and game mechanics may help make game authoring more accessible. In this work, we present Tile Rewrite Rule Behavior Trees (TRRBTs): a concept for a domain-specific language for authoring tile-based, turn-based games. The approach combines tile rewrite rules and behavior trees. Using TRRBTs, a game’s state is represented as a grid of tiles, the behavior trees describe the overall flow of the game, and the rewrite rules at leaf nodes describe changes in game state. We include transform nodes, which apply transformations to other nodes in the behavior tree, allowing more complex mechanics to be expressed in a compact way. We demonstrate a text-based approach to using TRRBTs to create several simple games, show how the approach allows re-use of trees to build on existing games, and show how they can provide a unified representation for procedural content generation and enemy AI along with game mechanics.

We presentlatent combinational game design—an approach for generating playable games that blend a given set of games in a desired combination using deep generative latent variable models. We use Gaussian mixture variational autoencoders (GMVAEs), which model the VAE latent space via a mixture of Gaussian components. Through supervised training, each component encodes levels from one game and lets us define blended games as linear combinations of these components. This enables generating new games that blend the input games as well as controlling the relative proportions of each game in the blend. We also extend prior blending work using conditional VAEs and compare against the GMVAE and additionally introduce a hybrid conditional GMVAE architecture, which lets us generate whole blended levels and layouts. Results show that these approaches can generate playable games that blend the input games in specified combinations. We use both platformers and dungeon-based games to demonstrate our results.

Game level blending via machine learning, the process of combining features of game levels to create unique and novel game levels using Procedural Content Generation via Machine Learning (PCGML) techniques, has gained increasing popularity in recent years. However, many existing techniques rely on human-annotated level representations, which limits game level blending to a limited number of annotated games. Even with annotated games, researchers often need to author an additional shared representation to make blending possible. In this paper, we present a novel approach to game level blending that employs Clustering-based Tile Embeddings (CTE), a learned level representation technique that can serve as a level representation for unannotated games and a unified level representation across games without the need for human annotation. CTE represents game level tiles as a continuous vector representation, unifying their visual, contextual, and behavioral information. We apply this approach to two classic Nintendo games, Lode Runner and The Legend of Zelda. We run an evaluation comparing the CTE representation to a common, human-annotated representation in the blending task and find that CTE has comparable or better performance without the need for human annotation.

Determining the completability of levels generated by procedural generators such as machine learning models can be challenging, as it can involve the use of solver agents that often require a significant amount of time to analyze and solve levels. Active learning is not yet widely adopted in game evaluations, although it has been used successfully in natural language processing, image and speech recognition, and computer vision, where the availability of labeled data is limited or expensive. In this paper, we propose the use of active learning for learning level completability classification. Through an active learning approach, we train deep-learning models to classify the completability of generated levels for Super Mario Bros., Kid Icarus, and a Zelda-like game. We compare active learning for querying levels to label with completability against random queries. Our results show using an active learning approach to label levels results in better classifier performance with the same amount of labeled data.

Player paths are an important consideration when procedurally generating levels. However, player paths are typically incorporated as part of an evaluation process or overlaid on level structure in training data. Here we present a constraint-based level generation approach that takes a path through the level as input and attempts to generate a level containing that path. We describe an interactive level generation tool where the generator creates levels based on paths the user draws. We characterize the types of paths and levels generated by the approach.

Generating game levels via constraint satisfaction can become slow as levels grow large. In this work, we explore constraint-based generation of large levels by iteratively extending smaller intermediate levels. In each iteration, a new segment is created by generating a new, larger, intermediate level. The tiles and solution path at the beginning of the new intermediate level are constrained to match the previous intermediate level. We evaluated our approach using three games and two low-level solvers. We found that generating a level using more segments improves performance up to a point, and then decreases performance. We also saw a slight but noticeable decrease in the tile-based differences between levels generated as the number of segments increases, and saw that later segments usually took more time to generate than earlier segments.

Procedural level generation techniques that learn local neighborhoods from example levels (such as WaveFunctionCollapse) have risen in popularity. Usually the neighborhood structure (such as a regular grid) onto which a level is generated is fixed in advance and not generated. In this work, we present a constraint-based approach for graph generation that learns local neighborhood patterns (in the form of labeled nodes and edges) from example graphs. This allows the approach to generate graphs with varying structures that are still locally similar to the examples. We demonstrate the approach on several applications, such as abstract graphs describing Legend of Zelda dungeons. Additionally, using Super Mario Bros. levels, we show how techniques that run on a grid may be considered a special case of graph generation where each tile is a node connecting to its neighboring tiles’ nodes.

Completability is a key aspect of procedural level generation. In this work, we present a constraint-based approach to level generation for 2D tile-based games that simultaneously generates a level and an example playthrough of the level demonstrating its completability. The approach represents game mechanics as tile rewrite rules, which allows a variety of games and mechanics (beyond simple pathfinding) to be incorporated. The mechanics are represented as constraints in the same problem along with the constraints used to generate the level itself. Thus, the solution to the constraint problem contains both a level and a playthrough of the level. We demonstrate the flexibilty of the system and of tile rewrite rules in several applications, including lock-and-key dungeons, platformers, puzzles, and match-three style games.

We present re-trainable procedural level generation via machine learning (RT-PLGML), a game mechanic of providing in-game training examples for a PLGML system. We discuss opportunities and challenges, along with concept RT-PLGML games.

Citizen science games enable public participation in scientific research, yet these games often struggle to engage wide audiences. As a potential solution, some game developers look to narrative as an experience-enhancing feature. Yet the impacts and affordances of narrative in citizen science games remain understudied, especially for games that require significant onboarding. Therefore, we investigated the effects of wrapping a story around the tutorial puzzles of the citizen science game Foldit. We found that the narrative increased the time players spent engaging with the game’s tutorial and its scientific puzzles but did not substantially affect their progress through the tutorial. This article provides two major contributions: (1) empirical evidence detailing the impact of narrative on gameplay metrics in a citizen science game, including the relevant effects of genre preferences on engagement; and (2) recommendations on the use of narrative and its capacities in citizen science games. We conclude that the inclusion of a narrative can add valuable depth to the experience when designed thoughtfully and intentionally.

Expertise-centric citizen science games (ECCSGs) can be powerful tools for crowdsourcing scientific knowledge production. However, to be effective these games must train their players on how to become experts, which is difficult in practice. In this study, we investigated the path to expertise and the barriers involved by interviewing players of three ECCSGs: Foldit, Eterna, and Eyewire. We then applied reflexive thematic analysis to generate themes of their experiences and produce a model of expertise and its barriers. We found expertise is constructed through a cycle of exploratory and social learning but prevented by instructional design issues. Moreover, exploration is slowed by a lack of polish to the game artifact, and social learning is disrupted by a lack of clear communication. Based on our analysis we make several recommendations for CSG developers, including: collaborating with professionals of required skill sets; providing social features and feedback systems; and improving scientific communication.

An increasingly common area of study in procedural content generation is the creation of level segments: short pieces that can be used to form larger levels. Previous work has used concatenation to form these larger levels. However, even if the segments themselves are completable and well-formed, concatenation can fail to produce levels that are completable and can cause broken in-game structures (e.g. malformed pipes in Mario). We show this with three tile-based games: a side-scrolling platformer, a vertical platformer, and a top-down roguelike. To address this, we present a Markov chain and a tree search algorithm that finds a link between two level segments, which uses filters to ensure completability and unbroken in-game structures in the linked segments. We further show that these links work well for multi-segment levels. We find that this method reliably finds links between segments and is customizable to meet a designer’s needs.

Prior work using skill chains for matchmaking-based dynamic difficulty adjustment in human computation games required skill chains to be manually defined for a game, and each level to be manually annotated with the individual skills needed to complete that level. In this work, we present two approaches for defining level orderings for DDA in the platformer HCG Iowa James without using such manually-defined skill chains and annotations. The first involves sequences of action-context pairs found in gameplay traces. The second consists of applying K-means clustering on segments of levels. Our results show that both new approaches outperform baseline random level ordering and perform similarly to the skill chain approach.

Games can be a powerful medium for encouraging reflection. Game designers can elect to include reflective elements within games otherwise designed for the mass-market. In order to better understand how mass-market games encourage players to reflect and which game elements were successful or unsuccessful in aiding reflection, we conducted an online survey. This survey was exploratory in nature and asked participants about their feelings towards games that had attempted to include deeper messages as part of gameplay. Inductive thematic analysis on the 139 responses led to identification of elements players valued within games and the creation of four themes underlying the player experience. Beyond the themes, a key takeaway was that games encourage reflection when they provide a space for players to project and explore their own experiences and philosophies, or when games enable implicit conversation between the player and the designer on the message of the game.

Learners’ backgrounds, skills, and knowledge vary as they attempt to learn a new subject. To address this variation and allow learners to progress at their own speed, many researchers are suggesting adaptive learning as a solution. Adaptive content has been successful in learning environments such as intelligent tutoring systems, but it has not been thoroughly researched within video games, especially in terms of adaptive procedural levels. In this paper, we analyze the effects of procedural levels that are generated and inserted at run-time in between pre-designed levels in the educational programming game May’s Journey. Our study with 94 Amazon Mechanical Turkers shows that players encountered fewer code-related errors in the adaptive version, however, their engagement levels were similar, if not slightly higher in the non-adaptive version.

Citizen science projects that rely on human computation can attempt to solicit volunteers or use paid microwork platforms such as Amazon Mechanical Turk. To better understand these approaches, this paper analyzes crowdsourced image label data sourced from an environmental justice project looking at wetland loss off the coast of Louisiana. This retrospective analysis identifies key differences between the two populations: while Mechanical Turk workers are accessible, cost-efficient, and rate more images than volunteers (on average), their labels are of lower quality, whereas volunteers can achieve high accuracy with comparably few votes. Volunteer organizations can also interface with the educational or outreach goals of an organization in ways that the limited context of microwork prevents.

Variational autoencoders (VAEs) have been used in prior works for generating and blending levels from different games. To add controllability to these models, conditional VAEs (CVAEs) were recently shown capable of generating output that can be modified using labels specifying desired content, albeit working with segments of levels and platformers exclusively. We expand these works by using CVAEs for generating whole platformer and dungeon levels, and blending levels across these genres. We show that CVAEs can reliably control door placement in dungeons and progression direction in platformer levels. Thus, by using appropriate labels, our approach can generate whole dungeons and platformer levels of interconnected rooms and segments respectively as well as levels that blend dungeons and platformers. We demonstrate our approach using The Legend of Zelda, Metroid, Mega Man and Lode Runner.

We present a fully online system for skill and ratings-based dynamic difficulty adjustment (DDA) in human computation games (RCGs). Prior work used an initial offline phase for collecting level ratings to avoid cold-start when using the system online. Further, such systems set a fixed target score win/lose threshold for each level. In this work, we address these issues by 1) using an ∊-greedy variant of the ratings and skill-based DDA algorithm and 2) applying rating arrays, proposed in past work but not yet used in an online setting. We demonstrate our system in two online matchmaking experiments using two HCGs. Results suggest that the ∊-greedy modification helps address the cold-start and that using rating arrays leads to players completing more levels.

Participant disengagement in citizen science tasks remains a significant challenge for crowdsourcing platform creators, in their efforts to generate meaningful data and connect with their membership. Reinforcement learning is increasingly used to take advantage of the plethora of available data to learn to sequence tasks for participants. To this end, we extend the reinforcement learning techniques used in Tile-o-Scope Grid, an image matching web game, by introducing an adaptive Q-learning based approach that incorporates participant performance in sequencing the difficulty of levels. We compared our adaptive version against both a previous non-adaptive algorithm, as well as a greedy approach. We found that the adaptive extension outperformed both, in terms of total reward. This work contributes to the growing literature on reinforcement learning approaches applied to citizen science.

In the context of procedural content generation via machine learning (PCGML), quality-diversity (QD) algorithms are a powerful tool to generate diverse game content. A branch of QD uses genetic operators to generate content (e.g. MAP-Elites). Problematically, levels generated with these operators have no guarantee of matching the style of a game. This can be addressed by incorporating whether a level is generable by an n-gram into the fitness function. Unfortunately, this leads to wasted computation and poor results. In this work, we introduce n-gram genetic operators, which produce only solutions that are generable by the n-gram model; we call MAP-Elites combined with these operators Gram-Elites. We test on a tile-based side-scrolling platformer, vertical platformer, and roguelike. For all three, n-gram operators outperform standard operators and random n-gram generation, finding more usable (i.e. completable and generable) solutions at a faster rate. By integrating structure into operators, instead of fitness, these genetic operators could be beneficial to QD in PCGML.

Science is increasingly being achieved through citizen science (CitSci) — research conducted in part by non-professional scientists. In order to increase recruitment and retention, CitSci is sometimes gamified. Yet, research suggests that being a game does not alone make CitSci more enticing; rather, retention is dependent on the player’s experience. One recent trend in gaming is Augmented Reality (AR), as demonstrated by games like Pokémon Go. In this work, we examine the potential benefits of applying AR to CitSci gaming. We conducted a user study where we invited participants to play two citizen science adaptations (browser and AR) of the traditional game of Memory. We identified four major themes through thematic analysis. We conclude that AR is a leisurely toy: while the browser interface is better suited to rapid, accessible, productive play, AR is better suited to reflective, one-time, unstructured experiences. Our findings reveal potential towards future work on exploring AR in contexts such as museums and classrooms.

Player engagement and task effectiveness are crucial factors in human computation games. However, collecting data and making design changes towards these goals can be time-consuming. In this work, we incorporate rapid crowdsourced playtesting via the ARAPID (As Rapid As Possible Iterative Design) system to iterate on the design of a human computation platformer game. For each level in the game, the player’s goal is to collect items relevant to a given scenario while avoiding irrelevant items. We extended the visualization modules in the existing ARAPID system to include a multi-level data visualization and item collection task effectiveness plot. A designer from the project team used the system to iterate on the game’s level design, with the goal of increasing relevant and decreasing irrelevant items collected by players. A large-scale test with the game versions created during the iterative analysis found that the designer was able to use ARAPID to improve the specified goal parameters.

Several works have demonstrated the use of variational autoencoders (VAEs) for generating levels in the style of existing games and blending levels across different games. Further, quality-diversity (QD) algorithms have also become popular for generating varied game content by using evolution to explore a search space while focusing on both variety and quality. To reap the benefits of both these approaches, we present a level generation and game blending approach that combines the use of VAEs and QD algorithms. Specifically, we train VAEs on game levels and run the MAP-Elites QD algorithm using the learned latent space of the VAE as the search space. The latent space captures the properties of the games whose levels we want to generate and blend, while MAP-Elites searches this latent space to find a diverse set of levels optimizing a given objective such as playability. We test our method using models for 5 different platformer games as well as a blended domain spanning 3 of these games. We refer to using MAP-Elites for blending as Blend-Elites. Our results show that MAP-Elites in conjunction with VAEs enables the generation of a diverse set of playable levels not just for each individual game but also for the blended domain while illuminating game-specific regions of the blended latent space.

For citizen science games (CSGs) to be successful in advancing scientific research, they must effectively train players. Designing tutorials for training can be aided through developing a skill chain of required skills and their dependencies, but skill chain development is an intensive process. In this work, we hypothesized that free recall may be a simpler yet effective method of directly eliciting skill chains. We elicited 23 skill chains from players and developers and augmented our reflexive thematic analysis with 11 semi-structured interviews in order to determine how players and developers conceptualize skill trees and whether free recall can be used as an alternative to more resource-intensive cognitive task analyses. We provide three main contributions: (1) a comparison of skill chain conceptualizations between players and developers and across prior literature; (2) insights to the process of free recall in eliciting CSG skill chains; and (3) a preliminary toolkit of CSG skill-based design recommendations based on our findings. We conclude CSG developers should: give the big picture up front; embrace social learning and paratext use; reinforce the intended structure of knowledge; situate learning within applicable, meaningful contexts; design for discovery and self-reflection; and encourage practice and learning beyond the tutorial. Free recall was ineffective for determining a traditional skill chain but was able to elicit the core gameplay loops, tutorial overviews, and some expert insights.

In recent years, machine learning (ML) systems have been increasingly applied for performing creative tasks. Such creative ML approaches have seen wide use in the domains of visual art and music for applications such as image and music generation and style transfer. However, similar creative ML techniques have not been as widely adopted in the domain of game design despite the emergence of ML-based methods for generating game content. In this paper, we argue for leveraging and repurposing such creative techniques for designing content for games, referring to these as approaches for Game Design via Creative ML (GDCML). We highlight existing systems that enable GDCML and illustrate how creative ML can inform new systems via example applications and a proposed system.

Existing methods of level generation using latent variable models such as VAEs and GANs do so in segments and produce the final level by stitching these separately generated segments together. In this paper, we build on these methods by training VAEs to learn a sequential model of segment generation such that generated segments logically follow from prior segments. By further combining the VAE with a classifier that determines whether to place the generated segment to the top, bottom, left or right of the previous segment, we obtain a pipeline that enables the generation of arbitrarily long levels that progress in any of these four directions and are composed of segments that logically follow one another. In addition to generating more coherent levels of non-fixed length, this method also enables implicit blending of levels from separate games that do not have similar orientation. We demonstrate our approach using levels from Super Mario Bros., Kid Icarus and Mega Man, showing that our method produces levels that are more coherent than previous latent variable-based approaches and are capable of blending levels across games.

Crowdsourcing games involving image labeling tasks are commonly digital, played online, and have rules set by designers. In this work we explore the potential of tabletop image labeling games, incorporating physical elements, in-person community-based gameplay, and support for customizable rules. We developed an augmented reality game toolkit called Tile-o-Scope AR and conducted two studies. The first study demonstrates how the toolkit can facilitate in-person discussions through collaborative image labeling, and the toolkit’s potential adaptability to other games and applications. The second study, using three different activities designed for the toolkit, demonstrates the toolkit’s flexibility for creating customized experiences for audiences of different backgrounds.

Player engagement within a game is often influenced by its difficulty curve: the pace at which in-game challenges become harder. Thus, finding an optimal difficulty curve is important. In this paper, we present a flexible and formal approach to transforming game difficulty curves by leveraging function composition. This allows us to describe changes to difficulty curves, such as making them "smoother", in a more precise way. In an experiment with 400 players, we used function composition to modify the existing difficulty curve of the puzzle game Paradox to generate new curves. We found that transforming difficulty curves in this way impacted player engagement, including the number of levels completed and the estimated skill needed to complete those levels, as well as perceived competence. Further, we found some transformed curves dominated others with respect to engagement, indicating that different design goals can be traded-off by considering a subset of curves.

Prior work has focused on formalizing difficulty curves by using function composition to give precise definitions to curves and their transformations. However, the proposed framework was demonstrated using a single game, and the curves and transformations were defined with respect to the game's ratings-based dynamic difficulty system. In this work, we infer difficulty curves from gameplay data using a method that is based on the aforementioned difficulty system but that can also be generalized to other games for which information on player-vs-level win/loss outcomes is available. Moreover, since this method uses the same difficulty mechanism as past work, it lets us similarly leverage function composition to compare difficulty curves across games, having either a fixed or dynamic level ordering, using a clearly defined vocabulary. We use four different games to demonstrate our method, which relies on an adjustment to traditional playback of ratings-based match data, which we also present in this work.

Many studies have already shown that games can be a useful tool to make boring or difficult tasks more engaging. However, with serious game design being a relatively nascent field, such experiences can still be hard to learn and not very motivating. In this paper, we explore the use of learning and motivation frameworks to improve player experience in the well-known citizen science game Foldit. Using Cognitive Load Theory (CLT) and Self Determination Theory (SDT), we developed six interface and mechanical changes to the tutorial levels in Foldit designed to increase engagement and retention. We tested these features with new players of Foldit and collected both behavioral data, using game metrics, and prior experience data, using self-report measures. This study offers three major contributions: (1) we document the process of operationalizing CLT and SDT as new game features, a unique methodology not used in game design previously; (2) the user interface, specifically the level selection screen, significantly impacts how players progress through the game; and (3) a player's expertise, whether from prior domain knowledge or prior gaming experience, increases their engagement. We discuss both implications of these findings as well as how these implementations can generalize to other designs.

Rating systems (like Elo and Glicko-2) have previously been used for predicting the expected score that a player will achieve on a level. We present an approach that predicts not a single score, but an approximate cumulative distribution function over possible scores. This approach assigns each level an array of multiple ratings for different score thresholds. Our long-term goal is twofold: first, to dynamically change level difficulty for each player by using this CDF to tailor the target score required to complete a level; second, in human computation games (HCGs), to identify players capable of setting new high scores that could correspond to improved solutions to underlying tasks. To move towards this goal, we explore the rating array approach using two datasets: one gathered from the HCG Paradox, and one generated from idealized players and levels. We examine the accuracy of the CDF and the expected scores it predicts, as well as its use in serving levels to players who could set new high scores.

Software verification addresses the important societal problem of software correctness by using tools to mechanically prove that software is free of errors. Since the software verification problem is undecidable, automated tools have limited capabilities; hence, to verify non-trivial software, engineers use human-in-the-loop theorem provers that depend on human-provided insights such as loop invariants. The effective use of modern theorem provers requires significant expertise and recent work has explored the possibility of creating human computation games that enable non-experts to find useful loop invariants. A common feature of these games is that they do not show the code to be verified. We present and evaluate a game which does show players code. Showing code poses a number of design challenges, such as avoiding cognitive overload, but, as our experimental evaluation confirms, also provides an opportunity for richer human-computer interactions that lead to more effective human-in-the-loop systems which augment the ability of programmers who are not verification experts to find loop invariants.

Scientific software is often developed with professional scientists in mind, resulting in complex tools with a steep learning curve. Citizen science games, however, are designed for citizen scientists---members of the general public. These games maintain scientific accuracy while placing design goals such as usability and enjoyment at the forefront. In this paper, we identify an emerging use of game-based technology, in the repurposing of citizen science games to be software tools for professional scientists in their work. We discuss our experience in two such repurposings: Foldit, a protein folding and design game, and Eyewire, a web-based 3D neuron reconstruction game. Based on this experience, we provide evidence that the software artifacts produced for citizen science can be useful for professional scientists, and provide an overview of key design principles we found to be useful in the process of repurposing.

Previous work has demonstrated the effectiveness of rating system-based matchmaking for level ordering within the particular constraints of human computation games. However, players were not informed about the rating system, nor allowed to choose the difficulty of upcoming levels. Informing players of the ratings used in the system and offering them choice of upcoming level difficulty may enhance feelings of competence and control respectively, thereby further improving player engagement. Thus, we attempted to improve player experience by both exposing players to the underlying rating system, as well as offering them choice of level difficulty. We found that players cognizant of ratings both attempted and completed more levels than those who were not. Though additionally offering choice did not significantly affect behavior, we found that player choice was influenced by the outcome of the preceding level. Moreover, we did not observe any significant impact on self-reported measures of subjective experience.

Paid platforms like Mechanical Turk are popular for recruiting players for playtesting and experiments. However, it is unclear if paid players have similar behavior or experiences as volunteers (i.e. players recruited for free through banner ads or game portals). In this work, we studied the impact of recruitment within human computation games, using two experiments. First, we compared voluntary recruitment versus paid recruitment with different compensation levels. We found that the highest paid players completed more levels (i.e. achieved a higher volume of completed tasks) and reported greater engagement than both volunteers and players paid less while volunteers completed levels of higher difficulty (i.e. achieved a higher quality of completed tasks) than paid players. Additionally, we also varied both recruitment strategy and the game's design and found no interaction effects, suggesting that while differences exist between volunteer and paid players, experimental changes do not impact those players differently.

Multiplayer Online Battle Arena (MOBA) games have received increasing popularity recently. In a match of such games, players compete in two teams of five, each controlling an in-game avatar, known as heroes, selected from a roster of more than 100. The selection of heroes, also known as pick or draft, takes place before the match starts and alternates between the two teams until each player has selected one hero. Heroes are designed with different strengths and weaknesses to promote team cooperation in a game. Intuitively, heroes in a strong team should complement each other's strengths and suppress those of opponents. Hero drafting is therefore a challenging problem due to the complex hero-to-hero relationships to consider. In this paper, we propose a novel hero recommendation system that suggests heroes to add to an existing team while maximizing the team's prospect for victory. To that end, we model the drafting between two teams as a combinatorial game and use Monte Carlo Tree Search (MCTS) for estimating the values of hero combinations. Our empirical evaluation shows that hero teams drafted by our recommendation algorithm have a significantly higher win rate against teams constructed by other baseline and state-of-the-art strategies.

While many popular casual games use three-star systems, which give players up to three stars based on their performance in a level, this technique has seen limited application in human computation games (HCGs). This gives rise to the question of what impact, if any, a three-star system will have on the behavior of players in HCGs. In this work, we examined the impact of a three-star system implemented in the protein folding HCG Foldit. We compared the basic game's introductory levels with two versions using a three-star system, where players were rewarded with more stars for completing levels in fewer moves. In one version, players could continue playing levels for as many moves as they liked, and in the other, players were forced to reset the level if they used more moves than required to achieve at least one star on the level. We observed that the three-star system encouraged players to use fewer moves, take more time per move, and replay completed levels more often. We did not observe an impact on retention. This indicates that three-star systems may be useful for re-enforcing concepts introduced by HCG levels, or as a flexible means to encourage desired behaviors.

Human computation games lack established ways of balancing the difficulty of tasks or levels served to players, potentially contributing to their low engagement rates. Traditional player rating systems have been suggested as a potential solution: using them to rate both players and tasks could estimate player skill and task difficulty and fuel player-task matchmaking. However, neither the effect of difficulty balancing on engagement in human computation games nor the use of player rating systems for this purpose has been empirically tested. We therefore examined the engagement effects of using the Glicko-2 player rating system to order tasks in the human computation game Paradox. An online experiment (n=294) found that both matchmaking-based and pure difficulty-based ordering of tasks led to significantly more attempted and completed levels than random ordering. Additionally, both matchmaking and random ordering led to significantly more difficult tasks being completed than pure difficulty-based ordering. We conclude that poor balancing contributes to poor engagement in human computation games, and that player rating system-based difficulty rating may be a viable and efficient way of improving both.

SUMMARY: Foldit Standalone is an interactive graphical interface to the Rosetta molecular modeling package. In contrast to most command-line or batch interactions with Rosetta, Foldit Standalone is designed to allow easy, real-time, direct manipulation of protein structures, while also giving access to the extensive power of Rosetta computations. Derived from the user interface of the scientific discovery game Foldit (itself based on Rosetta), Foldit Standalone has added more advanced features and removed the competitive game elements. Foldit Standalone was built from the ground up with a custom rendering and event engine, configurable visualizations and interactions driven by Rosetta. Foldit Standalone contains, among other features: electron density and contact map visualizations, multiple sequence alignment tools for template-based modeling, rigid body transformation controls, RosettaScripts support and an embedded Lua interpreter. AVAILABILITY AND IMPLEMENTATION: Foldit Standalone is available for download at https://fold.it/standalone , under the Rosetta license, which is free for academic and non-profit users. It is implemented in cross-platform C ++ and binary executables are available for Windows, macOS and Linux. CONTACT: [email protected].

We propose a novel sensing technique called proactive sensing. Proactive sensing continually repositions a camera-based sensor as a way to improve hand pose estimation. Our core contribution is a scheme that effectively learns how to move the sensor to improve pose estimation confidence while requiring no ground truth hand poses. We demonstrate this concept using a low-cost rapid swing arm system built around the state-of-the-art commercial sensing system Leap Motion. The results from our user study show that proactive sensing helps estimate users' hand poses with higher confidence compared to both static and random sensing. We further present an online model update to improve performance for each user.

One of the key challenges of video game design is teaching new players how to play. Although game developers frequently use tutorials to teach game mechanics, little is known about how tutorials affect game learnability and player engagement. Seeking to estimate this value, we implemented eight tutorial designs in three video games of varying complexity and evaluated their effects on player engagement and retention. The results of our multivariate study of over 45,000 players show that the usefulness of tutorials depends greatly on game complexity. Although tutorials increased play time by as much as 29% in the most complex game, they did not significantly improve player engagement in the two simpler games. Our results suggest that investment in tutorials may not be justified for games with mechanics that can be discovered through experimentation.

Program verification is the only way to be certain that a given piece of software is free of (certain types of) errors --- errors that could otherwise disrupt operations in the field. To date, formal verification has been done by specially-trained engineers. Labor costs have heretofore made formal verification too costly to apply beyond small, critical software components.

Secondary game objectives, optional challenges that players can choose to pursue or ignore, are a fundamental element of game design. Still, little is known about how secondary objectives affect player behavior. It is commonly believed that secondary objectives such as coins or collectible items can increase a game's flexibility, replayability, and depth. In contrast, we present results from analysis of two popular online Flash games showing that secondary objectives can easily harm the retention of many players. We support our findings with data collected from over 27,000 players through large-scale A/B tests in which we measured play time, progress, and return rate. We show that while secondary objectives can encourage long-term players to extend their playtime, they can also cause many players to play for less time. By modifying secondary objectives so that they reinforce the primary goal of the game instead of distracting from it, we are able to avoid negative consequences and still maintain the retention of long-term players. Our results suggest that secondary objectives that support the primary goal of the game are consistently useful, while secondary objectives that do not support the main goal require extensive testing to avoid negative consequences.

As games grow in complexity, gameplay needs to provide players with powerful means of managing this complexity. One approach is to give automation tools to players. In this paper, we analyze an in-game automation tool, the Foldit cookbook, for the scientific discovery game Foldit. The cookbook allows players to write recipes that can automate their strategies. Through analysis of cookbook usage, we observe that players take advantage of social mechanisms in the game to share, run, and modify recipes. Further, players take advantage of both a simplified visual programming interface and a text-based scripting interface for creating recipes. This indicates that there is potential for using automation tools to disseminate expert knowledge, and that it is useful to provide support for multiple authoring styles, especially for games where the final game goal is unbounded or hard to attain.

Visual data mining is a powerful technique allowing game designers to analyze player behavior. Playtracer, a new method for visually analyzing play traces, is a generalized heatmap that applies to any game with discrete state spaces. Unfortunately, due to its low discriminative power, Playtracer's usefulness is significantly decreased for games of even medium complexity, and is unusable on games with continuous state spaces. Here we show how the use of state features can remove both of these weaknesses. These state features collapse larger state spaces without losing salient information, resulting in visualizations that are significantly easier to interpret. We evaluate our work by analyzing player data gathered from three complex games in order to understand player behavior in the presence of optional rewards, identify key moments when players figure out the solution to the puzzle, and analyze why players give up and quit. Based on our experiences with these games, we suggest general principles for designers to identify useful features of game states that lead to effective play analyses.

Incorporating the individual and collective problem solving skills of non-experts into the scientific discovery process could potentially accelerate the advancement of science. This paper discusses the design process used for Foldit, a multiplayer online biochemistry game that presents players with computationally difficult protein folding problems in the form of puzzles, allowing ordinary players to gain expertise and help solve these problems. The principle challenge of designing such scientific discovery games is harnessing the enormous collective problem-solving potential of the game playing population, who have not been previously introduced to the specific problem, or, often, the entire scientific discipline. To address this challenge, we took an iterative approach to designing the game, incorporating feedback from players and biochemical experts alike. Feedback was gathered both before and after releasing the game, to create the rules, interactions, and visualizations in Foldit that maximize contributions from game players. We present several examples of how this approach guided the game's design, and allowed us to improve both the quality of the gameplay and the application of player problem-solving.

This paper describes an approach to building real-time highly-controllable characters. A kinematic character controller is built on-the-fly during a capture session, and updated after each new motion clip is acquired. Active learning is used to identify which motion sequence the user should perform next, in order to improve the quality and responsiveness of the controller. Because motion clips are selected adaptively, we avoid the difficulty of manually determining which ones to capture, and can build complex controllers from scratch while significantly reducing the number of necessary motion samples.

We present a real-time crowd model based on continuum dynamics. In our model, a dynamic potential field simultaneously integrates global navigation with moving obstacles such as other people, efficiently solving for the motion of large crowds without the need for explicit collision avoidance. Simulations created with our system run at interactive rates, demonstrate smooth flow under a variety of conditions, and naturally exhibit emergent phenomena that have been observed in real crowds.