Mark J. Nelson

dblp:76/6619 · DBLP profile ↗
← Back
28ranked-venue papers
7as first author
8since 2021 · last 2026
0000-0003-1882-8896ORCID · verified

Domains — the database's venue-derived domains; a paper can count in several

Human-computer interaction and ubiquitous computing · 19 · 5 first-author · 7 since 2021Graphics, computer vision, multimedia, augmented reality and games · 17 · 4 first-author · 6 since 2021Artificial intelligence and machine learning · 7 · 1 first-author · 1 since 2021Applied, interdisciplinary, general and emerging computing · 3 · 1 first-author
YearPublicationVenuePosition
2026 ParaAdapt: Parametric User Modeling & Adaptation for Navigating Complex Interactive Domains
Swen E. Gaudl, Mark J. Nelson, Günter Wallner
UMAP2
2025 Slice of Life: A Social Physics Game with Interactive Conversations using Symbolically Grounded LLM-Based Generative Dialogue
abstract
This paper describes the social physics game Slice of Life.In Slice of Life, the player strives to achieve various social goals by choosing social interactions for characters to engage in.These interactions are governed by a social simulation system called Ensemble with Social Practices (ESP).The ways to achieve the player's social goals are numerous and any given playthrough of the game will result in drastically different social worlds.Slice of Life also makes use of the underlying social simulation system's detailed state to generate symbolically grounded prompts for a large language model (LLM) that generates context-appropriate character dialogue.Rather than using LLMs for novelty or for economic reasons, the underlying social simulation technology, we argue, necessitates this approach in order to make it feasible to have nuanced dialogue that reflects the many ways characters could have gotten themselves into particular social situations.The purpose of this paper is to provide a detailed account of Slice of Life's design, how its social physics simulation enables interactive conversations based on social practices, and to illustrate how the generative possibilities of LLMs can be uniquely useful when applied as its natural language generation (NLG) system, without giving up authorial control of the gameplay or story.
Mike Treanor, Ben Samuel, Mark J. Nelson
FDG3
2024 Prompt Wrangling: On Replication and Generalization in Large Language Models for PCG Levels
abstract
The ChatGPT4PCG competition calls for participants to submit inputs to ChatGPT or prompts that guide its output toward instructions to generate levels as sequences of Tetris-like block drops. Prompts submitted to the competition are queried by ChatGPT to generate levels that resemble letters of the English alphabet. Levels are evaluated based on their similarity to the target letter and physical stability in the game engine. This provides a quantitative evaluation setting for prompt-based procedural content generation (PCG), an approach that has been gaining popularity in PCG, as in other areas of generative AI. This paper focuses on replicating and generalizing the competition results. The replication experiments in the paper first aim to test whether the number of responses gathered from ChatGPT is sufficient to account for the stochasticity requery the original prompt submissions to rerun the original scripts from the competition on different machines about six months after the competition organizers. We re-run the competition, using the original scripts, but on our own machines, several months later, and with varying sample sizes. We find that results largely replicate, except that two of the 15 submissions do much better in our replication, for reasons we can only partly determine. When it comes to generalization, we notice that the top-performing prompt has instructions for all 26 target levels hardcoded, which is at odds with the PCGML goal of generating new, previously unseen content from examples. We perform experiments in a more restricted few-shot prompting scenario, and find that generalization remains a challenge for current approaches.
Arash Moradi Karkaj, Mark J. Nelson, Ioannis Koutis, Amy K. Hoover
FDG2
2024 Prototyping Slice of Life: Social Physics with Symbolically Grounded LLM-based Generative Dialogue
abstract
This paper describes a prototype for the social physics game Slice of Life, and how it makes use of the underlying social simulation system’s sophisticated state to generate symbolically grounded prompts for a large language model (LLM) in order to generate context appropriate character dialogue. Rather than using LLMs for novelty or economic reasons, the underlying social simulation technology, we argue, necessitates this approach in order to make it feasible to have nuanced dialogue that reflects the many ways the characters could have gotten themselves into particular social situations. The primary goal of this paper is to illustrate how the generative possibilities of LLMs can be uniquely useful when applied as a controlled natural language generation (NLG) system, without giving up authorial control of the gameplay or story, or sacrificing accepted good game design practice.
Mike Treanor, Ben Samuel, Mark J. Nelson
FDG3
2024 Language Model Crossover: Variation through Few-Shot Prompting
abstract
This article pursues the insight that language models naturally enable an intelligent variation operator similar in spirit to evolutionary crossover. In particular, language models of sufficient scale demonstrate in-context learning, i.e., they can learn from associations between a small number of input patterns to generate outputs incorporating such associations (also called few-shot prompting). This ability can be leveraged to form a simple but powerful variation operator, i.e., to prompt a language model with a few text-based genotypes (such as code, plain-text sentences, or equations), and to parse its corresponding output as those genotypes’ offspring. The promise of such language model crossover (which is simple to implement and can leverage many different open source language models) is that it enables a simple mechanism to evolve semantically rich text representations (with few domain-specific tweaks), and naturally benefits from current progress in language models. Experiments in this article highlight the versatility of language-model crossover, through evolving binary bit-strings, sentences, equations, text-to-image prompts, and Python code. The conclusion is that language model crossover is a flexible and effective method for evolving genomes representable as text.
Elliot Meyerson, Mark J. Nelson, Herbie Bradley, Adam Gaier, Arash Moradi Karkaj, Amy K. Hoover, Joel Lehman
ACM Trans. Evol. Learn. Optim.2
2022 Neighborly: A Sandbox for Simulation-based Emergent Narrative
abstract
This paper presents Neighborly, a customizable, community-scale social simulation engine for procedurally generating settlements of characters for use in research experimentation or entertainment media. Neighborly is a rational reconstruction of Talk of the Town (TotT), an earlier social simulation for emergent narrative focused on simulating small American towns and the townspeople’s lives. Based on Talk of the Town’s previous success as part of the experimental game Bad News, we wanted to reconstruct it as a general-use social simulation authoring tool. In this paper, we delineate the design space of TotT-like social simulation and compare TotT-likes to other academic projects and commercial social simulation games. Finally, we provide an overview of how Neighborly embodies the essence of TotT while offering users a customizable tool for creating community-scale social simulations.
Shi Johnson-Bey, Mark J. Nelson, Michael Mateas
CoG2
2022 Exploring the Design Space of Social Physics Engines in Games
Shi Johnson-Bey, Mark J. Nelson, Michael Mateas
ICIDS2
2021 Estimates for the Branching Factors of Atari Games
abstract
The branching factor of a game is the average number of new states reachable from a given state. It is a widely used metric in AI research on board games, but less often computed or discussed for videogames. This paper provides estimates for the branching factors of 103 Atari 2600 games, as implemented in the Arcade Learning Environment (ALE). Depending on the game, ALE exposes between 3 and 18 available actions per frame of gameplay, which is an upper bound on branching factor. This paper shows, based on an enumeration of the first 1 million distinct states reachable in each game, that the average branching factor is usually much lower, in many games barely above 1. In addition to reporting the branching factors, this paper aims to clarify what constitutes a distinct state in ALE.
Mark J. Nelson
CoG1
2020 Notes on Using Google Colaboratory in AI Education
abstract
We discuss our experiences using Google Colaboratory (Colab), a hosted version of Jupyter Notebooks, in undergraduate artificial intelligence (AI) courses at two universities. Colab was designed for AI and data science researchers to share reproducible experiments and explanations of techniques, but we have also found it well suited to classroom use. The primary benefit is that it provides students computational resources sufficient to run modern AI techniques interactively, and avoids students needing to separately configure software packages and dependencies, since they can run notebooks shared by the instructor. We briefly outline two of our notebooks, for teaching deep learning with Tensorflow, and reinforcement learning with OpenAI Gym.
Mark J. Nelson, Amy K. Hoover
ITiCSE1
2019 Order-fulfillment games: an analysis of games about serving customers
abstract
Consider the set of games, which we'll call order-fulfillment games, where the player fulfills customers' orders in a food-service setting under time pressure. We will use BurgerTime (1982), Tapper (1983), Diner Dash (2004), and Overcooked (2016) as examples. We argue that, although these games don't form a genre per se, they form a coherent taxonomic grouping defined by their core game loop, thematic elements, and typical player experiences. That these games share similarities may seem obvious, but we have found it illuminating to dig into precisely how this grouping is constituted, where its boundaries lie, and how it overlaps with well-recognized game genres. Besides analyzing this grouping for its own sake, a secondary contribution of this paper is as a case study in applying two analytical tools that have been proposed but little applied: Lessard's high-level design pattern formations and Sicart's version of the core game loop.
Mike Treanor, Mark J. Nelson
FDG2
2019 Towards Simulated Morality Systems: Role-Playing Games as Artificial Societies
abstract
Computer role-playing games (RPGs) often include a simulated morality system as a core design element. Games' morality systems can include both god's eye view aspects, in which certain actions are inherently judged by the simulated world to be good or evil, as well as social simulations, in which non-player characters (NPCs) react to judgments of the player's and each others' activities. Games with a larger amount of social simulation have clear affinities to multi-agent systems (MAS) research on artificial societies. They differ in a number of key respects, however, due to a mixture of pragmatic game-design considerations and their typically strong embeddedness in narrative arcs, resulting in many important aspects of moral systems being represented using explicitly scripted scenarios rather than through agent-based simulations. In this position paper, we argue that these similarities and differences make RPGs a promising challenge domain for MAS research, highlighting features such as moral dilemmas situated in more organic settings than seen in game-theoretic models of social dilemmas, and heterogeneous representations of morality that use both moral calculus systems and social simulation. We illustrate some possible approaches using a case study of the morality systems in the game The Elder Scrolls IV: Oblivion.
Joan Casas-Roma, Mark J. Nelson, Joan Arnedo-Moreno, Swen E. Gaudl, Rob Saunders
ICAART (1)2
2019 Orchestrating Game Generation
abstract
—The design process is often characterized by and realized through the iterative steps of evaluation and refinement. When the process is based on a single creative domain such as visual art or audio production, designers primarily take inspiration from work within their domain and refine it based on their own intuitions or feedback from an audience of experts from within the same domain. What happens, however, when the creative process involves more than one creative domain such as in a digital game? How should the different domains influence each other so that the final outcome achieves a harmonized and fruitful communication across domains? How can a computational process orchestrate the various computational creators of the corresponding domains so that the final game has the desired functional and aesthetic characteristics? To address these questions, this paper identifies game facet orchestration as the central challenge for artificial-intelligence-based game generation, discusses its dimensions, and reviews research in automated game generation that has aimed to tackle it. In particular, we identify the different creative facets of games, propose how orchestration can be facilitated in a top-down or bottom-up fashion, review indicative preliminary examples of orchestration, and conclude by discussing the open questions and challenges ahead.
Antonios Liapis, Georgios N. Yannakakis, Mark J. Nelson, Mike Preuss, Rafael Bidarra
IEEE Trans. Games3
2018 Curious users of casual creators
abstract
Casual creators are a type of design tool identified by Compton & Mateas, characterised by an orientation towards enjoyable, intrinsically motivated creative exploration, rather than task-oriented designer productivity. In our experiments holding rapid game jams with Wevva, a casual creator for mobile game design, we have noticed, however, that users seem to vary considerably even within the context of using a casual creator. Some people focus on designing specific games, while others explore the design space extensively, or even focus exclusively on prodding the edges of the design space looking for its possibilities and limits. We hypothesise that the latter group of users is driven primarily by curiosity about a casual creator and its design space. This results in different patterns of behaviour to the former group (of design-oriented users), which may worth characterising and perhaps explicitly designing for.
Mark J. Nelson, Swen E. Gaudl, Simon Colton, Sebastian Deterding
FDG1
2018 A Parameter-Space Design Methodology for Casual Creators
Peter Ivey, Blanca Pérez Ferrer, Rob Saunders, Swen E. Gaudl, Edward J. Powley, Mark J. Nelson, Simon Colton, Michael Cook 0001
ICCC6
2017 Fluidic Games in Cultural Contexts
Mark J. Nelson, Swen E. Gaudl, Simon Colton, Edward J. Powley, Blanca Pérez Ferrer, Rob Saunders, Peter Ivey, Michael Cook 0001
ICCC1
2017 The Narrative Logic of Rube Goldberg Machines
David Olsen, Mark J. Nelson
ICIDS2
2017 1st Workshop on the History of Expressive Systems
James Owen Ryan, Mark J. Nelson
ICIDS2
2015 General Video Game Evaluation Using Relative Algorithm Performance Profiles
Thorbjørn S. Nielsen, Gabriella A. B. Barros, Julian Togelius, Mark J. Nelson
EvoApplications4
2015 Game mechanics telling stories? An experiment
Kristian Hjaltason, Steffen Christophersen, Julian Togelius, Mark J. Nelson
FDG4
2014 X-COM: UFO Defense vs. XCOM: Enemy Unknown - Using gameplay design patterns to understand game remakes
Alessandro Canossa, Staffan Björk, Mark J. Nelson
FDG3
2014 A prototype using territories and an affordance tree for social simulation gameplay
Tilman Geishauser, Yun-Gyung Cheong, Mark J. Nelson
FDG3
2013 Competitive coevolution in Ms. Pac-Man
abstract
In this paper we investigate the suitability of the arcade game Ms. Pac-Man, as implemented in the recent Pac-Man versus Ghost Teams Competition, as a testbed for competitive coevolution. To that end, we explore competitive co-evolution techniques to co-evolve Pac-Man and Ghosts team controllers. We analyze in some detail the dynamics of evolution between the two classes and compare them with single-objective evolution and static controllers. We note differences between evolutions of the two classes, having observed higher fitness transitivity in Pac-Man than in the Ghosts. The problem of finding a well-performing general purpose Pac-Man is far different than that of finding a good and general Ghosts controller.
Andrew Borg Cardona, Julian Togelius, Mark J. Nelson
IEEE Congress on Evolutionary Computation3
2013 Design metaphors for procedural content generation in games
abstract
Procedural content generation (PCG), the algorithmic creation of game content with limited or indirect user input, has much to offer to game design. In recent years, it has become a mainstay of game AI, with significant research being put towards the investigation of new PCG systems, algorithms, and techniques. But for PCG to be absorbed into the practice of game design, it must be contextualised within design-centric as opposed to AI or engineering perspectives. We therefore provide a set of design metaphors for understanding potential relationships between a designer and PCG. These metaphors are: tool, material, designer, and domain expert. By examining PCG through these metaphors, we gain the ability to articulate qualities, consequences, affordances, and limitations of existing PCG approaches in relation to design. These metaphors are intended both to aid designers in understanding and appropriating PCG for their own contexts, and to advance PCG research by highlighting the assumptions implicit in existing systems and discourse.
Rilla Khaled, Mark J. Nelson, Pippin Barr
CHI2
2013 Player Perspectives to Unexplained Agency-Related Incoherence
Miika Pirtola, Yun-Gyung Cheong, Mark J. Nelson
ICIDS3
2009 A requirements analysis for videogame design support tools
abstract
Designing videogames involves weaving together systems of rules, called game mechanics, which support and structure compelling player experiences. Thus a significant portion of game design involves reasoning about the effects of different potential game mechanics on player experience. Unlike some design fields, such as architecture and mechanical design, that have CAD tools to support designers in reasoning about and visualizing designs, game designers have no tools for reasoning about and visualizing systems of game mechanics. In this paper we perform a requirements analysis for design-support tool for game design. We develop a proposal in two phases. First, we review the design-support-system and game-design literatures to arrive at a plausible system that helps designers reason about game mechanics and gameplay. We then refine these requirements in a study of three teams of game designers, investigating their current design problems and gauging interest in our tool proposals and reactions to prototype tools. Our study finds that a game design assistant that is able to formally reason about abstract game mechanics would provide significant leverage to designers during multiple stages of the design process.
Mark J. Nelson, Michael Mateas
FDG1
2008 Another Look at Search-Based Drama Management
Mark J. Nelson, Michael Mateas
AAAI1
2008 An interactive game-design assistant
abstract
Game-design novices increasingly hope to use game-like expression as a way to express content such as opinions and educational material. Existing game-design toolkits such as Game Maker ease the programming burden, bringing the design of small games within the technical reach of low-budget, non-expert groups. The design process itself remains a roadblock, however: It is not at all obvious how to present topics such as political viewpoints or bike safety in a way that makes use of the unique qualities of the interactive game medium. There are no tools to assist in that aspect of the game design process, and as a result virtually all expressive games come from a small number of game studios founded by experts in designing such games. We propose a game-design assistant that acts in a mixed-initiative fashion, helping the author understand the content of her design-in-progress, providing suggestions or automating the process where possible, and even offering the possibility for parts of the game to be dynamically generated at runtime in response to player interaction. We describe a prototype system that interactively helps authors define spaces of games in terms of common-sense constraints on their real-world references, provides support for them to understand and iteratively refine such spaces, and realizes specific games from the spaces as playable mobile-phone games in response to user input.
Mark J. Nelson, Michael Mateas
IUI1
2006 Targeting Specific Distributions of Trajectories in MDPs
David L. Roberts 0001, Mark J. Nelson, Charles L. Isbell Jr., Michael Mateas, Michael L. Littman
AAAI2