Play style identification can provide valuable game design insights and enable adaptive experiences, with the potential to improve game playing agents. Previous work relies on domain knowledge to construct play trace representations using handcrafted features. More recent approaches incorporate the sequential structure of play traces but still require some level of domain abstraction. In this study, we explore the use of unsupervised CNN-LSTM autoencoder models to obtain latent representations directly from low-level play trace data in MicroRTS. We demonstrate that this approach yields a meaningful separation of different game playing agents in the latent space, reducing reliance on domain expertise and its associated biases. This latent space is then used to guide the exploration of diverse play styles within studied AI players.

Large Language Models (LLMs) have shown great success as high-level planners for zero-shot game-playing agents However, these agents are primarily evaluated on games where long-term planning is relatively straightforward. In contrast agents tested in more dynamic environments face limitations due to simplistic environments with only a few objects and interactions. To fill this gap in the literature, we present NetPlay, the first LLM powered zero-shot agent for the challenging roguelike NetHack. NetHack is a particularly challenging environment due to its diverse set of items and monsters, complex interactions, and many ways to die. NetPlay uses an architecture designed for dynamic robot environments, modified for NetHack. Like previous approaches, it prompts the LLM to choose from predefined skills and tracks past interactions to enhance decision-making. Given NetHack’s unpredictable nature, NetPlay detects important game events to interrupt running skills, enabling it to react to unforeseen circumstances. While NetPlay demonstrates considerable flexibility and proficiency in interacting with NetHack’s mechanics, it struggles with ambiguous task descriptions and a lack of explicit feedback. Our findings demonstrate that NetPlay performs best with detailed context information, indicating the necessity for dynamic methods in supplying context information for complex games such as NetHack.

Exploration is found in a variety of game genres, but there has been little research in the context of PCG. This paper investigates the potential for exploratory agents to provide feedback on how well levels support exploration, with the ultimate goal of guiding level generation. We propose several motivations which might drive exploratory behaviour and model these as metrics within an agent framework based on context steering. We present a study of how the different metrics influence exploration of six game levels. It was found that combinations of metrics lead to distinct exploratory behaviours, mostly within our expectations.

Many human-AI authoring tools are used in a playful way, while being primarily designed for task-achievement—not playfulness. We argue that playfulness is an important yet overlooked factor of user behaviour and experience when interacting with such tools. Motivating and rewarding playfulness as an exploratory, task-agnostic, open, and subversive attitude can support the satisfaction of more diverse user goals, and have a strong, positive effect on the user experience, the emerging human-AI interaction, and the resulting artefact. In this paper, we motivate the importance of playfulness as user experience in human-AI authoring tools, and propose concrete strategies to design for playfulness in the human user through UI design, in the AI through algorithms, or through interventions to their dialog. We conclude with an outlook of the research agenda.

In order to advance the field of procedural content generation, and transfer knowledge from academic research to everyday use, we need to develop tools that make generative systems easier to understand and control. In this article, we introduce Danesh, a plugin to the unity game development environment, which helps provide a suite of tools that provide automation or analysis of different aspects of procedural generators. We describe here the features of Danesh, including automatic analysis of generated content, the visualization of generative spaces, automatic parameter discovery, and interface smoothing. We also provide reflections on our development of the tool so far.

Action advising is a budget-constrained knowledge exchange mechanism between teacher–student peers that can help tackle exploration and sample inefficiency problems in deep reinforcement learning (RL). Most recently, student-initiated techniques that utilize state novelty and uncertainty estimations have obtained promising results. However, the approaches built on these estimations have some potential weaknesses. First, they assume that the convergence of the student’s RL model implies less need for advice. This can be misleading in scenarios with teacher’s absence early on where the student is likely to learn suboptimally by itself; yet also ignore the teacher’s assistance later. Second, the delays between encountering states and having them to take effect in the RL model updates in the presence of the experience replay dynamics cause a feedback lag in what the student actually needs advice for. We propose a student-initiated algorithm that alleviates these by employing random network distillation (RND) to measure the novelty of a piece of advice. Furthermore, we perform RND updates only for the advised states to ensure that the student’s own learning does not impair its ability to leverage the teacher. Experiments inGridWorldandMinAtarshow that our approach performs on par with the state of the art and demonstrates significant advantages in the scenarios where the existing methods are prone to fail.

Deep Reinforcement Learning (RL) techniques can benefit greatly from leveraging prior experience, which can be either self-generated or acquired from other entities. Action advising is a framework that provides a flexible way to transfer such knowledge in the form of actions between teacher-student peers. However, due to the realistic concerns, the number of these interactions is limited with a budget; therefore, it is crucial to perform these in the most appropriate moments. There have been several promising studies recently that address this problem setting especially from the student's perspective. Despite their success, they have some shortcomings when it comes to the practical applicability and integrity as an overall solution to the learning from advice challenge. In this paper, we extend the idea of advice reusing via teacher imitation to construct a unified approach that addresses both advice collection and advice utilisation problems. We also propose a method to automatically tune the relevant hyperparameters of these components on-the-fly to make it able to adapt to any task with minimal human intervention. The experiments we performed in 5 different Atari games verify that our algorithm either surpasses or performs on-par with its top competitors while being far simpler to be employed. Furthermore, its individual components are also found to be providing significant advantages alone.

We introduce the concept of Adversarial Behaviour Debugging (ABD), using intelligent agents to assist in the debugging of human-authored game AI systems, and Bonobo, an ABD system for Unity. To investigate the differences between ABD and traditional playtesting, we collected gameplay data from Bonobo agents and human testers playing against a buggy AI opponent. We present a comparison of the differences in gameplay and an online study on whether these differences affect observers perceptions of the AI opponent. We found that while there were clear differences, ABD compares favourably to human testing and has the potential to offer a distinct perspective.

Most generative systems built in game development are parameter-driven, but the relationship between parameters and the output of the system is often unclear. This makes them frustrating to use for both experts and novices, and as a result generators are often filtered post-hoc, or tweaked through time-consuming trial and error. In this paper we introduce two analytical techniques: smoothness and codependence. We show how these features help analyse the impact of a parameter change on a generative system and suggest ways this could feed back into more intelligent tools that make working with procedural generators more precise and pleasant.

Deep Reinforcement Learning (RL) algorithms can solve complex sequential decision tasks successfully. However, they have a major drawback of having poor sample efficiency which can often be tackled by knowledge reuse. In Multi-Agent Reinforcement Learning (MARL) this drawback becomes worse, but at the same time, a new set of opportunities to leverage knowledge are also presented through agent interactions. One promising approach among these is peer-to-peer action advising through a teacher-student framework. Despite being introduced for single-agent RL originally, recent studies show that it can also be applied to multi-agent scenarios with promising empirical results. However, studies in this line of research are currently very limited. In this paper, we propose heuristics-based action advising techniques in cooperative decentralised MARL, using a nonlinear function approximation based task-level policy. By adopting Random Network Distillation technique, we devise a measurement for agents to assess their knowledge in any given state and be able to initiate the teacher-student dynamics with no prior role assumptions. Experimental results in a gridworld environment show that such an approach may indeed be useful and needs to be further investigated.

Automatically generating content for videogames has long been a staple of game development and the focus of much successful research. Such forays into content generation usually concern themselves with producing a specific game component, such as a level design. This has proven a rich and challenging area of research, but in focusing on creating separate parts of a larger game, we miss out on the most challenging and interesting aspects of game development. By expanding our scope to the automated design of entire games, we can investigate the relationship between the different creative tasks undertaken in game development, tackle the higher level creative challenges of game design, and ultimately build systems capable of much greater novelty, surprise, and quality in their output. This paper, the first in a series of two, describes two case studies in automating game design, proposing cooperative coevolution as a useful technique to use within systems that automate this process. We show how this technique allows essentially separate content generators to produce content that complements each other. We also describe systems that have used this to design games with subtle emergent effects. After introducing the technique and its technical basis in this paper, in the second paper in the series we discuss higher level issues in automated game design, such as potential overlap with computational creativity and the issue of evaluation.

Procedural content generation is generally viewed as a means to an end-a tool employed by designers to overcome technical problems or achieve a particular design goal. When we move from generating single parts of games to automating the entirety of their design, however, we find ourselves facing a far wider and more interesting set of problems than mere generation. When the designer of a game is a piece of software, we face questions about what it means to be a designer, about computational creativity, and about how to assess the growth of these automated game designers and the value of their output. Answering these questions can lead to new ideas in how to generate content procedurally, and produce systems that can further the cutting edge of game design. This paper describes work done to take an automated game designer and advance it towards being a member of a creative community. We outline extensions made to the system to give it more autonomy and creative independence, in order to strengthen claims that the software is acting creatively. We describe and reflect upon the software's participation in the games community, including entering two game development contests, and show the opportunities and difficulties of such engagement. We consider methods for evaluating automated game designers as creative entities, and underline the need for automated game design to be a major frontier in future games research.

Computational analysis of player style has significant potential for video game design: it can provide insights into player behavior, as well as the means to dynamically adapt a game to each individual's style of play. To realize this potential, computational methods need to go beyond considerations of challenge and ability and account for aesthetic aspects of player style. We describe here a semiautomatic unsupervised learning approach to modeling player style using multiclass linear discriminant analysis (LDA). We argue that this approach is widely applicable for modeling player style in a wide range of games, including commercial applications, and illustrate it with two case studies: the first for a novel arcade game called Snakeotron, and the second for Rogue Trooper, a modern commercial third-person shooter video game.

Abstract This article reports on a longitudinal study of information seeking by undergraduate information management students. It describes how they found and used information, and explores their motivation and decision making. We employed a use‐in‐context approach where students were observed conducting, and were interviewed about, information‐seeking tasks carried out during their academic work. We found that participants were reluctant to engage with a complex range of information sources, preferring to use the Internet. The main driver for progress in information seeking was the immediate demands of their work (e.g., assignments). Students used their growing expertise to justify a conservative information strategy, retaining established strategies as far as possible and completing tasks with minimum information‐seeking effort. The time cost of using library material limited the uptake of such resources. New methods for discovering and selecting information were adopted only when immediately relevant to the task at hand, and tasks were generally chosen or interpreted in ways that minimized the need to develop new strategies. Students were driven by the demands of the task to use different types of information resources, but remained reluctant to move beyond keyword searches, even when they proved ineffective. They also lacked confidence in evaluating the relative usefulness of resources. Whereas existing literature on satisficing has focused on stopping conditions, this work has highlighted a richer repertoire of satisficing behaviors.

Abstract A user's understanding of the libraries they work in, and hence of what they can do in those libraries, is encapsulated in their “mental models” of those libraries. In this article, we present a focused case study of users' mental models of traditional and digital libraries based on observations and interviews with eight participants. It was found that a poor understanding of access restrictions led to risk‐averse behavior, whereas a poor understanding of search algorithms and relevance ranking resulted in trial‐and‐error behavior. This highlights the importance of rich feedback in helping users to construct useful mental models. Although the use of concrete analogies for digital libraries was not widespread, participants used their knowledge of Internet search engines to infer how searching might work in digital libraries. Indeed, most participants did not clearly distinguish between different kinds of digital resource, viewing the electronic library catalogue, abstracting services, digital libraries, and Internet search engines as variants on a theme.

Tools to design, analyse and evaluate user interfaces can be used in user interface design research and in interface modelling research. This demonstration shows two working systems: one in Mathematica that is mathematically sophisticated, and one as a ‘conventional ’ rapid application development environment, where the mathematics is hidden, and which could form the basis of a professional design tool — but which is based rigorously on the same algebraic formalism.