Behavior Trees (BTs) enable dynamic and strategic gameplay by controlling non-player characters (NPCs) in video games, but their construction and iterative refinement remain time-consuming and technically demanding for game developers. This project explores the integration of Large Language Models (LLMs) within Unreal Engine 5 as a tool to automatically generate BTs from natural language prompts, thereby reducing the complexity of designing game AI and accelerating the production pipeline. In order to choose the adapted model in terms of coherence and practicability for local integration, multiple models were benchmarked. Results revealed that models under 2.5 GB provided the best trade-off between performance and output quality. The LLaMA 3.2 3B Instruct quantized model was found to be particularly suitable. In this study, we have thus developed a tool based on the Llama-Unreal Plugin to implement a local LLM into the engine. A custom prompt interface was implemented in the editor, allowing users to submit natural language inputs. The model responds with an XML description of the BT, which is then parsed and instantiated as a native BT asset in Unreal Engine. The XML schema was carefully designed to accommodate different node types (tasks, composites, decorators). The implemented system was tested with simple NPC behaviors like seeking and chasing the player for a guardian, or like a boss fight sequence for a dragon. It successfully produced functional BTs from simple prompts in various gameplay scenarios. Early results have shown how LLMs can streamline the creation of Game AI. The system could be extended to support more complex and hierarchical BTs, improve error handling in XML parsing, and integrate with other game engines such as Unity or Godot.

There is a global shortage of therapeutic, educational, and social support for neurodivergent children and youth, with substantial downstream consequences for individual well-being and societal costs in adulthood. While early and sustained support is known to be both humane and economically effective, existing service models struggle to scale. Therapeutically Assisted Role-Playing Games (TARPGs) have recently emerged as a promising, group-based intervention that combines guided play, narrative structure, and social skill rehearsal to support neurodivergent development. Case studies consistently report positive outcomes, including increased engagement, social connection, emotional regulation, and self-efficacy. However, current research remains dominated by small-scale and non-experimental case studies, and practical barriers persist for wider adoption, especially for carers without formal therapeutic training. Therapists are often not available, and carers are the next, and more numerous, line of support for neurodivergent children. This paper introduces AURA (AI-Assisted Understanding of Role-play), a novel system designed to support therapists and carers running TARPG sessions. AURA addresses a key practical challenge in TARPG facilitation: documenting complex, multi-player narrative sessions without disrupting live interaction. By analysing audio and/or video recordings of sessions, AURA generates structured post-session summaries, and key event timeline, enabling facilitators to focus on participants rather than note-taking. AURA aims to lower barriers to entry, extend TARPG capacity beyond scarce clinical settings, and support more scalable, evidence-informed interventions for neurodivergent children and youth.

The Upper Confidence Bounds For Trees (UCT) algorithm is not agnostic to the reward scale of the game it is applied to. For zero-sum games with the sparse rewards of$\lbrace -1,0,1\rbrace$at the end of the game, this is not a problem, but many games often feature dense rewards with hand-picked reward scales, causing a node's Q-value to span different magnitudes across different games. In this paper, we evaluate various strategies for adaptively choosing the UCT exploration constant$\lambda$, called$\lambda$-strategies, that are agnostic to the game's reward scale. These$\lambda$-strategies include those proposed in the literature as well as five new strategies. Given our experimental results, we recommend using one of our newly suggested$\lambda$-strategies, which is to choose$\lambda$as$2 \cdot \sigma$where$\sigma$is the empirical standard deviation of all state-action pairs' Q-values of the search tree. This method outperforms existing$\lambda$-strategies across a wide range of tasks both in terms of a single parameter value and the peak performances obtained by optimizing all available parameters.

Competitive environments, whether in traditional sports or competitive online gaming, often foster intense emotions and harsh language. In Competitive Online Multiplayer Games, where players are not physically present together, verbal toxicity primarily manifests through voice and text chat. As online gaming remains a predominantly male-dominated space, much of this toxicity disproportionately targets non-male players. Behind the veil of anonymity, non-male players frequently face gender discrimination, hate speech, and unwanted sexual advances. To effectively address these issues, game developers must first understand how such harassment manifests and how it is perceived by players. This work examines gender discrimination in Competitive Online Multiplayer Games through two online surveys. The first survey gathered reports from 61 non-male players who had experienced harassment, resulting in 171 coded statements describing gender discrimination. A second survey presented these statements to 281 players across all genders, who rated their perceived severity and authenticity. Our results indicate that gender discrimination is generally perceived as equally severe across all gender groups. However, notable differences are visible in how players respond to harassment, with those from marginalized groups exhibiting higher levels of rumination. This highlights the compounded impact of gender-based toxicity in online gaming. These insights provide valuable direction for game developers seeking to create more inclusive and supportive gaming environments.

Tales of Tribute is a strategic deck-building card game that presents unique challenges for artificial intelligence due to its complex action space, hidden information, and long-term planning requirements. In this work, we propose a reinforcement learning agent that learns to play Tales of Tribute without relying on handcrafted heuristics. Our approach introduces a generalizable game state representation and a scalable action evaluation mechanism based on preference vectors. We train our agent using Proximal Policy Optimization within the Scripts of Tribute framework and demonstrate competitive performance against established search-based agents, including Monte Carlo Tree Search. The results validate the viability of reinforcement learning in Tales of Tribute and highlight the potential of preference-based action evaluation in domains with large and variable action spaces. Further experiments will be required to test its viability in other card games.

Baba Is You is a challenging puzzle game that requires agents to not only navigate spatial environments but also dynamically manipulate the game's rules. In this work, we extend prior evolutionary approaches to solving Baba Is You levels by introducing tree-based search policies evolved through genetic programming. These policies leverage a rich set of base heuristics to guide search more effectively than linear combinations used in earlier work. To improve generalization, we construct a minimal set of reusable search policies through integer linear programming and develop ensemble agents that select appropriate policies for new levels using learned classifiers. Our experiments show that tree-based policies outperform linear ones in terms of level coverage, and that nearest neighbor classifiers generalize well to previously unseen levels.

One paradigm of Monte Carlo Tree Search (MCTS) improvements is to build and use state and/or action abstractions during the tree search. Non-exact abstractions, however, introduce an approximation error making convergence to the optimal action in the abstract space impossible. Hence, as proposed as a component of Elastic Monte Carlo Tree Search by Xu et al., abstraction algorithms should eventually drop the abstraction. In this paper, we propose two novel abstraction dropping schemes, namely OGA-IAAD and OGA-CAD which can yield clear performance improvements whilst being safe in the sense that the dropping never causes any notable performance degradations contrary to Xu's dropping method. OGA-IAAD is designed for time critical settings while OGA-CAD is designed to improve the MCTS performance with the same number of iterations.

Balancing game difficulty in video games is a key task to create interesting gaming experiences for players. Mismatching the game difficulty and a player’s skill or commitment results in frustration or boredom on the player’s side, and hence reduces time spent playing the game. In this work, we explore balancing game difficulty using machine learning-based agents to challenge players based on their current behavior. This is achieved by a combination of two agents, in which one learns to imitate the player, while the second is trained to beat the first. In our demo, we investigate the proposed framework for personalized dynamic difficulty adjustment of AI agents in the context of the fighting game AI competition.

We propose RHEA CL, which combines Curriculum Learning (CL) with Rolling Horizon Evolutionary Algorithms (RHEA) to automatically produce effective curricula during the training of a reinforcement learning agent. RHEA CL optimizes a population of curricula, using an evolutionary algorithm, and selects the best-performing curriculum as the starting point for the next training epoch. Performance evaluations are conducted after every curriculum step in all environments. We evaluate the algorithm on the DoorKey and DynamicObstacles environments within the Minigrid framework. It demonstrates adaptability and consistent improvement, particularly in the early stages, while reaching a stable performance later that is capable of outperforming other curriculum learners. In comparison to other curriculum schedules, RHEA CL has shown to yield performance improvements for the final Reinforcement learning (RL) agent at the cost of additional evaluation during training.

Many works in the domain of artificial intelligence in games focus on board or video games due to the ease of reimplementing their mechanics [1], [2]. Decision-making problems in real-world sports share many similarities to such domains. Nevertheless, not many frameworks on sports games exist. In this paper, we present the tennis match simulation environment Match Point AI, in which different agents can compete against real-world data-driven bot strategies. Next to presenting the framework, we highlight its capabilities by illustrating, how MCTS can be used in Match Point AI to optimize the shot direction selection problem in tennis. While the framework will be extended in the future, first experiments already reveal that generated shot-by-shot data of simulated tennis matches show realistic characteristics when compared to real-world data. At the same time, reasonable shot placement strategies emerge, which share similarities to the ones found in real-world tennis matches.

Markov Junior is a probabilistic programming language used for procedural content generation across various domains. However, its reliance on manually crafted and tuned probabilistic rule sets, also called grammars, presents a significant bottleneck, diverging from approaches that allow rule learning from examples. In this paper, we propose a novel solution to this challenge by introducing a genetic programming-based optimization framework for learning hierarchical rule sets automatically. Our proposed method “Markov Senior” focuses on extracting positional and distance relations from single input samples to construct probabilistic rules to be used by Markov Junior. Using a Kullback-Leibler divergence-based fitness measure, we search for grammars to generate content that is coherent with the given sample. To enhance scalability, we introduce a divide-and-conquer strategy that enables the efficient generation of large-scale content We validate our approach through experiments in generating image-based content and Super Mario levels, demonstrating its flexibility and effectiveness. In this way, “Markov Senior” allows for the wider application of Markov Junior for tasks in which an example may be available, but the design of a generative rule set is infeasible.

Playing strategy games is a challenging problem for artificial intelligence (AI). One of the major challenges is the large search space due to a diverse set of game components. In recent works, state abstraction has been applied to search-based game AI and has brought significant performance improvements. State abstraction techniques rely on reducing the search space, e.g., by aggregating similar states. However, the application of these abstractions is hindered because the quality of an abstraction is difficult to evaluate. Previous works hence abandon the abstraction in the middle of the search to not bias the search to a local optimum. This mechanism introduces a hyper-parameter to decide the time to abandon the current state abstraction. In this work, we propose a size-constrained state abstraction (SCSA), an approach that limits the maximum number of nodes being grouped together. We found that with SCSA, the abstraction is not required to be abandoned. Our empirical results on 3 strategy games show that the SCSA agent outperforms the previous methods and yields robust performance over different games. Codes are opensourced at https://anonymous.4open.science/r/SCSA-DB44/.

One of the notorious issues for Reinforcement Learning (RL) is poor sample efficiency. Compared to single agent RL, the sample efficiency for Multi-Agent Reinforcement Learning (MARL) is more challenging because of its inherent partial observability, non-stationary training, and enormous strategy space. Although much effort has been devoted to developing new methods and enhancing sample efficiency, we look at the widely used episodic training mechanism. In each training step, tens of frames are collected, but only one gradient step is made. We argue that this episodic training could be a source of poor sample efficiency. To better exploit the data already collected, we propose to increase the frequency of the gradient updates per environment interaction (a.k.a. Replay Ratio or Update-To-Data ratio). To show its generality, we evaluate 3 MARL methods on 6 SMAC tasks. The empirical results validate that a higher replay ratio significantly improves the sample efficiency for MARL algorithms. The codes to reimplement the results presented in this paper are open-sourced at https://github.com/egg-west/rr_for_MARL.

This paper presents a genetic assessment agent and a student and machine co-learning model for high-school students' computational intelligence (CI) experience. We invited the IEEE CIS High School Outreach (HSO) subcommittee members of the years 2021–2022 to provide lectures at CIS activities and conferences and constructed a basic CI conceptual knowledge structure for high-school student learning. From 2021 to 2022 in Taiwan, we collected high-school students' learning data, including labels, attitudes, environment, and effort, from the CI&AI-FML platform using robots and learning tools, then processed the data using natural language processing (NLP) techniques to efficiently evaluate high-school students' learning state. We then applied three evolutionary computation techniques: genetic algorithm (GA), particle swarm optimization (PSO), and genetic algorithm neural network (GANN) in the proposed genetic assessment agent for the co-learning model, with learning performance regression analysis. In this paper, a CI&AI-FML human and machine co-learning Metaverse model is presented as a solution, which provides hands-on learning and experience while also supporting student-centered online learning during the COVID-19 pandemic. Students participated in the course during the 2022 Spring semester to learn basic CI concepts and experience CI applications through interaction with machines using the developed CI&AI- FML learning tools. The experimental results indicate that the genetic assessment agent with the GANN method has better performance in the student and machine co-learning model as compared to the other two methods, and it is effective for student and machine co-learning model construction.

Baba is You is a challenging puzzle game in which the player can modify the rules of the game. This yields a large variety of puzzles and an enormous state space to be searched through. Recently, the Feature Space Search algorithm has shown great results in Sokoban, which apart from the rule modification shares many similarities to Baba is You. It uses multiple heuristics to guide the search into promising regions of the search space. In this work, we are proposing a similar concept for solving Baba is You based on multiple heuristics that are aggregated for guiding a tree-based search process. However, finding parameters for weighting/prioritizing the different heuristics is a non-trivial task. This process is done, by applying evolutionary algorithms for single- and multi-objective optimization. Specifically, we compare the effects of these different optimization schemes on the agents' general level-solving capabilities. In all cases, the agent was able to adapt well to the training and test set with no significant differences among the optimization schemes. Compared with state-of-the-art Baba is You agents our search-based approach shows an improved performance in terms of the number of levels being solved, as well as a reduction in the average time required to solve a level.

Context steering is a local approach to control an agent’s movement in a dynamically changing scene. Recent works have formalized the context-steering approach by Fray and presented a multiobjective view of the context-steering problem. Combining a variety of different behaviors, which can be used multiple times in different configurations for different context maps, introduces a large number of parameters that need to be tuned to obtain well-performing agents. This work aims to use evolutionary algorithms to optimize context-steering agents for various environments. A special focus lies on the evolution of agents that perform robustly across multiple variations of the same environment. To this end, we develop a real-valued encoding for a context-steering agent along with three different fitness functions to represent different goals of the agent. Our experimental evaluation shows that an evolutionary optimization can produce agent configurations that perform well with respect to different tasks and show a high intratask robustness. The proposed approach based on evolutionary optimization enables the user to optimize context-steering agents such that they can explore environments while avoiding dynamic obstacles.

Strategy games are a challenge for the design of artificial intelligence agents due to their complexity and the combinatorial search space they produce. State abstraction has been applied in different domains to shrink the search space. Automatic state abstraction methods have gained much success in the planning domain and their transfer to strategy games raises a question of scalability. In this article, we propose elastic Monte Carlo tree search (MCTS), an algorithm that uses automatic state abstraction to play strategy games. In elastic MCTS, tree nodes are clustered dynamically. First, nodes are grouped by state abstraction for efficient exploration, to later be separated for refining exploitable action sequences. Such an elastic tree benefits from efficient information sharing while avoiding using an imperfect state abstraction during the whole search process. We provide empirical analyses of the proposed method in three strategy games of different complexity. Our empirical results show that in all games, elastic MCTS outperforms MCTS baselines by a large margin, with a considerable search tree size reduction at the expense of small computation time.

Steering and its many variants have been used to model multi-agent behavior in simulated worlds. Each variant of the steering algorithm introduces its own concepts and components. This work aims to generalize those in a modular building block design which allows to recreate existing steering algorithms and draft new versions. The concept is demonstrated in small-scale study on aggregation functions which shows the impact of a single building block on the agents’ behavior.

Strategy video games challenge AI agents with their combinatorial search space caused by complex game elements. State abstraction is a popular technique that reduces the state space complexity. However, current state abstraction methods for games depend on domain knowledge, making their application to new games expensive. State abstraction methods that require no domain knowledge are studied extensively in the planning domain. However, no evidence shows they scale well with the complexity of strategy games. In this paper, we propose Elastic MCTS, an algorithm that uses state abstraction to play strategy games. In Elastic MCTS, the nodes of the tree are clustered dynamically, first grouped together progressively by state abstraction, and then separated when an iteration threshold is reached. The elastic changes benefit from efficient searching brought by state abstraction but avoid the negative influence of using state abstraction for the whole search. To evaluate our method, we make use of the general strategy games platform Stratega to generate scenarios of varying complexity. Results show that Elastic MCTS outperforms MCTS baselines with a large margin, while reducing the tree size by a factor of 10. Code can be found at https://github.com/egg-west/Stratega

Portfolio methods represent a simple but efficient type of action abstraction which has shown to improve the performance of search-based agents in a range of strategy games. We first review existing portfolio techniques and propose a new algorithm for optimization and action-selection based on the Rolling Horizon Evolutionary Algorithm. Moreover, a series of variants are developed to solve problems in different aspects. We further analyze the performance of discussed agents in a general strategy game-playing task. For this purpose, we run experiments on three different game-modes of the Stratega framework. For the optimization of the agents' parameters and portfolio sets we study the use of the N-tuple Bandit Evolutionary Algorithm. The resulting portfolio sets suggest a high diversity in play-styles while being able to consistently beat the sample agents. An analysis of the agents' performance shows that the proposed algorithm generalizes well to all game-modes and is able to outperform other portfolio methods.

When implementing intelligent agents for strategy games, we observe that search-based methods struggle with the complexity of such games. To tackle this problem, we propose a new variant of Monte Carlo Tree Search which can incorporate action and game state abstractions. Focusing on the latter, we developed a game state encoding for turn-based strategy games that allows for a flexible abstraction. Using an optimization procedure, we optimize the agent's action and game state abstraction to maximize its performance against a rule-based agent. Furthermore, we compare different combinations of abstractions and their impact on the agent's performance based on the Kill the King game of the Stratega framework. Our results show that action abstractions have improved the performance of our agent considerably. Contrary, game state abstractions have not shown much impact. While these results may be limited to the tested game, they are in line with previous research on abstractions of simple Markov Decision Processes. The higher complexity of strategy games may require more intricate methods, such as hierarchical or time-based abstractions, to further improve the agent's performance.

This work concentrates on decision-making for autonomous movement of agents to simultaneously optimize several objectives which occur in their local environment. Such behavior can be achieved with steering algorithms, which have originally been designed for moving numerous agents simultaneously where occasional uncertainties are not noticeable by players. Nevertheless, concentrating on single individuals can reveal major flaws in their movement patterns such as oscillatory movement. For avoiding such problems, game makers are forced to develop higher-level abstractions for handling game-relevant special cases. Thus, eliminating the initial benefit of steering behaviors to be highly modular, lightweight, and controllable. This work enhances the context steering approach by Fray, which introduced discretized contextual information in the aggregation of a steering behavior's components. We combine this method with multi-criteria decision-making for controlling the agent's velocity direction and magnitude. The resulting approach is tested based on selected scenarios which show that the resulting approach is well suited to improve the agent's smooth and natural movement. Based on our observations we propose suitable parameterizations of the designed method and discuss advantages and disadvantages of made enhancements.

Designing agents that are able to achieve different play-styles while maintaining a competitive level of play is a difficult task, especially for games for which the research community has not found super-human performance yet, like strategy games. These require the AI to deal with large action spaces, long-term planning and partial observability, among other well-known factors that make decision-making a hard problem. On top of this, achieving distinct play-styles using a general algorithm without reducing playing strength is not trivial. In this paper, we propose Portfolio Monte Carlo Tree Search with Progressive Unpruning for playing a turn-based strategy game (Tribes) and show how it can be parameterized so a quality-diversity algorithm (MAP-Elites) is used to achieve different play-styles while keeping a competitive level of play. Our results show that this algorithm is capable of achieving these goals even for an extensive collection of game levels beyond those used for training.

This article provides an overview of the recently proposed forward model approximation framework for learning games of the general video game artificial intelligence (GVGAI) framework. In contrast to other general game-playing algorithms, the proposed agent model does not need a full description of the game but can learn the game's rules by observing game state transitions. Based on hierarchical knowledge bases, the forward model can be learned and revised during game-play, improving the accuracy of the agent's state predictions over time. This allows the application of simulation-based search algorithms and belief revision techniques to previously unknown settings. We show that the proposed framework is able to quickly learn a model for dynamic environments in the context of the GVGAI framework.

In this paper, we are going to explain the design process for our GVGAI game-learning agent, which is going to be submitted to the GVGAI competition's learning track 2020. The agent relies on a local forward modeling approach, which uses predictions of future game-states to allow the application of simulation-based search algorithms. We first explain our process in identifying repeating tiles throughout a pixel-based state observation. Using the tile information, a local forward model is trained to predict the future state of each tile based on its current state and its surrounding tiles. We accompany this approach with a simple reward model, which determines the expected reward of a predicted state transition. The proposed approach has been tested using multiple games of the GVGAI framework. Results show that the approach seems to be especially feasible for learning how to play deterministic games. Except for one non-deterministic game, the agent performance is very similar to agents using the true forward model. Nevertheless, the prediction accuracy needs to be further improved to facilitate a better game-playing performance.

This paper examines learning approaches for forward models based on local cell transition functions. We provide a formal definition of local forward models for which we propose two basic learning approaches. Our analysis is based on the game Sokoban, where a wrong action can lead to an unsolvable game state. Therefore, an accurate prediction of an action’s resulting state is necessary to avoid this scenario.In contrast to learning the complete state transition function, local forward models allow extracting multiple training examples from a single state transition. In this way, the Hash Set model, as well as the Decision Tree model, quickly learn to predict upcoming state transitions of both the training and the test set. Applying the model using a statistical forward planner showed that the best models can be used to satisfying degree even in cases in which the test levels have not yet been seen.Our evaluation includes an analysis of various local neighbourhood patterns and sizes to test the learners’ capabilities in case too few or too many attributes are extracted, of which the latter has shown do degrade the performance of the model learner.

This paper investigates the effect of learning a forward model on the performance of a statistical forward planning agent. We transform Conway's Game of Life simulation into a single-player game where the objective can be either to preserve as much life as possible or to extinguish all life as quickly as possible. In order to learn the forward model of the game, we formulate the problem in a novel way that learns the local cell transition function by creating a set of supervised training data and predicting the next state of each cell in the grid based on its current state and immediate neighbours. Using this method we are able to harvest sufficient data to learn perfect forward models by observing only a few complete state transitions, using either a look-up table, a decision tree, or a neural network. In contrast, learning the complete state transition function is a much harder task and our initial efforts to do this using deep convolutional auto-encoders were less successful.We also investigate the effects of imperfect learned models on prediction errors and game-playing performance, and show that even models with significant errors can provide good performance.