This study examines the impact of Large Language Model-generated narratives in a climate-change-themed Visual Novel, comparing two versions: First, the story is generated using thematic keywords in the prompts, and second, the story is generated without keywords. Fifty participants (21 female, 29 male) completed the study. Results showed that participants in the group without thematic keywords had higher levels of narrative engageability score, as measured by the Narrative Engageability Scale, than those with thematic keywords. This indicated that the ability to engage with the story was stronger in the group without keywords. However, when assessing the narrative experience using the Game User Experience Satisfaction Scale, both groups reported similar levels of satisfaction, suggesting that while the ability to engage with the narrative differed between groups, the overall narrative experience was mainly the same. These findings suggested that thematic keywords in prompts significantly impacted participants’ narrative experience of the game.

This study investigates the extent to which multimodal large language models (MLLMs) demonstrate physical reasoning capabilities in dynamic, visually grounded environments. We evaluate whether using images as context in a prompt can enhance the ability of MLLMs to simulate and predict the outcomes of physical interactions. Using Science Birds, an Angry Birds-like physicsbased platform, we design a suite of tasks that probe core competencies in binary, comparative stability, and forward simulation using visual and textual inputs. Our evaluation shows that MLLMs can perform visual reasoning tasks with quantifiable accuracy, especially when making predictions based on image-rich input. However, their performance varies considerably depending on the specific model architecture and the type of input modality used. These findings highlight that incorporating both visual and textual data is crucial for accurate physical inference. Nonetheless, the evaluated MLLMs still have substantial limitations in structured visual reasoning tasks. By systematically analyzing the strengths and weaknesses of different models, our work provides practical guidance for advancing MLLM-based physical reasoning and supports the development of future benchmarks and competitions in this area, including contributions to the LLMs4PCG challenge. We make our source code and raw data available for future research.11https://anonymous.4open.science/r/cog2025-game-physics-eval/.

This paper presents the second ChatGPT4PCG competition at the 2024 IEEE Conference on Games. In this edition of the competition, we follow the first edition, but make several improvements and changes. We introduce a new evaluation metric along with allowing a more flexible format for participants’ submissions and making several improvements to the evaluation pipeline. Continuing from the first edition, we aim to foster and explore the realm of prompt engineering (PE) for procedural content generation (PCG). While the first competition saw success, it was hindered by various limitations; we aim to mitigate these limitations in this edition. We introduce diversity as a new metric to discourage submissions aimed at producing repetitive structures. Furthermore, we allow submission of a Python program instead of a prompt text file for greater flexibility in implementing advanced PE approaches, which may require control flow, including conditions and iterations. We also make several improvements to the evaluation pipeline with a better classifier for similarity evaluation and better-performing function signatures. We thoroughly evaluate the effectiveness of the new metric and the improved classifier. Additionally, we perform an ablation study to select a function signature to instruct ChatGPT for level generation. Finally, we provide implementation examples of various PE techniques in Python and evaluate their preliminary performance. We hope this competition serves as a resource and platform for learning about PE and PCG in general1.1Source code and raw data: https://github.com/chatgpt4pcg/experiments2024

This paper investigates an interesting phenomenon where we observe performance increases in large language models (LLMs) when providing a prompt that causes and exploits hallucination.We propose null-shot prompting, a counter-intuitive approach where we deliberately instruct LLMs to reference a null, nonexistent, section.We evaluate null-shot prompting across a variety of tasks, including arithmetic reasoning, commonsense reasoning, and reading comprehension.Notably, we observe a substantial increase in performance in arithmetic reasoning tasks for various models, with up to a 44.62% increase compared to a baseline in one model.Additional experiments on more complex mathematical problem-solving and hallucination detection benchmarks also reveal similar benefits from this approach.Furthermore, we explore the effects of combining reasoning, which typically mitigates hallucination, with hallucination within the prompt and find several cases of performance improvements.We hope this paper stimulates further interest, investigation, and discussion on how hallucination in prompts may not only affect LLMs but, in certain cases, enhance their performance.

This study summarizes the first ChatGPT4PCG competition held at the 2023 IEEE Conference on Games. The goal of the competition is to explore emergent abilities of publicly available LLMs in performing complex tasks related to procedural content generation, specifically physics-based level generation for Angry Bird-like games. Participants are tasked with submitting their prompts for ChatGPT to generate Angry Birds-like game structures that resemble English uppercase characters. A structure is a collection of stacked game objects comprising a part of an entire Angry Birds-like level. A prompt is an input for large language models (LLMs) including ChatGPT. Two evaluation metrics, i.e., stability and similarity, are used to evaluate the submitted prompts. Stability measures the sturdiness of a structure to withstand in-game gravity, while similarity measures a structure's resemblance to the target character. With such evaluation, participants are challenged not only to produce character-like but also stable structures by utilizing prompt engineering techniques. Finally, the competition's results are discussed to provide valuable insights for future studies and competitions.

This paper presents the first ChatGPT4PCG Competition at the 2023 IEEE Conference on Games. The objective of this competition is for participants to create effective prompts for ChatGPT–enabling it to generate Science Birds levels with high stability and character-like qualities–fully using their creativity as well as prompt engineering skills. ChatGPT is a conversational agent developed by OpenAI. Science Birds is selected as the competition platform because designing an Angry Birds-like level is not a trivial task due to the in-game gravity; the quality of the levels is determined by their stability. To lower the entry barrier to the competition, we limit the task to the generation of capitalized English alphabetical characters. We also allow only a single prompt to be used for generating all the characters. Here, the quality of the generated levels is determined by their stability and similarity to the given characters. A sample prompt is provided to participants for their reference. An experiment is conducted to determine the effectiveness of several modified versions of this sample prompt on level stability and similarity by testing them on several characters. To the best of our knowledge, we believe that ChatGPT4PCG is the first competition of its kind and hope to inspire enthusiasm for prompt engineering in procedural content generation.

This demo paper presents a smile interface to promote the spectator’s emotion. Here, a smile interface is defined as a human-computer interaction system that detects the user’s smile. A past study suggested that watching Science Birds gameplay featuring Rube Goldberg Machine mechanisms with a domino effect can promote the spectator’s emotion. Science Birds is a clone version of Angry Birds for research purposes. Furthermore, other studies suggested that playing Science Birds with a smile interface has the benefit of promoting the player’s emotion. However, such benefit from the spectator’s perspective is yet to be investigated. The presented interface will be used in such investigation in the future.

This paper proposes a method to generate interesting Angry Birds-like game levels featuring the Rube Goldberg machine (RGM) mechanism using Constrained Novelty Search (CNS). An RGM level in Angry Birds emphasizes a domino effect, which allows it to be completed by only one bird shooting. By evolving the feasible population and infeasible population in CNS, our results show that the entropy of block-type frequencies is higher than the entropy by our previous generator and that two requirements to achieve playable RGM levels - the 100% stability and the perfect-shot rate - are met. The results indicate that the proposed method can generate levels with more diversity than our previous generator while maintaining their playability.

This paper presents a game system aimed towards promoting mental health, which implements singing as an input modality to control the bird-shots in an Angry-Birds-like game. Based on existing research, singing can act as a catalyst that medically reduces stress and thus can be used for rehabilitation. Our system works by analyzing singing by the player and adjusting the shooting performance based on the singing score. This paper focuses on how the system is developed and its functions.

This study proposes a method for generating Angry Birds-like game levels featuring a domino effect generated based on Rube Goldberg Machine (RGM) mechanisms, which allow them to be completed by one shot of a bird. The proposed method generates a level by selecting predefined segments consisting of several objects arranged in a way that creates a domino effect among them. To increase the variability of generated levels, the proposed method procedurally generates a varying structure on the top of certain blocks in a predefined segment. Our results show that the proposed RGM generator is comparable to two existing generators, including the winner of the 2018 AIBIRDS Level Generation Competition, in terms of stability while it outperforms both baseline generators with respect to running time and an expressivity metric called "dynamic" which is introduced in this work to measure the time period where moving objects, including a shooting bird, reside in a given level. In addition, from the perspective on the destructive power of a shot, the proposed generator can generate levels featuring a successful domino effect with a high probability, in particular for levels with three to four segments.

This paper investigates to which degrees of playing games by using hand motions improve brain memory. Based on previous studies in psychology reporting that hand exercise could affect brain function and memory, we develop an interface for playing games by using hand motions. It works by detecting hand motions through image processing, translating them into commands for keyboard press, and sending such commands to the target game application; it works with any existing games without the need to modify the game source code and only requires an off-the-self webcam. An experiment is conducted on a jumping game, where three types of game interfaces: a keyboard, a Kinect, and the proposed interface are compared. The results show that playing games using hand clenching/spreading leads to the best memory test performance according to the N-back test, a commonly used cognitive task for the assessment of working memory.