This study examines the impact of Large Language Model-generated narratives in a climate-change-themed Visual Novel, comparing two versions: First, the story is generated using thematic keywords in the prompts, and second, the story is generated without keywords. Fifty participants (21 female, 29 male) completed the study. Results showed that participants in the group without thematic keywords had higher levels of narrative engageability score, as measured by the Narrative Engageability Scale, than those with thematic keywords. This indicated that the ability to engage with the story was stronger in the group without keywords. However, when assessing the narrative experience using the Game User Experience Satisfaction Scale, both groups reported similar levels of satisfaction, suggesting that while the ability to engage with the narrative differed between groups, the overall narrative experience was mainly the same. These findings suggested that thematic keywords in prompts significantly impacted participants’ narrative experience of the game.

This study investigates the emotional dynamics from different cultural backgrounds using a multimodal approach that combines Facial Emotion Recognition and Heart Rate Variability (HRV) analysis. A total of 109 participants from Italy, Japan, and Korea (mean age = 24.5 years) played two casual games, namely Snake and Matching Pairs, to investigate cultural differences in emotional and physiological responses. The results revealed distinct cultural patterns in static emotional expression using generalised linear mixed models (GLMMs). The Italian cultural group showed higher levels of positive facial expressions (FE), particularly happiness; the Korean cultural group showed more frequent negative FE, while the Japanese cultural group showed restrained FE, particularly related to fear. Moreover, emotional transitions were analysed using a Markov-inspired continuousstate operator derived from probabilistic FE vectors, which characterised the temporal structure of emotional changes and uncovered systematic cross-cultural and task-dependent differences in emotional latency. These findings show that emotional transitions are shaped by cultural norms and the cognitive demands of the games. Furthermore, integrating FE and HRV features into GLMMs showed that autonomic indices predict performance and vary across game types. Overall, this study provides three key contributions. First, it indicates that FEs of emotion during gameplay differ significantly across cultures, in accordance with cultural display norms. Second, it demonstrates that emotional transitions are dynamic and influenced by game performance, with cultural background shaping these patterns. Third, it identifies cross-cultural differences in physiological responses, specifically bodily signals such as HRV. These findings enhance understanding of how games elicit and regulate emotional and physiological responses, suggesting applications beyond entertainment.

This study investigates the extent to which multimodal large language models (MLLMs) demonstrate physical reasoning capabilities in dynamic, visually grounded environments. We evaluate whether using images as context in a prompt can enhance the ability of MLLMs to simulate and predict the outcomes of physical interactions. Using Science Birds, an Angry Birds-like physicsbased platform, we design a suite of tasks that probe core competencies in binary, comparative stability, and forward simulation using visual and textual inputs. Our evaluation shows that MLLMs can perform visual reasoning tasks with quantifiable accuracy, especially when making predictions based on image-rich input. However, their performance varies considerably depending on the specific model architecture and the type of input modality used. These findings highlight that incorporating both visual and textual data is crucial for accurate physical inference. Nonetheless, the evaluated MLLMs still have substantial limitations in structured visual reasoning tasks. By systematically analyzing the strengths and weaknesses of different models, our work provides practical guidance for advancing MLLM-based physical reasoning and supports the development of future benchmarks and competitions in this area, including contributions to the LLMs4PCG challenge. We make our source code and raw data available for future research.11https://anonymous.4open.science/r/cog2025-game-physics-eval/.

This paper presents a genetic search-based approach to automatic prompt optimization for the 2025 LLMs4PCG competition. Large language models can enhance the longevity and replayability of games through procedural content generation, but reliance on proprietary third-party APIs raises concerns about cost and sustainability. Open-weight models return control to developers, though often with reduced capabilities. We propose a genetic algorithm that automatically optimizes prompts to help close this performance gap on certain tasks. Experimental results and competition benchmarks suggest this is a promising step toward enabling practical use of smaller, open-weight models for PCG in games.

This paper presents a multi-year evaluation (2022-2024) of winning entries from the DareFightingICE Sound Design Competition, analyzing the interplay between audio aesthetics and gameplay functionality for visually impaired players and the Blind AI agent. Through user studies ($\mathrm{n} {=} {2 6}$and$\mathrm{n} {=} {2 1}$) and a comprehensive ablation study of the 2024 winning sound design, we reveal three key findings: (1) While winning entries consistently achieved high aesthetic ratings, scores plateaued despite year-on-year improvements in Blind AI performance, indicating a decoupling of these dimensions; (2) Systematic muting identified “critical” sounds whose absence severely impaired AI decision-making, whereas static background music introduced detrimental noise; (3) Drastically reducing sound design to${1 0 \%}$of sound effects degraded both AI win ratios and human aesthetic perception. These results demonstrate that practical accessible audio requires prioritizing unambiguous cues for frequent actions, minimizing sonic redundancy like non-adaptive BGM, and validating sound designs through dual AI/human evaluation. We provide actionable strategies for designers targeting future competitions and inclusive gaming.

This paper introduces Sonic Doom, an accessibility focused enhancement of the ViZDoom First-person shooter (FPS) platform, integrating an advanced sound design and two aim-assist systems: Auto Aim (automated crosshair adjustment) and Sonic Aim (audio feedback for target proximity). To tackle the issue of FPS games remaining largely inaccessible to visually impaired players (VIPs) due to the game's reliance on visual cues for navigation and combat. We evaluate our approach through a dual-method framework: (1) AI agents trained to play blindly using only an audio input, and (2) human participants in both sighted and blindfolded conditions. Results demonstrate that enhanced sound design improves navigation efficiency (e.g., faster maze completion by the AI agent) and combat accuracy (e.g., higher enemy kill rates in human trials). While Auto Aim achieved superior objective performance, subjective evaluation revealed a strong user preference for Sonic Aim, emphasizing the need to balance assistance with player autonomy. Our AI-driven evaluation framework, the first of its kind in FPS accessibility research, provides scalable, objective metrics for assessing sound designs. This work advances accessible game design by empirically validating sonification techniques in combat-intensive scenarios and establishing a methodology for future research in multi-modal game accessibility.

In this article, we present BenchING, a new benchmark for evaluating large language models (LLMs) on their ability to follow structured output format instructions in text-based procedural content generation (PCG) tasks. The ability to condition LLMs to output in specified formats proves useful, as downstream components in LLM-integrated games often require structured outputs for exchanging information. However, there is a gap in evaluating this aspect of LLMs, especially in narrative PCG tasks, making it difficult to select LLMs and design games or applications integrating these LLMs. To demonstrate the potential of our benchmark, we evaluate nine LLMs for their ability to generate parseable formatted outputs using five selected text-based PCG tasks. We report on the performance of these LLMs on these tasks. In addition, we categorize more detailed error types and propose solutions by utilizing LLMs to fix these errors. We also conduct a scaling study, investigating an emergent point of LLMs for their ability to fix malformed formatted content using eight quantized LLMs with varying original sizes from 0.62 to 72.3 B. Furthermore, we perform a qualitative study to assess the quality of the generated content. We make our source code and raw data available for future research.

Biofeedback in serious games is becoming increasingly relevant to objectively assessing players’ engagement and performance. This study administered a Matching Pairs (MP) game to a group of healthy volunteers while acquiring eye-tracking data, specifically pupil dilation and blinking behavior. A dedicated algorithm has been implemented for game score assessment. A set of linear and nonlinear features were extracted from physiological signals. Statistical analysis was performed to understand whether oculometric parameters differ between the best and worst MP game trials. Moreover, correlation analysis investigated possible relationships between physiological measures and players’ performance. Results showed statistically significant smaller pupil dilation velocity, higher Index of Pupillary Activity (IPA), and shorter blink rate in the best MP trial than in the worst one. Our outcomes could highlight better cognitive resource management and greater focus in the best trial. Moreover, participants’ scores were negatively correlated with blinking rate and the time the eyes were closed during the game. It showed that more focus on specific game tasks leads to better performance, therefore limiting interruptions of the information flow due to blinking. These findings may suggest that eye parameters in serious gaming platforms could be a powerful tool for intervention programs targeting older populations or people with cognitive impairments.

This paper presents the second ChatGPT4PCG competition at the 2024 IEEE Conference on Games. In this edition of the competition, we follow the first edition, but make several improvements and changes. We introduce a new evaluation metric along with allowing a more flexible format for participants’ submissions and making several improvements to the evaluation pipeline. Continuing from the first edition, we aim to foster and explore the realm of prompt engineering (PE) for procedural content generation (PCG). While the first competition saw success, it was hindered by various limitations; we aim to mitigate these limitations in this edition. We introduce diversity as a new metric to discourage submissions aimed at producing repetitive structures. Furthermore, we allow submission of a Python program instead of a prompt text file for greater flexibility in implementing advanced PE approaches, which may require control flow, including conditions and iterations. We also make several improvements to the evaluation pipeline with a better classifier for similarity evaluation and better-performing function signatures. We thoroughly evaluate the effectiveness of the new metric and the improved classifier. Additionally, we perform an ablation study to select a function signature to instruct ChatGPT for level generation. Finally, we provide implementation examples of various PE techniques in Python and evaluate their preliminary performance. We hope this competition serves as a resource and platform for learning about PE and PCG in general1.1Source code and raw data: https://github.com/chatgpt4pcg/experiments2024

This paper investigates how prompt compression, a technique to reduce the number of tokens in the prompt while maintaining prompt performance, affects inherent biases in large language models (LLMs) for the story ending of the game story generation task. Previous studies have explored inherent biases in LLMs and found an innate inclination of LLMs towards generating positive-ending stories. While prompt compression is known to retain task performance and utilize fewer tokens in the prompt, we explore a different perspective on how prompt compression could affect inherent biases in LLMs. We follow existing studies’ approach in evaluating story ending biases of six LLMs comparing uncompressed and compressed prompts. We find that prompt compression does not affect story generation from positive-ending story synopses, to which these LLMs are inclined. However, it is not the same for negative-ending story synopses: prompt compression either makes the LLMs generate a higher amount of negative-ending stories or not at all. We also notice that the classification of other types of story endings, other than those specified in the prompt, aligns with an existing study. We recommend game developers and future studies to always perform empirical tests on prompt compression, as it is not straightforward and may greatly alter model behaviors.

This paper presents an initial step towards general evaluations of open-weight large language models (LLMs) using the ChatGPT4PCG platform, a Science Birds level generation challenge designed to evaluate LLMs on the complex task of generating stable, English-character-resembling, and diverse levels. While ChatGPT4PCG competitions have their own merit in providing a comprehensive platform for evaluating ChatGPT on complex tasks, the competitions focus solely on ChatGPT is rather limiting considering the fact that there are many available choices of open-weight LLMs. We report 13 LLMs from five model families of various properties in their design choices and sizes to evaluate on a modified ChatGPT4PCG 2 competition platform. We observe that the scaling law holds in general, but the inherent capabilities of LLMs due to their pre-training and architecture choices also play an equal role. We open-source the modification of the ChatGPT4PCG platform to support future research on evaluating LLMs in this area1.1https://github.com/Pittawat2542/llm4pcg-python and https://github.com/Pittawat2542/llm4pcg-experiment

This paper investigates an interesting phenomenon where we observe performance increases in large language models (LLMs) when providing a prompt that causes and exploits hallucination.We propose null-shot prompting, a counter-intuitive approach where we deliberately instruct LLMs to reference a null, nonexistent, section.We evaluate null-shot prompting across a variety of tasks, including arithmetic reasoning, commonsense reasoning, and reading comprehension.Notably, we observe a substantial increase in performance in arithmetic reasoning tasks for various models, with up to a 44.62% increase compared to a baseline in one model.Additional experiments on more complex mathematical problem-solving and hallucination detection benchmarks also reveal similar benefits from this approach.Furthermore, we explore the effects of combining reasoning, which typically mitigates hallucination, with hallucination within the prompt and find several cases of performance improvements.We hope this paper stimulates further interest, investigation, and discussion on how hallucination in prompts may not only affect LLMs but, in certain cases, enhance their performance.

In this paper, we present a Tabletop Role-Playing game (TTRPG) driven by ChatGPT. Prompts are employed to instruct ChatGPT to act as Game Masters (GMs). In crafting each prompt to integrate a distinctive role, three roles denoted as Role 1, Role 2, and Role 3, are established. Subsequently, we perform pre-game and post-game emotional assessments employing the Positive and Negative Affect Schedule (PANAS) questionnaire to scrutinize players’ emotional dynamics throughout the gaming experience. Upon analyzing the collected data, we observe that Role 1 and Role 2 affect players’ positive emotions. Notably, Role 2 exhibits the most pronounced influence on players’ positive emotions. Our findings demonstrate that a TTRPG GM powered by ChatGPT can significantly enhance players’ positive emotions. This leads us to recognize that TTRPG GM powered by ChatGPT plays a positive role in enhancing the mental well-being of specific populations.

This paper presents a cost-effective approach to utilizing large language models (LLMs) as part of advanced driver-assistance systems (ADAS) through a knowledge-distilled model for driving assessment. LLMs have recently been employed across various domains. However, due to their size, they require sufficient computing infrastructure for deployment and ample time for generation. These characteristics make LLMs challenging to integrate into applications requiring real-time feedback, including ADAS. An existing study employed a vector database containing responses generated from an LLM to act as a surrogate model. However, this approach is limited when handling out-of-distribution (OOD) scenarios, which LLMs excel at. We propose a novel approach that utilizes a distilled model obtained from an established knowledge distillation technique to perform as a surrogate model for a target LLM, offering high resilience in handling OOD situations with substantially faster inference time. To assess the performance of the proposed approach, we also introduce a new dataset for driving scenarios and situations (DriveSSD), containing 124,248 records. Additionally, we augment randomly selected 12,425 records, 10% of our DriveSSD, with text embeddings generated from an embedding model. We distill the model using 10,000 augmented records and test all approaches on the remaining 2,425 records. We find that the distilled model introduced in this study has better performance across metrics, with half of the inference time used by the previous approach. We make our source code and data publicly available1.

This study summarizes the first ChatGPT4PCG competition held at the 2023 IEEE Conference on Games. The goal of the competition is to explore emergent abilities of publicly available LLMs in performing complex tasks related to procedural content generation, specifically physics-based level generation for Angry Bird-like games. Participants are tasked with submitting their prompts for ChatGPT to generate Angry Birds-like game structures that resemble English uppercase characters. A structure is a collection of stacked game objects comprising a part of an entire Angry Birds-like level. A prompt is an input for large language models (LLMs) including ChatGPT. Two evaluation metrics, i.e., stability and similarity, are used to evaluate the submitted prompts. Stability measures the sturdiness of a structure to withstand in-game gravity, while similarity measures a structure's resemblance to the target character. With such evaluation, participants are challenged not only to produce character-like but also stable structures by utilizing prompt engineering techniques. Finally, the competition's results are discussed to provide valuable insights for future studies and competitions.

This paper presents our work to enhance the background music (BGM) in DareFightingICE by adding adaptive features. The adaptive BGM consists of three different categories of instruments playing the BGM of the winner sound design from the 2022 DareFightingICE Competition. The BGM adapts by changing the volume of each category of instruments. Each category is connected to a different element of the game. We then run experiments to evaluate the adaptive BGM by using a deep reinforcement learning AI agent that only uses audio as input (Blind DL AI). The results show that the performance of the Blind DL AI improves while playing with the adaptive BGM as compared to playing without the adaptive BGM.

This demo paper presents a deep reinforcement learning blind agent (blind agent) that uses only sound as the input on the DareFightingICE platform at the DareFightingICE Competition in IEEE CoG 2022 and 2023. A past study used reinforcement learning to train a blind agent and used it as the sample for the AI track of DareFightingICE Competition. However, this approach suffered from significant time expenses due to training the blind agent to recognize sounds and act in an end-to-end manner. To address this challenge, we propose a novel approach for blind agents where sound recognition and decision-making tasks are trained separately.

This paper proposes a delay mechanism to mitigate the impact of latency differences in the gRPC framework—a high-performance, open-source universal remote procedure call (RPC) framework—between different programming languages on the performance of agents in DareFightingICE, a fighting-game research platform. The study finds that gRPC latency differences between Java and Python can significantly impact real-time decision-making. Without a delay mechanism, Java-based agents outperform Python-based ones due to lower gRPC latency on the Java platform. However, with the proposed delay mechanism, both Java-based and Python-based agents exhibit similar performance, leading to a fair comparison between agents developed using different programming languages. Thus, this work underscores the crucial importance of considering gRPC latency when developing and evaluating agents in DareFighting-ICE, and the insights gained could potentially extend to other gRPC-based applications.

This paper presents the first ChatGPT4PCG Competition at the 2023 IEEE Conference on Games. The objective of this competition is for participants to create effective prompts for ChatGPT–enabling it to generate Science Birds levels with high stability and character-like qualities–fully using their creativity as well as prompt engineering skills. ChatGPT is a conversational agent developed by OpenAI. Science Birds is selected as the competition platform because designing an Angry Birds-like level is not a trivial task due to the in-game gravity; the quality of the levels is determined by their stability. To lower the entry barrier to the competition, we limit the task to the generation of capitalized English alphabetical characters. We also allow only a single prompt to be used for generating all the characters. Here, the quality of the generated levels is determined by their stability and similarity to the given characters. A sample prompt is provided to participants for their reference. An experiment is conducted to determine the effectiveness of several modified versions of this sample prompt on level stability and similarity by testing them on several characters. To the best of our knowledge, we believe that ChatGPT4PCG is the first competition of its kind and hope to inspire enthusiasm for prompt engineering in procedural content generation.

This demo paper presents a smile interface to promote the spectator’s emotion. Here, a smile interface is defined as a human-computer interaction system that detects the user’s smile. A past study suggested that watching Science Birds gameplay featuring Rube Goldberg Machine mechanisms with a domino effect can promote the spectator’s emotion. Science Birds is a clone version of Angry Birds for research purposes. Furthermore, other studies suggested that playing Science Birds with a smile interface has the benefit of promoting the player’s emotion. However, such benefit from the spectator’s perspective is yet to be investigated. The presented interface will be used in such investigation in the future.

This study implements a virtual character as a moderator in JUSTIN, a game designed and developed for collecting descriptions of ukiyo-e artworks on a live-streaming platform. The game has shown to be effective. However, the repetitive nature and the necessity to play JUSTIN many rounds make its players less interested in continuing the game. Hence, we examine if a virtual character can improve the player’s enjoyment and engagement experience. To conduct a control experiment, we develop a prototype that simulates JUSTIN, but with only one player playing the game at a time, and run an experiment with it. Our preliminary results show that a virtual character that changes its expression according to the game situation is promising in promoting the enjoyment and engagement of JUSTIN players.

This paper presents a new competition-at the 2022 IEEE Conference on Games (CoG)- called DareFightingICE Competition. The competition has two tracks: a sound design track and an AI track. The game platform for this competition is also called DareFightingICE, a fighting game platform. DareFightingICE is a sound-design-enhanced version of FightingICE, used earlier in a competition at CoG until 2021 to promote artificial intelligence (AI) research in fighting games. In the sound design track, participants compete for the best sound design, given the default sound design of DareFightingICE as a sample, where we define a sound design as a set of sound effects combined with the source code that implements their timing-control algorithm. Participants of the AI track are asked to develop their AI algorithm that controls a character given only sound as the input (blind AI) to fight against their opponent; a sample deep-learning blind AI will be provided by us. Our means to maximize the synergy between the two tracks are also described. This competition serves to come up with effective sound designs for visually impaired players, a group in the gaming community which has been mostly ignored. To the best of our knowledge, DareFightingICE Competition is the first of its kind within and outside of CoG.

This paper presents a deep reinforcement learning agent (AI) that uses sound as the input on the DareFightingICE platform at the DareFightingICE Competition in IEEE CoG 2022. In this work, an AI that only uses sound as the input is called blind AI. While state-of-the-art AIs rely mostly on visual or structured observations provided by their environments, learning to play games from only sound is still new and thus challenging. We propose different approaches to process audio data and use the Proximal Policy optimization algorithm for our blind AI. We also propose to use our blind AI in evaluation of sound designs submitted to the competition and define two metrics for this task. The experimental results show the effectiveness of not only our blind AI but also the proposed two metrics.

The GDMC AI settlement generation challenge is a procedural content generation (PCG) competition about producing an algorithm that can create a settlement in the game Minecraft. In contrast to the majority of AI competitions, the GDMC entries are evaluated by human experts on several criteria such as adaptability, functionality, evocative narrative, and visual aesthetics – all of which represent challenges to state-of-the-art PCG systems. This paper contains a collection of written experiences with this competition, by participants, judges, organizers and advisors. We asked people to reflect both on the artifacts themselves, and on the competition in general. The aim of this paper is to offer a shareable and edited collection of experiences and qualitative feedback which have the potential to push forward PCG and computational creativity, but would be lost once the individual assessments are compressed to scalar ratings. We reflect upon organizational issues for AI competitions, and discuss the future of the GDMC competition.

Abstract This paper investigates the integration of a surrogate-assisted multi-objective evolutionary algorithm (MOEA) and a parallel computation scheme to reduce the computing time until obtaining the optimal solutions in evolutionary algorithms (EAs). A surrogate-assisted MOEA solves multi-objective optimization problems while estimating the evaluation of solutions with a surrogate function. A surrogate function is produced by a machine learning model. This paper uses an extreme learning surrogate-assisted MOEA/D (ELMOEA/D), which utilizes one of the well-known MOEA algorithms, MOEA/D, and a machine learning technique, extreme learning machine (ELM). A parallelization of MOEA, on the other hand, evaluates solutions in parallel on multiple computing nodes to accelerate the optimization process. We consider a synchronous and an asynchronous parallel MOEA as a master-slave parallelization scheme for ELMOEA/D. We carry out an experiment with multi-objective optimization problems to compare the synchronous parallel ELMOEA/D with the asynchronous parallel ELMOEA/D. In the experiment, we simulate two settings of the evaluation time of solutions. One determines the evaluation time of solutions by the normal distribution with different variances. On the other hand, another evaluation time correlates to the objective function value. We compare the quality of solutions obtained by the parallel ELMOEA/D variants within a particular computing time. The experimental results show that the parallelization of ELMOEA/D significantly reduces the computational time. In addition, the integration of ELMOEA/D with the asynchronous parallelization scheme obtains higher quality of solutions quicker than the synchronous parallel ELMOEA/D.

In this article, we propose a fighting-game artificial intelligence (AI) that selects its actions from the perspective of highlight generation using Monte Carlo tree search with three highlight cues in the evaluation function. The intended use of this proposed AI is to generate gameplay in live streaming platforms such as Twitch and YouTube where a large number of spectators watch gameplay to entertain themselves. A gameplay analysis and user study are conducted using FightingICE, a fighting-game platform in an international fighting-game AI competition. Results show that gameplay generated by two AI players of the proposed method has more promising characteristics than not only gameplay generated according to an existing method but also gameplay by the top two AI players in the 2019 competition and that it is more entertaining than the latter gameplay.

In the last decade, multi-objective evolutionary algorithms (MOEAs) have been utilized for many real-world applications. However, it takes a great deal of computation time for the majority of real-world problems to obtain the optimal solutions due to the expensive fitness evaluation cost. In order to reduce the computation time for optimization, surrogate-assisted MOEAs have been studied. Our previous study analyzed ELMOEA/D, one of the surrogate-assisted MOEA combining MOEA/D with an extreme learning machine (ELM), from the relation between search performance and search generations. Our previous analysis revealed that the search generations on the surrogate space must be determined to make the accuracy of the surrogate model low. For this fact, this paper proposes the automatic adjustment methods for the search generations of ELMOEA/D. We conduct experiments with several well-known multi-objective benchmark problems and compare the proposed methods with the conventional ELMOEA/D with the fixed number of generations. The experimental results reveal that the proposed methods achieve a more stable search performance than ELMOEA/D with the fixed number of generations regardless of the target problems.

This paper proposes a method to improve the CPU utilization of parallel differential evolution (PDE) by incorporating the interleaving generation mechanism. Previous research proposed the interleaving generation evolutionary algorithm (IGEA) and its improved variants (iIGEA). IGEA reduces the computation time by generating new offspring, which parents have been determined even when all individuals have not evaluated. However, the previous research only used a simple EA method, which is not suitable for practical use. For this issue, this paper explores the applicability of IGEA and iIGEA to practical EA methods. In particular, we choose differential evolution (DE), which is widely used in real-world applications, and propose IGDE and its improved variant, iIGDE. We conduct experiments to investigate the effectiveness of IGDE with several features of the evaluation time on a simulated parallel computing environment. The experimental results reveal that the IGDE variants have higher CPU utilization than a simple PDE and reduce the computation time required for optimization. Besides, iIGDE outperforms the original IGDE for all features of the evaluation time.

Multimodal program optimizations (MMPOs) have been studied in recent years. MMPOs aims at obtaining multiple optimal programs with different structures simultaneously. This paper proposes novel MMPO benchmark problems to evaluate the performance of the multimodal program search algorithms. In particular, we propose five MMPOs, which have different characteristics, the similarity between optimal programs, the complexity of optimal programs, and the number of local optimal programs. We apply multimodal genetic programming (MMGP) proposed in our previous work to the proposed MMPOs to verify their difficulty and effectiveness, and evaluate the performance of MMGP. The experimental results reveal that the proposed MMPOs are difficult and complex to obtain the global and local optimal programs simultaneously as compared to the conventional benchmark. In addition, the experimental results clarify mechanisms to improve the performance of MMGP.

This paper proposes a method to generate interesting Angry Birds-like game levels featuring the Rube Goldberg machine (RGM) mechanism using Constrained Novelty Search (CNS). An RGM level in Angry Birds emphasizes a domino effect, which allows it to be completed by only one bird shooting. By evolving the feasible population and infeasible population in CNS, our results show that the entropy of block-type frequencies is higher than the entropy by our previous generator and that two requirements to achieve playable RGM levels - the 100% stability and the perfect-shot rate - are met. The results indicate that the proposed method can generate levels with more diversity than our previous generator while maintaining their playability.

This paper presents a game system aimed towards promoting mental health, which implements singing as an input modality to control the bird-shots in an Angry-Birds-like game. Based on existing research, singing can act as a catalyst that medically reduces stress and thus can be used for rehabilitation. Our system works by analyzing singing by the player and adjusting the shooting performance based on the singing score. This paper focuses on how the system is developed and its functions.

This paper presents a system for rap-style comment generation for enhancing video game live streaming. As creative storytelling can make live streaming truely entertaining, we investigate music-style comment generation in a video game. Regarding that "rap" is a popular genre of music, which has also been claimed success in promoting the enjoyment and contributing to rehabilitation, we propose a system that generates rap-style comments in real-time for promoting audience experience.

This paper presents JUSTIN standing for Japanese Ukiyo-e Streaming That Improves Narrative, a game system that is designed for collecting descriptive data for artwork images (Japanese ukiyo-e in this study). The proposed game is the first "Audience Participation Game With a Purpose (APGWAP)," which combines two existing concepts: Audience Participation Game and Game With A Purpose. The APGWAP concept is aimed at addressing problems that computers can hardly solve by utilizing human participation on a live streaming platform. The descriptive data of ukiyo-e images can be beneficial in humanity research and in promoting cultural heritage. However, such data are currently insufficient, sparse or poor in quality. JUSTIN proves to be a powerful tool in exploiting crowd-sourcing for addressing such problems through solid evidence provided from the results of conducted experiments in this work in terms of the quality of collected descriptions as well as player experience.

This paper presents two AIs that enable a fighting game to be played or live-streamed as an audience participation game. The proposed fighting game AIs imitates a social facilitation in human psychology by dynamically adjusting its strength based on audience responses during gameplay. The two AIs exploit Monte-Carlo Tree Search. They have three important mechanisms, for dynamic difficulty adjustment, human-like behavior promotion, and social facilitation integration. We developed the two AIs with different evaluation functions. The common feature is an integrated social facilitation parameter. However, the difference is that one AI is developed by generalizing an existing AI by simply adding a parameter for setting the targeted HP difference, whereas the other was particularly designed for an social facilitation by a strict control of damage score difference. Our experiment result shows that the former one yields more human-like behavior, while the latter one yields slightly better strength adjustment.

This paper presents a video game system with a smile interface designed to promote the player experience. We introduce a new mechanism to an existing endless runner game, "Runner." Such mechanism allows smiles to be taken as an input to the game system, for triggering boosting period. To detect smiles, a deep-learning facial recognition toolkit named Affdex, by Affectiva, is employed. Evaluation on player experience was done using a shorten version of Game User Experience Satisfaction Scale. Three different game modes are compared: Standard Mode (no smile and boosting), Smile Mode (with smile for boosting), and Auto Mode (auto boosting, without smile). Our results show that the presence of the smile mechanism leads to more enjoyable gameplay.

The eight papers in this special section focus on serious game computer applications for the health care field. During this period, we witnessed the creation of several conferences, providing a forum to discuss and share knowledge, experiences, and scientific and technical results, related to state-of-the-art solutions and technologies on serious games and applications for health and healthcare. The growth of computational capacity, development of new forms of interaction with the user (patient), and theoretical foundations produced by the scientific community over the past few years have now made it possible and viable to develop solutions, based on serious games, to face real problems. The possibility to create scenarios for medical training and simulation, the use of video games in rehabilitation procedures in an extra hospital environment, and the engagement provided by forms of teaching based on video games are just some examples of how this technology can be used. This special issue aims to highlight some of the high-quality research produced in the field of serious games applied to health.

This study proposes a method for generating Angry Birds-like game levels featuring a domino effect generated based on Rube Goldberg Machine (RGM) mechanisms, which allow them to be completed by one shot of a bird. The proposed method generates a level by selecting predefined segments consisting of several objects arranged in a way that creates a domino effect among them. To increase the variability of generated levels, the proposed method procedurally generates a varying structure on the top of certain blocks in a predefined segment. Our results show that the proposed RGM generator is comparable to two existing generators, including the winner of the 2018 AIBIRDS Level Generation Competition, in terms of stability while it outperforms both baseline generators with respect to running time and an expressivity metric called "dynamic" which is introduced in this work to measure the time period where moving objects, including a shooting bird, reside in a given level. In addition, from the perspective on the destructive power of a shot, the proposed generator can generate levels featuring a successful domino effect with a high probability, in particular for levels with three to four segments.

In this paper, we propose a fighting game AI that selects its actions from the perspective of highlight generation using Monte-Carlo tree search (MCTS) with three highlight cues in the evaluation function. The proposed AI is targeted for being used to generate gameplay in live streaming platforms such as Twitch and YouTube where a large number of spectators watch gameplay to entertain themselves. Our results in a user study conducted using FightingICE, a fighting game platform used in an international game AI competition since 2013, show that gameplay generated by the proposed AI is more entertaining than that by a typical MCTS AI. Detailed analyses of gameplay from all the methods assessed in the user study are also given in the paper.

This paper proposes a motion gaming AI that encourages players to use their body parts in a well-balanced manner while promoting their player experience. The proposed AI is an enhanced version of our previous AI in two aspects. First, it uses time series forecasting to more precisely predict what actions the player will perform with respect to its candidate actions, based on which the amount of movement to be produced on each body part of the player against each of such candidates is derived; as in our previous work. the AI selects its action from those candidates with a goal of making the player’s movement of their body parts on both sides equal. Second, this AI employs Monte-Carlo tree search that finds candidate actions according to dynamic difficulty adjustment. Our results show that the proposed game AI outperforms our previous AI in terms of the player’s body-movement balancedness, enjoyment, engrossment, and personal gratification.

This paper investigates to which degrees of playing games by using hand motions improve brain memory. Based on previous studies in psychology reporting that hand exercise could affect brain function and memory, we develop an interface for playing games by using hand motions. It works by detecting hand motions through image processing, translating them into commands for keyboard press, and sending such commands to the target game application; it works with any existing games without the need to modify the game source code and only requires an off-the-self webcam. An experiment is conducted on a jumping game, where three types of game interfaces: a keyboard, a Kinect, and the proposed interface are compared. The results show that playing games using hand clenching/spreading leads to the best memory test performance according to the N-back test, a commonly used cognitive task for the assessment of working memory.

This paper presents a motion-based rhythm game that facilitates rehabilitation at home. In this game, the player has to collect procedurally generated nodes, based on a pool of pre-recorded moves which can be modified by medical experts. A targeted percentage of movement can be specified for each part of the body; the game will generate a level that makes the player’s movement match the targeted percentages.

This paper presents an opponent fighting game AI for promoting balancedness in use of body segments of the player during full-body motion gaming. The proposed AI, named PDAHP-AI, is based on Monte Carlo tree-search and employs a recently purposed concept called Player Dominance Adjustment, where the AI determines its actions based on the player’s inputs so as to adjust the player’s dominant power. The basic idea is to let the player dominate the game when they perform healthy movement and on the contrary to have the AI take a strong action against the player when she or he performs unhealthy movement. The AI outperforms an existing dynamic difficulty adjustment AI designed for the same propose.

The twelve papers in this special section focus on game competition frameworks for the research and education markets. Presents highlights of some high-quality research and remarkable educational applications using the game competition frameworks.

Summary Universal Kinect‐type‐controller by ICE Lab (UKI, pronounced as ‘You‐key’) was developed to allow users to control any existing application by using body motions as inputs. The middleware works by converting detected motions into keyboard and/or mouse‐click events and sending them to a target application. This paper presents the structure and design of core modules, along with examples from real cases to illustrate how the middleware can be configured to fit a variety of applications. We present our designs for interfaces that decode all configuration details into a human‐interpretable language, and these interfaces significantly promote user experience and eliminate the need for programming skill. The performance of the middleware is evaluated on fighting‐game motion data, and we make the data publicly available so that they can be used in other researches. UKI welcomes its use by everyone without any restrictions on use; for instance, it can be used to promote healthy life through a means of gaming and/or used to conduct serious research on motion systems. The middleware serves as a shortcut in the development of motion applications—coding of an application to detect motions can be replaced with simple clicks on UKI. Copyright © 2017 John Wiley & Sons, Ltd.

This paper evaluates the performance of Monte-Carlo Tree Search (MCTS) in a fighting game AI and proposes an improvement for the algorithm. Most existing fighting game AIs rely on rule bases and react to every situation with predefined actions, making them predictable for human players. We attempt to overcome this weakness by applying MCTS, which can adapt to different circumstances without relying on predefined action patterns or tactics. In this paper, an AI based on Upper Confidence bounds applied to Trees (UCT) and MCTS is first developed. Next, the paper proposes improving the AI with Roulette Selection and a rule base. Through testing and evaluation using FightingICE, an international fighting game AI competition platform, it is proven that the aforementioned MCTS-based AI is effective in a fighting game, and our proposed improvement can further enhance its performance.

This paper proposes a procedural generation algorithm that creates game levels for the Angry Birds game. Game levels are automatically generated using a genetic algorithm (GA), and their difficulty is adjusted by a parameter introduced in the fitness function of GA. In addition, the Extended Rectangle Algebra (ERA) is used in order to analyze these levels. By calculating ERA relations between objects, effects of a bird hitting on an object are quantified by the aforementioned parameter. Our experiment proves that this parameter has a strong correlation of 96% with the players' winning percentage. Accordingly, it shows that any difficulty level can be generated by regulating the aforementioned parameter.

Real-time strategy (RTS) is a sub-genre of strategy video game which typically involves resource gathering, base building, strategy planning, and combat scenarios. With complicated gameplay, vast state and action spaces, RTS games have been proven to be an excellent platform for artificial intelligence research. One of the most challenging problems posed by RTS games is the detailed control of units in combat, i.e., unit micromanagement. In this paper, we present a method of integrating fuzzy integral and fast heuristic search for improving the quality of unit micromanagement in the popular RTS game StarCraft. Experiments are reported at the end of this paper, showing promising results and the potential of the proposed method in this domain.

In this paper, we present an application of Monte Carlo tree search (MCTS) to control ghosts in the game called Ms. Pac-Man. Our proposed ghost team consists of a ghost controlled by rules and three ghosts controlled individually by different MCTS. Given a limited time response, in order to increase the reliability of MCTS results, we introduce a mechanism for predicting Ms. Pac-Man's future movements and use this mechanism for simulating Ms. Pac-Man during Monte Carlo simulations. Our ghost team won the first Ms. Pac-Man Versus Ghost Team Competition at the 2011 IEEE Congress on Evolutionary Computation (CEC). Its performances for a variety of design choices are also shown and discussed.

In our previous work, we developed a fuzzy classifier with ellipsoidal regions that has a training capability. In this paper, we extend the fuzzy classifier to diagnosis problems, in which the training data belonging to abnormal classes are difficult to obtain while the training data belonging to normal classes are easily obtained. Assuming that there are no data belonging to abnormal classes, we first train the fuzzy classifier with only the data belonging to normal classes. We then introduce the threshold of the minimum-weighted distance from the centers of the clusters for the data belonging to normal classes. If the unknown data is within the threshold, we classify the data into normal classes and, if not, abnormal classes. The operator checks whether the diagnosis is correct. If the incoming data is classified into the same normal class both by the classifier and the operator, nothing is done. But if the input data is classified into the different normal classes by the classifier and the operator, or if the incoming data is classified into an abnormal class, but the operator classified it into a normal class, the slopes of the membership functions of the fuzzy rules are tuned. If the operator classifies the data into an abnormal class, the classifier is retrained adding the newly obtained data irrespective of the classifier's classification result. The online training is continued until a sufficient number of the data belonging to abnormal classes are obtained. Then the threshold is optimized using the data belonging to both normal and abnormal classes. We evaluate our method using the Fisher iris data, blood cell data, and thyroid data, assuming some of the classes are abnormal.

We present an efficient method for extracting fuzzy rules directly from numerical input-output data for function approximation problems. First, we convert a given function approximation problem into a pattern classification problem. This is done by dividing the universe of discourse of the output variable into multiple intervals, each regarded as a class, and then by assigning a class to each of the training data according to the desired value of the output variable. Next, we partition the data of each class in the input space to achieve a higher accuracy in approximation of class regions. Partition terminates according to a given criterion to prevent excessive partition. For class region approximation, we discuss two different types of representations using hyperboxes and ellipsoidal regions, respectively. Based on a selected representation, we then extract fuzzy rules from the approximated class regions. For a given input datum, we convert, or in other words, defuzzify, the resulting vector of the class membership degrees into a single real value. This value represents the final result approximated by the method. We test the presented method on a synthetic nonlinear function approximation problem and a real-world problem in an application to a water purification plant. We also compare the presented method with a method based on neural networks.

This paper gives introduction to four knowledge acquisition systems that use hyperboxes to represent knowledge. Hyperbox representation is interesting because it can uniformly represent both knowledge acquired by human experts and knowledge automatically learned from training examples. Potential applications of these systems are described. Discussions on performance issues and on future research topics are given.

In their previous work, the authors have developed a method for selecting features based on the analysis of class regions approximated by hyperboxes. They select features analyzing class regions approximated by ellipsoids. First, for a given set of features, each class region is approximated by an ellipsoid with the center and the covariance matrix calculated by the data belonging to the class. Then, similar to their previous work, the exception ratio is defined to represent the degree of overlaps in the class regions approximated by ellipsoids. From the given set of features, they temporally delete each feature, one at a time, and calculate the exception ratio. Then, the feature whose associated exception ratio is the minimum is deleted permanently. They iterate this procedure while the exception ratio or its increase is within a specified value by feature deletion. The simulation results show that the current method is better than the principal component analysis (PCA) and performs better than the previous method, especially when the distributions of class data are not parallel to the feature axes.

We present a neural-network approach which allows sequential extraction of source signals from a linear mixture of multiple sources in the order determined by absolute values of normalized kurtosis. To achieve this, we develop a non-linear Hebbian learning rule for extraction of a single signal. We discuss several techniques which enable extraction of signals not randomly but in the desired order. To prevent the same signals from being extracted several times, a robust deflation technique is used which eliminates from the mixture the already extracted signals. Extensive computer simulations confirm the validity and high performance of our method.

In this paper, we discuss a fuzzy classifier with ellipsoidal regions which has a learning capability. First, we divide the training data for each class into several clusters. Then, for each cluster, we define a fuzzy rule with an ellipsoidal region around a cluster center. Using the training data for each cluster, we calculate the center and the covariance matrix of the ellipsoidal region for the cluster. Then we tune the fuzzy rules, i.e., the slopes of the membership functions, successively until there is no improvement in the recognition rate of the training data. We evaluate our method using the Fisher iris data, numeral data of vehicle license plates, thyroid data, and blood cell data. The recognition rates (except for the thyroid data) of our classifier are comparable to the maximum recognition rates of the multilayered neural network classifier and the training times (except for the iris data) are two to three orders of magnitude shorter.

This paper presents a novel approach to feature selection based on analysis of class regions which are generated by a fuzzy classifier. A measure for feature evaluation is proposed and is defined as the exception ratio. The exception ratio represents the degree of overlaps in the class regions, in other words, the degree of having exceptions inside of fuzzy rules generated by the fuzzy classifier. It is shown that for a given set of features, a subset of features that has the lowest sum of the exception ratios has the tendency to contain the most relevant features, compared to the other subsets with the same number of features. An algorithm is then proposed that performs elimination of irrelevant features. Given a set of remaining features, the algorithm eliminates the next feature, the elimination of which minimizes the sum of the exception ratios. Next, a terminating criterion is given. Based on this criterion, the proposed algorithm terminates when a significant increase in the sum of the exception ratios occurs due to the next elimination. Experiments show that the proposed algorithm performs well in eliminating irrelevant features while constraining the increase in recognition error rates for unknown data of the classifiers in use.