Instrumental music conveys rich affective experiences through acoustic cues, yet instrumental passages often remain inaccessible to Deaf and Hard-of-Hearing (DHH) audiences. Although captioning practices for vocal songs have expanded, instrumental music remains largely uncaptioned, with no established criteria for representing musical content in text. We propose µCap (Music Captions), an automatic instrumental music captioning system that transforms instrumental audio into time-aligned, non-lexical textual renderings enhanced with simple visuals. Drawing on Preliminary surveys with DHH individuals and expert group discussions, we developed a phonetic-like captioning schema grounded in music sound analysis and linguistics. We then implemented µCap using audio feature extraction and a retrieval-augmented generation pipeline to produce expressive, sound-mimetic captions. Two user evaluations with DHH participants (n=20 and n=15) showed that µCap enhanced music appreciation, immersion, and perceived presence of acoustic detail. This work contributes empirical evidence and insights for designing caption-based visual representations that make instrumental music more accessible.

Although students need to self-monitor and manage their learning process for effective metacognition, it can be particularly challenging in solitary e-learning environments that rely on pre-recorded videos. Unlike interactive e-learning or physical classrooms, typical e-learning environments prevent students from interacting with their teachers and peers, thereby hindering metacognitive support. To address this challenge, we introduce GPTalk, a system designed to support students’ learning experiences by facilitating interactions with LLM-based virtual companions. Through interviews with students and teachers, we identified design recommendations and implemented them in GPTalk. A user study involving 32 high-school students demonstrated that, compared to a baseline system, GPTalk fostered richer metacognitive engagement and self-regulated learning processes during video-based study (e.g., more monitoring questions and in-situ reflections), while short-term content understanding accuracy remained comparable across conditions. Overall, our findings suggest that students’ interactions with a virtual teacher and peer can support key aspects of their metacognition and self-regulated e-learning processes.

Recent research has highlighted the significance of natural language in enhancing the controllability of generative models. While various efforts have been made to leverage natural language for content generation, research on deep reinforcement learning (DRL) agents utilizing text-based instructions for procedural content generation remains limited. In this paper, we propose IPCGRL, an instruction-based procedural content generation method via reinforcement learning, which incorporates a sentence embedding model. IPCGRL fine-tunes task-specific embedding representations to effectively compress game-level conditions. We evaluate IPCGRL in a two-dimensional level generation task and compare its performance with a general-purpose embedding method. The results indicate that IPCGRL achieves up to a 21.4 % improvement in controllability and a 17.2 % improvement in generalizability for unseen instructions with varied condition expressions within the same task. Furthermore, the proposed method extends the modality of conditional input, enabling a more flexible and expressive interaction framework for procedural content generation.

In the domain of game artificial intelligence, tutorial systems have seen limited advancement despite their critical role in onboarding players. Traditional tutorials often neglect individual learning differences, leading to ineffective instruction. This study proposes a personalized in-game tutorial generation framework that leverages procedural content generation (PCG) and student modeling. The system integrates a Monte Carlo Tree Search (MCTS)-based state transition generator and a player modeling module to dynamically adapt tutorial content based on inferred internal knowledge. The approach is validated through large-scale user testing ($N$=88) in a commercial-style match-3 puzzle game environment. Results show that the personalized generator improves learning by up to 44.4% within short sessions. The findings highlight the potential of adaptive tutorials in accelerating skill acquisition and enhancing game experience through seamless, personalized learning contexts.

Driven by the rapid growth of machine learning, recent advances in game artificial intelligence (AI) have significantly impacted productivity across various gaming genres. Reward design plays a pivotal role in training game AI models, wherein researchers implement concepts of specific reward functions. However, despite the presence of AI, the reward design process predominantly remains in the domain of human experts, as it is heavily reliant on their creativity and engineering skills. Therefore, this paper proposes ChatPCG, a large language model (LLM)-driven reward design framework. It leverages humanlevel insights, coupled with game expertise, to generate rewards tailored to specific game features automatically. Moreover, ChatPCG is integrated with deep reinforcement learning, demonstrating its potential for multiplayer game content generation tasks. The results suggest that the proposed LLM exhibits the capability to comprehend game mechanics and content generation tasks, enabling tailored content generation for a specified game. This study not only highlights the potential for improving accessibility in content generation but also aims to streamline the game AI development process.

The balance of game content significantly impacts the gaming experience. Unbalanced game content diminishes engagement or increases frustration because of repetitive failure. Although game designers intend to adjust the difficulty of game content, this is a repetitive, labor-intensive, and challenging process, especially for commercial-level games with extensive content. To address this issue, the game research community has explored automated game balancing using artificial intelligence (AI) techniques. However, previous studies have focused on limited game content and did not consider the importance of the generalization ability of play-testing agents when encountering content changes. In this study, we propose RaidEnv, a new game simulator that includes diverse and customizable content for the boss raid scenario in the MMORPG games. Additionally, we design two benchmarks for the boss raid scenario that can aid in the practical application of game AI. These benchmarks address two open problems in automatic content balancing, and we introduce two evaluation metrics to provide guidance for AI in automatic content balancing. This novel game research platform expands the frontiers of automatic game balancing problems and offers a framework within a realistic game production pipeline. The open-source environment is available at a GitHub repository.

Multiplayer contents play an essential role in extending the lifespan of the game and positively affect the player’s experience. However, previous studies on procedural content generation focused on single-player games and few for multiplayer games. Multiplayer games have different ethics and mechanics associated with competition or cooperation. Thus, content designers are concerned about this interaction. In this paper, we propose a new method for generating multiplayer levels that encourage diverse cooperation among game players. Our contributions are summarized as follows: 1) We designed four cooperation patterns usable in controllable generation literature. 2) We proposed a controllable level generator to deploy the proposed patterns in the two-player cooperative cooking game, Overcooked!. 3) We discussed effective methods for leading players’ cooperation experience. Consequently, we found that the players’ interaction is most fundamental at the multiplayer level. The results of our study will lead to diverse cooperation experiences in future multiplayer game content generation studies.

Recently, several attempts have been made to train cooperative artificial intelligence (AI). From training superhuman-level agents to human-like agents, the purpose of an AI results in differences in the behavior policy. Indeed, training a human-like agent could enhance the experience of multiplayer game players. However, training human-like agents is challenging and there is little existing work concerning benchmarking cooperative agents with actual humans. As an initial step to address this problem, we suggest a software program and an experimental procedure to conduct Turing tests in multiplayer games. Our contribution will help current multiagent studies benchmark the human-likeness of the agents and investigate their characteristics.