Dr. Scratch automatically scores computational thinking (CT) in Scratch. While promising, these metrics are seldom evaluated in classroom realities. To compensate, we engaged six STEM teachers and 395 middle school students in qualitative classroom research using Dr. Scratch CT4ALL to support game-based learning (GBL). Through observations, field notes, interviews, and quantitative insights, we show how teachers personalize CT rubrics for GBL and how students score with these rubrics, while also documenting their experiences with automated CT metrics. We find that Dr. Scratch’s feedback often goes unheeded, confirming apprehensions from prior work. When CT is a secondary learning goal, students focus on code issues rather than CT points, questioning the usefulness of Dr. Scratch in supporting students’ motivation to learn about CT in GBL. Based on these results, we discuss the need for human-centered design of CT metrics, offering lessons learned and perspectives to improve CT metrics for classroom use.

The rise in popularity and value of esports motivates the creation of computational training tools (CTTs) for learning, assessment, and skill gain. While some tools exist commercially, much of the work in the research literature is rarely used outside of a lab, resulting in a lack of knowledge on the challenges involved in real-world integration. In this work, we develop a bespoke CTT for League of Legends—MySkills—based on prior work and deploy it at a professional training academy for three months. Based on two rounds of stakeholder interviews, we uncover insights into users’ perspectives on using CTTs in esports coaching and the challenges inherent in introducing a novel tool into an existing, real-world esports training context. From these results, we connect the domain of esports training technology to existing conversations on translational HCI, challenges in bridging research and practice, and present implications for future work.

Increasing numbers of people represent themselves in legal disputes—known as pro se litigants. Many lack the skills, experience, or knowledge to navigate legal proceedings without a lawyer, resulting in limited access to justice. Serious games may provide an effective, interactive, and engaging way of educating pro se litigants about the law and enabling their access to justice. Through participatory design with legal experts and an authoring tool, we co-designed RePresent, a serious game that helps individuals with limited access to legal support prepare for pro se litigation. A total of 965 people played RePresent, and 149 provided feedback on their player experience. Results show that RePresent was engaging and valuable for learning about the law and pro se litigation. Our work highlights avenues for co-design methodologies with co-creative authoring tools that facilitate serious game design, contributing a potentially scalable solution to enable access to justice via co-authored serious games.

Previous work demonstrated that esports players often leverage insights from other players and communities to learn and improve. However, little research examined social learning in esports, over time, in granular detail. Understanding the role of others in the esports learning process has implications for the design of computational support systems that can help esports players learn and make the games more accessible. Therefore, we perform an exploration of this topic using Co-Regulated Learning as a theoretical lens. In doing so, we hope to enrich existing knowledge on social learning in esports, provide insights for the future development of computational support, and a road-map for future work. Through an interview study of an esports community consisting of 14, college-aged, male players, we uncovered 10 themes regarding how Co-Regulated learning occurs within their teams. Based on these, we discuss three main takeaways and their implications for future research and development.

Educational games have emerged as potent tools for helping students understand complex concepts and are now ubiquitous in global classrooms, amassing vast data. However, there is a notable gap in research concerning the effective visualization of this data to serve two key functions: (a) guiding students in reflecting upon their game-based learning and (b) aiding them in analyzing peer strategies. In this paper, we engage educators, students, and researchers as essential stakeholders. Taking a Design-Based Research (DBR) approach, we incorporate UX design methods to develop an innovative visualization system that helps players learn through gaining insights from their own and peers’ gameplay and strategies.

The concept of dealing damage is established and widespread in video games. With growing complexity and countless interactions in modern games, capturing how damage unfolds becomes an intricate problem - for developers just as for players. Misunderstanding how to optimize damage potentials includes risks of game imbalances, game-breaking exploits, mismatches between player skill and challenge (harming flow), and impaired perceived competence. All of these considerably impact player experience, game reception, success, and retention, yet polishing optimal strategies remains often a player community effort. To accelerate, inform and ease this process, we implemented an interactive tool capable of simulating, visualizing, planning and comparing damage strategies in video games. Following a case study within the Guild Wars 2 community, we contribute a player-driven perspective on the problem of damage optimization, as well as an artifact that resulted in empirical improvements – advancing the fields of game analytics, game evaluation methods and self-regulated learning.

This paper introduces a scoping review of 55 articles to explore GPT’s potential for games, offering researchers a comprehensive understanding of the current applications and identifying both emerging trends and unexplored areas. We identify five key applications of GPT in current game research: procedural content generation, mixed-initiative game design, mixed-initiative gameplay, playing games, and game user research. Drawing from insights in each of these application areas, we propose directions for future research in each one. This review aims to lay the groundwork by illustrating the state of the art for innovative GPT applications in games, promising to enrich game development and enhance player experiences with cutting-edge AI innovations.

In 2023, the Hidden Heroes game jam allowed game developers to share untold stories of individuals who had a significant impact on their lives. As a result, 33 games were created embracing the stories of real people. Interviews were conducted with 12 of these developers to understand their experiences of creating games about authentic personal stories. Thematic analysis revealed eight themes and three categories that captured the experiences and challenges of making such personal narrative games. Based on these findings, we reflected on the future of authentic personal narrative games.

Mixed-initiative co-creative storytelling games have existed for some time as a way to merge storytelling with play. However, modern mixed-initiative co-creative storytelling games predominantly prioritize story creation over gameplay mechanics, which might not resonate with all players. As such, there is untapped potential for creating mixed-initiative games with more complex mechanics in which players can engage with both co-creation and gameplay goals. To explore the potential of more prominent gameplay in mixed-initiative co-creative storytelling games, we created Snake Story, a variation of the classic Snake game featuring a human-AI co-writing element. To explore how players interact with the mixed-initiative game, we conducted a qualitative playtest with 11 participants. Analysis of both think-aloud and interview data revealed that players’ strategies and experiences were affected by their perception of Snake Story as either a collaborative tool, a traditional game, or a combination of both. Based on these findings, we present design considerations for future development in mixed-initiative co-creative gaming.

Self-reflection, a process consisting of evaluation, attribution, and adaptation, is recognized as key to learning in esports games. To better facilitate self-reflection, many games and third-party tools feature retrospective visualizations that provide players with details about their performance. However, previous work suggested that existing visualizations, which typically aggregate data, do not present sufficient information to players. In this work, we explore the impact of process visualizations, which present gameplay as a sequence of actions, on self-reflection and performance in a League of Legends training exercise through a mixed-methods study. Our results found that players who reflected using a process visualization significantly improved their performance and demonstrated significant differences in how they evaluated themselves compared to those who reflected using an aggregate visualization. We discuss what these results mean in relation to self-reflection and learning in esports and how they impact the future use of process visualizations in games.

Adaptation, or ability and willingness to consider an alternative approach, is a critical component of learning through reflection, especially in educational games, where there are often multiple avenues to success. As a domain, educational games have shown increased interest in using retrospective visualizations to promote and support reflection. Such visualizations, which can facilitate comparison with peer data, may also have an impact on adaptation in educational games. This has, however, not been empirically examined within the domain. In this work, we examine how comparison with other players’ data influenced adaptation, a part of reflection, in the context of a game that teaches parallel programming. Our results indicate that comparison with peers does significantly impact willingness to try a different approach, but suggest that there may also be other ways. We discuss what these results mean for future use of retrospective visualizations in educational games and present opportunities for future work.

As games get more complicated, Artificially Intelligent (AI) agents need to be better developed to understand and replicate complex, goal-oriented, reactive behaviors. Many existing behavior language approaches do not often make the AI’s behavior transparent and readable to the human, making it difficult to debug and update the behavioral model. In this workshop paper, we present a prototype visualization system that uses a tree-based abstraction to present complex agent behavior to a human user, allowing them to understand and edit the agent’s behavioral logic in a visual manner.

Esports play can cultivate real world skills. However, the path to mastery is not easy, and difficulty progressing can result in discontinuation. In the absence of a human coach, computational tools may provide much-needed guidance. However, the specific improvement activities that players engage in and the exact challenges they face are not well defined in the context of computational support. As such, most tools can only support players based on a high level understanding of their practices. We present the results of an interview study (n=17) that identified four improvement activities: practicing, leveraging the knowledge of others, tracking performance, and reflecting on gameplay and setting goals for the future, and four challenges: coordinating and collaborating with teammates, knowing what to do next, tracking game state, and tracking skill and improvement. We discuss six implications for future design and development based on these results.

Computational support for learning in the domain of esports has seen a great deal of attention in recent years as an effective means of helping players learn and reap the benefits of play. However, previous work has not examined the tools from a learning theory perspective to assess if learning is prompted and supported in the right place and time. As a first step towards addressing this gap, this paper presents the results of two studies: a review of existing computational tools, and an online survey of esports' players' learning needs supplemented with qualitative interviews. Using Zimmerman's Cyclical Phase Model of Self-Regulated Learning as a lens, we identify patterns in the types of support offered by existing tools and players' support interests during different learning phases. We identify 11 opportunities for future research and development to better support self-regulated learning in esports.

Player agency is central to interactive narrative and games. While previous work focuses on analyzing player perception of agency through various lenses and phenomena, like meaningful choice and expectations, it is largely theoretical. Few user studies within games explore how players reason about and judge their own agency within interactive narratives. We present an interview study where participants rated their agency experiences within narrative-focused games and described their reasoning. The analysis suggests that agency perception depends on multiple factors beyond meaningful choice, such as social investment and genre-conventions. Participants described varying preferences and value judgements for different factors, indicating that individual differences have a deep impact on agency perception in narrative-focused gameplay. We discuss the implications of these cognitive variables on design, how they can be leveraged with other factors, and how our findings can help future work enhance and measure player agency, within interactive narrative and beyond.

In 2020, the rapid spread of COVID-19 forced many people to self-isolate, resulting in struggles with mental health [60], and an increase in gaming [3]. In this paper, we seek to examine how individuals used digital games during the quarantine. We conducted a two-tier qualitative study where we used a thematic analysis of tweets to derive questions for an online survey that we distributed. Results of thematic analysis of survey responses identified 15 themes. Some themes confirm previous works’ findings, particularly how games are used to increase social connection or distract oneself from unpleasant situations. We also found new themes unique to the quarantine, such as interactions with non-player characters used as a surrogate for real-world interaction and using in-game routines as a substitute to real-world routines lost due to the pandemic. This work discusses the use of games during the pandemic and can be seeds for future studies.

Game AI designers must manage complex interactions between the AI character, the game world, and the player, while achieving their design visions. Computational co-creativity tools can aid them, but first, AI and HCI researchers must gather requirements and determine design heuristics to build effective co-creative tools. In this work, we present a participatory design study that categorizes and analyzes game AI designers’ workflows, goals, and expectations for such tools. We evince deep connections between game AI design and the design of co-creative tools, and present implications for future co-creativity tool research and development.

To reduce frustration while performing no-risk tasks (e.g. in training and games) for BCI users, we propose increasing their perceived level of control through fabricated input - system-generated positive task outcomes. Two surrogate BCI studies injected fabricated input creating additional positive task outcomes to a 50% baseline. Users' perceived control increased significantly compared to the 50% baseline. In turn, frustration levels decreased. Fabricated input worked equally well in a game story context that provided an emotional stake in the protagonist's success and a simpler task lacking such incentives. People's number of input attempts during the tasks determined perceived control more than our controlled ratios of positive to negative task outcomes. Delays between users' input attempts and subsequent fabricated inputs further moderated their perceived control.

In recent years, the games industry has made a major move towards data-driven development, using data analytics and player modeling to inform design decisions. Data-driven techniques are beneficial as they allow for the study of player behavior at scale, making them very applicable to modern digital game development. However, with this move towards data driven decision-making comes a number of ethical concerns. Previous work in player modeling [45] as well as work in the fields of AI and machine learning [9, 53] have demonstrated several ways in which algorithmic decision-making can be flawed due to data or algorithmic bias or lack of data from specific groups. Further, black box algorithms create a trust problem due to lack of interpretability and transparency of the results or models developed based on the data, requiring blind faith in the results. In this position paper, we discuss several factors affecting the use of game data in the development cycle. In addition to issues raised by previous work, we also raise issues with algorithms marginalizing certain player groups and flaws in the resulting models due to their inability to reason about situational factors affecting players’ decisions. Further, we outline some work that seeks to address these problems and identify some open problems concerning ethics and game data science.

Many successful games rely heavily on data analytics to understand players and inform design. Popular methodologies focus on machine learning and statistical analysis of aggregated data. While effective in extracting information regarding player action, much of the context regarding when and how those actions occurred is lost. Qualitative methods allow researchers to examine context and derive meaningful explanations about the goals and motivations behind player behavior, but are difficult to scale. In this paper, we build on previous work by combining two existing methodologies: Interactive Behavior Analytics (IBA) [2] and sequence analysis (SA), in order to create a novel, mixed methods, human-in-the-loop data analysis methodology that uses behavioral labels and visualizations to allow analysts to examine player behavior in a way that is context sensitive, scalable, and generalizable. We present the methodology along with a case study demonstrating how it can be used to analyze behavioral patterns of teamwork in the popular multiplayer game Defense of the Ancients 2 (DotA 2).

Debugging is an integral part of programming and is difficult to master among novice programmers. We examine how middle-school learners approach debugging within a puzzle game, May's Journey, using a Playback Tool that provides a replay of student gameplay to support teacher practice and research. Teachers and researchers can use our labeling system and Playback Tool to observe students' debugging process and identify those who are struggling to learn how to debug their code.

be especially problematic for beginners. While there has been some work trying to understand the difficulties that beginners face with debugging, investigating common mistakes or specific error types they struggle with, there is little work that focuses on in-depth analysis of how novice programmers approach debugging, and how it changes over time. In this paper, we present MAADS (Mixed-Methods Approach for the Analysis of Debugging Sequences), a scalable and generalizable approach that combines quantitative and qualitative methods by using a state/action representation and visualization to gain knowledge about the debugging process through a step by step analysis. To demonstrate the utility of MAADS, we analyzed the debugging processes of middle school students who developed code within May's Journey, a game designed to teach basic programming principles. The approach showed great utility in identifying differences in students' debugging techniques and learning paths.

When studying the impact of interactive narratives, especially in high impact, sensitive domains such as education and health, it is important that the narrative design as a whole is reported in sufficient detail to allow for analysis and replication. However, the design of interactive narrative applications is often insufficiently documented and reported, despite being a central component and, often, a focus of study. This is due, in part, to a lack of a common lexicon that allows designers and practitioners to describe and compare their designs. This paper examines interaction design in interactive narrative games, specifically structure and progression mechanics, from the perspective of establishing common ground between designer and player. Using the interaction model for interactive narratives to guide our analysis, we found patterns of structure and narrative progression mechanics that offer insight into commercial design practices, which can inform and guide interaction design choices for narrative games.

Teamwork is becoming an important element of gameplay. Many games nowadays incorporate teamwork in one form or another. As a result, many game researchers have investigated aspects of teamwork. However, we have yet to develop effective surveys to gauge how people strategize or problem solve together within different games, and how such games enable these activities. In this paper, we propose the Teamwork and Adaptation in Games (TAG) survey. We developed this survey using theories of team adaptation, a process of how teams work together and adapt their strategies to changes in environments. Through three separate studies, the TAG survey was used to gauge team adaptation. Our results show that the TAG survey is an effective instrument allowing us to gauge team adaptation in different game environments. We further found that the instrument can be effectively used to compare games.

There are several educational games and tools that teach programming. However, very few offer a deep understanding of Computer Science concepts such as Abstraction, Modularity, Semantics, and Debugging. We present May's Journey, an educational game that supports learning of basic programming concepts, where players solve puzzles and interact with the environment by typing in a custom programming language. The game design seamlessly integrates learning goals, core mechanics, and narrative elements. We discuss how we integrate the CS concepts mentioned above using game mechanic metaphors.

Interactivity and player experience are inextricably entwined with the creation of compelling narratives for interactive digital media. Narrative shapes and buttresses many such experiences, and therefore designers must construct compelling narrative arcs while carefully considering the effects of interaction on both the story and the player. As the narrative becomes more structurally complex, due to choice-based branching and other player actions, designers need to employ commensurately capable models and visualizations to keep track of that growing complexity. However, previous models of interactive narrative have failed to fully capture interactive elements with automated, operationalized visualizations. In this paper, we describe an algorithm for automated construction of a framework-driven, graph-based representation of interactive narrative. This representation more fully and transparently models structural and interactive features of the narrative than did prior approaches. We present an initial evaluation of this representation, based on modified cognitive walkthroughs performed by interactive narrative design and research experts from our research team, and we describe the takeaways for future improvement on interactive narrative modeling and analysis.

A growing number of new games utilize rewind as a narrative mechanic and there is an increasing need for a reliable way to understand and evaluate storytelling in such games. This paper builds on previous work by conducting narrative interaction focused analysis on rewind mechanics in commercial video games, and focuses on rewind as a form of narrative progression. The analysis resulted in updated definitions of rewind mechanic types originally defined by Kleinman et al. [13]. An updated framework is proposed where rewind is described in terms of control, scope, initiation, presentation, story structure, and structure traversal. Common design patterns associated with rewind, such as narrative justification and feedback, are also discussed. This expanded framework can be used to better understand rewind mechanics, and evaluate storytelling and user experience in the video games that utilize them.