Creativity tools are a cornerstone of HCI, with systems for video, music, writing, and design deeply embedded in modern creative practice. Yet one key element of these systems remains undertheorized: the role of activity traces. Activity traces are the records of creator data, including artifact iterations, annotations, or reference materials, produced over the course of a creative process. To examine how activity traces are leveraged, we reviewed 133 creativity systems from major HCI venues. We structure our findings through a Living Framework for Trace Awareness, which captures both the characteristics of trace data and how systems engage with their temporal features. This framework offers the first systematic account of activity trace usage in creativity tools. We highlight overlooked assumptions about creator data in feature design and position activity traces as a core design material for shaping the next generation of creativity support systems.

Background music plays a crucial role in social media videos, yet finding the right music remains a challenge for video creators.These creators, often not music experts, struggle to describe their musical goals and compare options.AI text-to-music generation presents an opportunity to address these challenges by allowing users to generate music through text prompts; however, these models often require musical expertise and are difficult to control.In this paper, we explore how to incorporate music generation into video editing workflows.A formative study with video creators revealed challenges in articulating and iterating on musical preferences, as creators described music as "vibes" rather than with explicit musical vocabulary.Guided by these insights, we developed a creative assistant for music generation using editable vibe-based recommendations and structured refinement of music output.A user study showed that the assistant supports exploration, while direct prompting is more effective for precise goals.Our findings offer design recommendations for AI music tools for video creators.

In this work, we designed and developed Dice Adventure, a turnbased multiplayer game where three characters work together to reach their individual goals and then a shared team goal to complete each level.Using Dice Adventure as the environment, we hosted a game competition with two tracks: agent and player.Participants could join one or both by submitting an agent they developed and/or signing up to play with the agents submitted by other developers.We collected competition game play data as part of a human-AI teaming pilot study to understand team behaviors, performance, and to test our systems.Insights from the competition also informed the design of a randomized controlled study for future experiments and competitions, aimed at exploring key human-AI teaming questions-such as how role assignments, team compositions, and team dynamics influence team performance.Our work introduces a novel gaming environment to support future research on human-AI teaming and offers preliminary insights into the design of such studies.

Human–Agent Teaming (HAT) research spans machine learning, HCI, robotics, and cognitive science, but suffers from fragmented testbeds that limit generalization and comparability. We introduce Vertical Farm, a cooperative farming simulator explicitly designed to tackle three key challenges in HAT testbed design: expressiveness (supporting diverse research agendas), unification (enabling standardized benchmarks), and rich team dynamics (capturing the complexity of real-world teaming). Vertical Farm embodies three design principles (composability, human-agent complementarity, and flexible team & role structure) that allow it to support a wide range of experimental configurations, from role negotiation to mixed-initiative control. Together, these contributions provide a blueprint for more unified, expressive, and realistic HAT research.

This paper explores new ways to support blind and low vision (BLV) game stream participants. Prior work on game-aware streaming systems has focused on the potential for viewer interaction and personalization for sighted viewers, but how such systems impact interaction and personalization for BLV viewers remains largely unexplored. Most streaming experiences have significant visual information but no non-visual or sensemaking alternatives, which can exclude BLV viewers from understanding and interacting with the streaming experience. Building on the pre-existing system MARS, we developed a design probe that makes game data available to stream viewers in personalizable visual and non-visual formats. We use this probe to investigate the needs of BLV game stream viewers through qualitative interviews and live prototype testing sessions on Twitch. In addition to the technical contributions of our probe, our work addresses how game-aware streaming technologies can align with the needs and motivations of BLV viewers, and paves the way for novel designs in future iterations of game-aware streaming technologies.

In this paper we describe the need for a framework to support collaborative educational research with game data, then demonstrate a promising solution. We review existing efforts, explore a collection of use cases and requirements, then propose a new data architecture with related data standards. The approach provides modularity to the various stages of game data generation and analysis, exposing intermediate transformations and work products. Foregrounding flexibility, each stage of the pipeline generates datasets for use in other tools and workflows. A series of interconnected standards allow for the development of reusable analysis and visualization tools across games, while remaining responsive to the diversity of potential game designs. Finally, we demonstrate the feasibility of the approach through an existing implementation that uses this architecture to process and analyze data from a wide range of games developed by multiple institutions, at scale, supporting a variety of research projects.

Two pilot studies compare the impacts of scaffolded versus self-paced practice on teaming and performance on an open-ended design challenge. In both studies, guiding players early on in how to leverage AI assistance (scaffolded practice) led to much more robust teaming than allowing players to learn at their own pace, but did not improve task performance.

Research on live streaming systems that incorporate real-time data, such as game or viewer data, have been a topic of HCI research for some time. Despite the potential of data-driven game streaming interfaces, translating this research into practice faces two key challenges. First, the design space afforded by data-driven game streaming systems is not yet well understood, making it difficult to identify how designs might meet users' existing and potential needs. Second, adoption of these systems relies on engagement with the entire streaming ecosystem, which includes developers, streamers, moderators, and viewers, rather than with just one group. Through a two-phase design study, we investigate the expectations, desires, and experiences of streaming stakeholders, shedding light on how data-driven game streaming systems can meet their needs. Building upon these insights and drawing upon previous research, we propose a design framework aimed at analyzing and generating data-driven game streaming designs, thereby moving toward formalizing the design and development of such systems.

The importance of considering local context and partnering with target users is well established in co-design. Less common is an examination of the adaptations needed when deploying the same co-design program across heterogenous settings to maximize program efficacy and equity. We report on our experience co-designing educational games with six culturally and socioeconomically diverse afterschool sites over two years, and insights from interviewing ten program administrators across all sites. We found that even within the same afterschool program network, site differences in organizational culture and resources impacted the effectiveness of co-design programs, the co-design output, and expectations for student engagement. We characterize our afterschool partners into different archetypes – Safe Havens, Recreation Centers, Homework Helpers, and STEM Enrichment Centers. We provide recommendations for conducting co-design at each archetype and reflect on strategies for increasing equitable partnerships between researchers and afterschool centers.

We explore the future of synchronous, multiplayer mobile AR gaming through our game V-Light, which extends current mobile AR game capacities using edge computing. Mobile AR games are currently limited by on-board processing power, while offloading operations to the cloud introduces high latency costs. This is a critical issue for games needing real-time response to player input. V-Light demonstrates how mobile AR games can leverage the power of edge computing, bringing computational resources closer to the user, keeping latency low and bandwidth high. We share our development toolkit, analyze the design and development of V-Light through the lens of an existing model for shared-world mobile AR, and demonstrate that edge computing can provide a “time machine” that lets game designers prototype mobile AR games for devices that do not yet exist.

While recent advances in Artificial Intelligence and Machine Learning have demonstrated the potential for AI systems to outperform human experts in many domains, including games, AI systems still generally lack the ability to team with humans on complex tasks. One of the barriers to addressing this challenge is a lack of shared task domains in which to do basic research to study Human-Machine Teaming strategies. In our work, we employ speculative game design to create asymmetric cooperative games that can serve as test beds to study human-machine teaming challenges. In this paper, we will describe our general approach and detail the current state of our development efforts.

We present MARS (Metadata Augmented Real-time Streaming), a system that enables game-aware streaming interfaces for Twitch. Current streaming interfaces provide a video stream of gameplay and a chat channel for conversation, but do not allow viewers to interact with game content independently from the steamer or other viewers. With MARS, a Unity game’s metadata is rendered in real-time onto a Twitch viewer’s interface. The metadata can then power viewer-side interfaces that are aware of the streamer’s game activity and provide new capacities for viewers. Use cases include providing contextual information (e.g. clicking on a unit to learn more), improving accessibility (e.g. slowing down text presentation speed), and supporting novel stream-based game designs (e.g. asymmetric designs where the viewers know more than the streamer). We share the details of MARS’ architecture and capabilities in this paper, and showcase a working prototype for each of our three proposed use cases.

Creating technology products using codesign techniques often results in higher end-user engagement compared to expert-driven designs. Codesign sessions are typically structured in flexible and informal ways to achieve equal design partnerships, especially in adult-child interactions. This generally leads to better design output, however, it may also increase the enactment of socially constructed stereotypes and biases in ways that negatively affect the experiences of racial minorities and girls/women in design spaces. We codesigned a video game with a K-5 afterschool program located in a working-class, rural, predominantly white county over 20 weeks. We uncover ways that the codesign process and different activity types can create a permissive environment for enacting behaviors that are harmful to minorities. We discuss ways to manage and restructure codesign programs to be more conducive for children and adults from diverse backgrounds, ultimately leading to healthier design partnerships.

AI-enabled decision support systems have repeatedly failed in real world applications despite the underlying model operating as designed. Often this was because the system was used in an unexpected manner. Our goal is to enable better prediction of how systems will be used prior to their implementation as well as to improve existing designs, by taking human behavior into account. There are several challenges to collecting such data. Not having access to an existing prediction engine requires the simulation of such a system's behavior. This simulation must include not just the behavior of the underlying model but also the context in which the decision will be made in the real world. Additionally, collecting statistically valid samples requires that test subjects make repeated choices under slightly varied conditions. Unfortunately, in such repetitious conditions fatigue can quickly set in. Games provide us the ability to address both of these challenges by providing both systems context and narrative context. Systems context can be used to convey some or all of the information the player needs to make a decision in the game environment itself, which can help avoid the onset of fatigue. Narrative context can provide a broader environment within which the simulated system operates, adding a sense of progress, showing the effect of decisions, adding perceived social norms, and setting incentives and stakes. This broader environment can further prevent player fatigue while replicating many of the external factors that might affect choices in the real world. In this paper we describe the design of the Human-AI Decision Evaluation System (HADES), a test harness capable of interfacing with a game environment, simulating the behavior of an AI-enabled decision support system, and collecting the results of human decision making based upon such a system's predictions. Additionally, we present an analysis of data collected by HADES while interfaced with a visual novel game focused on software cyber-risk assessment.

As we design increasingly complex systems, we run up against fundamental limitations of human imagination. To support practice, it becomes essential to use authentic data and algorithms as design materials to augment designers' intuitions. Recent work has explored some dimensions of using data as a design material, suggesting the contours of a new space of design and prototyping methods. In this paper, we present Replay Enactments (REs, an extension of the User Enactments methods that uses data replay as a boundary object, making complex system behavior tangible to designers and stakeholders. We reflect on a set of case studies that have instantiated REs in diverse ways and discuss trade-offs between different ways of using data replays in design. We conclude by highlighting opportunities and challenges for future work.

Intelligent tutoring systems (ITSs) have consistently been shown to improve the educational outcomes of students when used alone or combined with traditional instruction. However, building an ITS is a time-consuming process which requires specialized knowledge of existing tools. Extant authoring methods, including the Cognitive Tutor Authoring Tools' (CTAT) example-tracing method and SimStudent's Authoring by Tutoring, use programming-by-demonstration to allow authors to build ITSs more quickly than they could by hand programming with model-tracing. Yet these methods still suffer from long authoring times or difficulty creating complete models. In this study, we demonstrate that Simulated Learners built with the Apprentice Learner (AL) Framework can be combined with a novel interaction design that emphasizes model transparency, input flexibility, and problem solving control to enable authors to achieve greater model completeness in less time than existing authoring methods.

How do the features of a learning environment's user interface impact learners' agency and, further, their learning? We explored this question in the context of Decimal Point, a digital learning game designed to support middle school students in learning decimals. Previous studies of the game showed that giving students the ability to choose the order and number of mini-games to play did not significantly impact their learning outcomes compared to a condition without choice. In this paper we explore whether some elements of the game's interface may have inadvertently exerted indirect control over students' choice, leading to the previous effects. We conducted a classroom study using a new version of the game that varied whether students saw a visual path connecting mini-games on the game map to modulate the level of indirect control students would experience with an implied ordering. Ultimately, we found that students in the no-line condition exercised significantly more agency but did not learn any less than the line condition. These results suggest that indirect control can be a subtle but powerful way to direct student attention in digital learning games.

The recent surge in interest in using educational data mining on student written programs has led to discoveries about which compiler errors students encounter while they are learning how to program. However, less attention has been paid to the actual code that students produce. In this paper, we investigate programming data by using learning curve analysis to determine which programming elements students struggle with the most when learning in Python. Our analysis extends the traditional use of learning curve analysis to include less structured data, and also reveals new possibilities for when to teach students new programming concepts. One particular discovery is that while we find evidence of student learning in some cases (for example, in function definitions and comparisons), there are other programming elements which do not demonstrate typical learning. In those cases, we discuss how further changes to the model could affect both demonstrated learning and our understanding of the different concepts that students learn.

As educational games have become a larger field of study, there has been a growing need for analytic methods that can be used to assess game design and inform iteration. While much previous work has focused on the measurement of student engagement or learning at a gross level, we argue that new methods are necessary for measuring the alignment of a game to its target learning goals at an appropriate level of detail to inform design decisions. We present a novel technique that we have employed to examine alignment in an open-ended educational game. The approach is based on examining how the game reacts to representative student solutions that do and do not obey target principles. We demonstrate this method using real student data and discuss how redesign might be informed by these techniques.

The field of Educational Games has seen many calls for added rigor. One avenue for improving the rigor of the field is developing more generalizable methods for measuring student learning within games. Throughout the process of development, what is relevant to measure and assess may change as a game evolves into a finished product. The field needs an approach for game developers and researchers to be able to prototype and experiment with different measures that can stand up to rigorous scrutiny, as well as provide insight into possible new directions for development. We demonstrate a toolkit and analysis tools that capture and analyze students' performance within open educational games. The system records relevant events during play, which can be used for analysis of player learning by designers. The tools support replaying student sessions within the original game's environment, which allows researchers and developers to explore possible explanations for student behavior. Using this system, we were able to facilitate a number of analyses of student learning in an open educational game developed by a team of our collaborators as well as gain greater insight into student learning with the game and where to focus as we iterate.