Part of a university initiative supporting responsible AI for social empowerment and education, the project-based RAICA (Responsible AI for Computational Action) curriculum supports middle/high school learners and novice AI literacy teachers use AI creatively for good. This paper offers a rare example of design-based implementation research (DBIR) in AI education across widely varied contexts, provides fine grain implementation data that contributes to a foundation for evaluating effectiveness and expanding access. We present a novel approach to analyzing fidelity of implementation data from RAICA’s computer vision module beta-test. Twelve educators working with ~282 students across nine pilot sites in four countries used a bespoke fidelity of implementation data collection tool (pre-made comment prompts in a Google Docs version of the teacher guide) to provide 236 qualitative responses about AI literacy and responsible design activities, plus 111 ordinal ratings of embedded teacher supports. Analyses revealed that while the curriculum was generally implemented as designed, educators frequently made modifications. Although most changes produced practical insights for improved curriculum design, others helped the design team anticipate and prevent changes that could obscure learning objectives and hinder outcomes. We discuss the pedagogical, design, and research implications of these findings for effective AI teaching/learning in diverse settings.

Initial discussion of AI literacy assessment has focused on competency frameworks and learning standards rather than materials for classroom use. Responsible AI for Computational Action (RAICA), a constructionist AI curriculum for middle and high school students, includes assessment materials to support teachers with the evaluation of student AI literacy competencies in their classrooms. These materials include exit tickets used as formative assessments at the end of each lesson and both teacher and student-facing rubrics. After beta-testing a module of the curriculum with nine teachers and 282 students, we reviewed teacher usage data and feedback as well as student responses. The review process surfaced a number of improvements to the materials to better align them with classroom teaching practice. These included clarifying language and adding visual scaffolds. We present the assessment materials and iterative design process used to bridge the gap between the theoretical AI literacy competencies and their practical implementation in classrooms.

The Model AI Assignments session seeks to gather and disseminate the best assignment designs of the Artificial Intelligence (AI) Education community. Recognizing that assignments form the core of student learning experience, we here present abstracts of thirteen AI assignments from the 2025 session that are easily adoptable, playfully engaging, and flexible for a variety of instructor needs. Assignment specifications and supporting resources may be found at http://modelai.gettysburg.edu

CHI ’25, Yokohama, Japan

Block-based programming environments, such as Scratch and StarLogo Nova, are popular in STEM education for introducing algorithmic thinking to non-technical users. However, capturing and understanding students' interactions with these environments in real-time is challenging for educators, limiting their ability to provide effective feedback and support. In this research, we developed an integrated replay system within StarLogo Nova that records and replays student interactions, providing teachers with the ability to review and reflect on these interactions in detail. Additionally, the system also includes analytics features that help educators summarize user interactions and identify key moments, offering perspectives that go beyond what a standard screen capture can provide. We conducted a preliminary evaluation of the system to assess its accuracy in capturing user interactions, confirming its effectiveness for educators in analyzing student behavior. The results are promising, showing improved analytics and a 99.65% reduction in file size compared to traditional screen capture.

This study explores how players engage in problem solving during a cross-platform collaborative learning game about cellular biology. We tested the game with 8 pairs of students: 2 pairs of middle school students, 2 pairs of high school students, and 4 pairs of students in a biotechnology workforce program. Players took on one of two roles: An Explorer in virtual reality head mounted display and a Navigator using a tablet. Players took on one of two roles: An Explorer in virtual reality head mounted display and a Navigator using a tablet. Videos of 40-minute game play sessions for 8 pairs of players were transcribed and examined for patterns of game play based on stages of collaborative problem solving (CPS) developed by Fiore et al (2017). Players' discussion during gameplay was analyzed and visualized through Epistemic Network Analysis. Results suggested that players engaged in two-way communication using biology terms throughout the game, and that communication styles were linked to roles. After initial orientation, players moved through multiple cycles of clue finding and discussion that followed similar patterns even in groups with different knowledge levels. These results suggest that collaborative problem solving can be initiated through roles in a serious game and documented through participants' conversation during a serious game. Future research will focus on whether the game can be used to learn, practice, and improve players' CPS skills.

This paper describes the results of a comparative study carried out in a Singapore High School to test four versions of our game for learning, to investigate the effectiveness of puzzle and narrative-based games in engaging students, how the games affect their learning experience and their understanding of the physics concepts of displacement and velocity.

Ubiquitous Games (UbiqGames) are browser-based multi-player games played primarily on mobile devices. UbiqGames are designed to be played outside of formal class time and over several days, alongside more traditional instruction in the topics covered in the games. Teachers can use class time to facilitate thoughtful reflection on game strategies and data generated through gameplay. In the UbiqGame Weatherlings, players collect virtual creature cards and "battle" other players in arenas based on real cities, which feature the cities' meteorological data from the recent past. Success in the game requires players to consider climate conditions and do short-term forecasting from this real weather data. In this pilot study, 20 Singaporean 10 and 11 year-olds played Weatherlings using mobile devices over four days, playing primarily outside formal class time. Survey and interview data, combined with server logs, demonstrate positive outcomes for students including strong appeal and high engagement, increased interest in weather, and self-reported improvement in weather prediction skills. Outcomes suggest opportunities for additional research around UbiqGames to probe this new model for game-centered learning.