Effective classroom teaching requires instructors to be responsive to their students, such as by pivoting their lectures in real-time to address common misconceptions that their students may have developed. Classroom response systems such as multiple-choice "clicker" systems are one method by which instructors can gauge their students’ understanding during classroom lectures, but open-ended questions that prompt students to engage in self-explanation are better suited to promoting critical thinking. Additionally, analyzing students’ natural language responses typically requires time-consuming manual analysis, which makes it challenging to implement in a classroom setting. To address this challenge, we present an LLM-driven method for automatically assessing students' responses and generating an aggregated summary of LLM-based evaluations for their self-explanations during undergraduate classroom lectures. Our approach extracts relevant knowledge components for a given question, tags students’ responses according to whether they correctly address each knowledge component, and generates class-level summaries that highlight common misconceptions and gaps in knowledge to support instructors in pivoting their lectures in real time. We evaluate the system’s effectiveness at these tagging and summarization tasks on data from an undergraduate computer science course, using quantitative and qualitative metrics such as relevance, sufficiency, hallucination rate, and alignment with instructional goals and desired feedback format gathered through instructor interviews. Results suggest that the explanation-based classroom response system can accurately analyze students’ natural language explanations.

Engaging middle school students in complex computational topics such as AI can pose unique challenges to educators, ranging from simplifying potentially difficult mathematical material to maintaining student interest in the subject. One approach to help address these challenges is to utilize hands-on, real-world examples. We conducted a week-long summer camp for 24 students, centering each day around activities aligned with one of the Five Big Ideas in AI. Students participated in exit ticket reflections following each activity. The pathfinding activities, which occurred on one of the days, incorporated real-world examples and digital simulations of three pathfinding algorithms (breadth-first search, depth-first search, and A*), including an activity modeled after the game Pac-Man. Thematic analysis of exit-ticket responses revealed four major themes regarding students' key takeaways from the activities: (1) pathfinding for character movement, notably in video games like Minecraft; (2) pathfinding as a means of efficient navigation; (3) theoretical reasoning regarding the speed of the A* algorithm compared to others, highlighting its intelligent search mechanism; and (4) empirical reasoning based on personal experience during activities, where some students noted A* consistently performed fastest. These findings indicate that students not only engaged with AI concepts but also demonstrated a nuanced understanding of algorithmic efficiency. We examine the implications of these findings on understanding student engagement with interactive pathfinding activities and highlight the potential for future work in this area.

In the evolving field of mixed-initiative game design, where procedural content generation plays a pivotal role, establishing a comprehensive approach that empowers non-technical designers to actively shape content generation is essential. Recent developments in large language models significantly alter the landscape of automated text-based content generation. These models offer a significant advantage in mixed-initiative procedural level generation by providing designers with intuitive, natural language interfaces. The framework presented in this paper interprets natural language inputs, detailing level design constraints and optimization goals, to aid in the cooperative development of game levels for a strategy game aimed at environmental sustainability education. It enables designers to articulate their vision concerning the problem domain, goal metrics, and desired difficulty level through a textual description. By utilizing large language models, the framework extracts semantic constraints and optimization objectives, which are then used to generate candidate game levels. The efficacy of these levels is assessed by game-playing agents trained through advanced deep reinforcement learning methods, ensuring alignment with the designer's original specifications. We further evaluate our framework with both experts and non-experts in designing levels for our strategy game. Their detailed responses confirm that our framework effectively translates natural language descriptions into playable game levels, accurately capturing the designers' intended objectives.

As the role of procedural content generation in mixed-initiative game design continues to grow, it is crucial to develop an end-to-end approach that enables non-technical designers to artfully guide content generation. Recent advances in large language models, such as GPT-4, are rapidly transforming the landscape of automated generation of text-based content. Large language models have significant potential for mixed-initiative procedural level generation by providing natural language interfaces for designers. This paper presents an end-to-end procedural level generation framework that interprets natural language descriptions of level design constraints and optimization objectives to facilitate the collaborative creation of game levels for a strategy game focused on environmental sustainability education. The framework enables designers to specify a problem domain, goal metrics, and target difficulty via natural language description. It then employs large language models for the semantic extraction of constraints and optimization targets to drive the generation of candidate levels. Generated game levels are evaluated via game-playing agents trained with deep reinforcement learning techniques to ensure the game levels meet the level designer’s specifications. Manual evaluation by authors shows that the proposed framework can effectively transform designers’ natural language descriptions into fully playable game levels that reflect their intended design objectives.

Tile maps are useful for a wide variety of games, particularly tabletop role-playing games. However, existing tools for creating them usually either require the map to be created entirely by hand, or procedurally generate the entire map with only a few parameters being controlled by the user. In previous work, we presented a mixed-initiative tool for generating tile maps that gives the user precise control over the generated map while still allowing them to take advantage of procedural generation. In this work, we empirically evaluate this tool by asking five users to complete three thinkaloud tasks and a post-task survey. We found that users found the tool fun to use and that it made map design easier. We discuss takeaways and how this tool and similar tools can be made better.

Reflection plays a critical role in learning by encouraging students to contemplate their knowledge and previous learning experiences to inform their future actions and higher-order thinking, such as reasoning and problem solving. Reflection is particularly important in inquiry-driven learning scenarios where students have the freedom to set goals and regulate their own learning. However, despite the importance of reflection in learning, there are significant theoretical, methodological, and analytical challenges posed by measuring, modeling, and supporting reflection. This paper presents results from a classroom study to investigate middle-school students’ reflection during inquiry-driven learning with Crystal Island, a game-based learning environment for middle-school microbiology. To collect evidence of reflection during game-based learning, we used embedded reflection prompts to elicit written reflections during students’ interactions with Crystal Island. Results from analysis of data from 105 students highlight relationships between features of students’ reflections and learning outcomes related to both science content knowledge and problem solving. We consider implications for building adaptive support in game-based learning environments to foster deep reflection and enhance learning, and we identify key features in students’ problem-solving actions and reflections that are predictive of reflection depth. These findings present a foundation for providing adaptive support for reflection during game-based learning.